- syslog-ng - lists.balabit.hu

few questions about patterndb
by Hendrik Pahl 25 Jun '10

25 Jun '10

Hi folks, i'm currently testing the patterndb with syslog-ng. After a few test runs,there are some open questions. Will there be some future releases of the patterndb by balabit? Last Snapshot has a timestamp from sep. 2009. Or is there a community somewhere out there, that i haven't seen yet?:-) I tested the classification into different logfiles (system,violation,unknown) with 1.5M lines of logfiles, and there was just one rule matching 'em all: 734adfc8-8a4d-40ad-923d-9d3fbabda34b, which is something about a newly connected usb mass storage device. This behaviour can't be correct? The patternfile "system-apache2.xml" still containts just plain messages, no XML anywhere - i thougt this should have been fixed yet? After some testing with logging messages to file like "messages-system/messages-violation/messages-unknown" i'm just asking myself about the profit i get when using patterndb. With a standard syslog-ng i usually can log things into files divided by the priority of the messages - that's all i want it to do, just have a logfile where all errors und failures are logged, just to get the pure number of lines down. When controlling a logfile i usually don't really care about a cronjob to be successful. What is the profit of using patterndb right there? Thanks in advance, i.A. Hendrik Pahl System Engineering team! datentechnik GmbH & Co.KG Werner von Siemens Straße 12a 49124 Georgsmarienhuette Tel.: +49 (0)5401-8226-50 Fax : +49 (0)5401-8226-55 E-Mail: pahl(a)team-datentechnik.de Internet: www.team-datentechnik.de HRA 110397, Amtsgericht Osnabrück Geschäftsführung: Reemt Lükenga Diese E-Mail enthält vertrauliche und/oder rechtlich geschützte Informationen. Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtümlich erhalten haben, informieren Sie bitte sofort den Absender und vernichten Sie diese E-Mail. Vielen Dank. Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser E-Mail ist nicht gestattet. This e-mail contains confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender and delete this message. Thank you. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.

5 8

proper filtering messages with java Exceptions
by Franciszek Zmarły 24 Jun '10

24 Jun '10

Hello I have a problem with filtering messages having java Exceptions. I send logs from many jvm processes to one syslog-ng server and save them to different files. Example one message from log4j (dump from wireshark) <139>Jun 18 10:53:03 td2/td2 [PROGRAM NAME - MainThread] ERROR - FATAL ERROR <139>Jun 18 10:53:03 td2/td2 java.lang.Exception: Invalid value TryCount: 0 <139>Jun 18 10:53:03 td2/td2 at com.td.Sender.initialize(Sender.java:30) <139>Jun 18 10:53:03 td2/td2 td2 at com.td.Main.main(Main.java:123) I use changed thread name (PROGRAM NAME) as filter in syslog-ng configuration filter f_td_p1 { match("PROGRAM NAME"); }; but in this configuration ony first line of message are saved (from log file) 2010-06-18 10:53:03 td2 [PROGRAM NAME-MainThread] ERROR - FATAL ERROR Is it possible to correct save all logs ? Of course all messages in examle is one log message. Log4j save it like this: 2010-06-18 10:53:03,242 [PROGRAM NAME-MainThread-MainThread] ERROR - FATAL ERROR java.lang.Exception: Invalid value TryCount: 0 at com.td.Sender.initialize(Sender.java:30) at com.td.Main.main(Main.java:123) I want to obtain same effect in syslog-ng. Have you any suggestion ? Please help. Regards Frank

4 4

FreeBSD 8 LOG_SECURITY and LOG_AUTH confused
by Brian A. Seklecki 23 Jun '10

23 Jun '10

Guys something shit the bed with 3.0.3 on 8.0-PL: ----- config ---------------- source src { unix-dgram("/var/run/log"); filter f_security { facility(security); }; filter f_auth { facility(auth); }; destination security { file("/var/log/security" template(cfi_template)); }; destination authlog { file("/var/log/auth.log" template(cfi_template)); }; log { source(src); filter(f_security); destination(security); flags(final); }; log { source(src); filter(f_auth); destination(authlog); flags(final); }; ------------------------- # /usr/local/sbin/syslog-ng -p /var/run/syslog.pid -tevdF Syslog connection established; fd='5', server='AF_INET(192.168.128.XXX:514)', local='AF_INET(0.0.0.0:0)' Running application hooks; hook='1' Running application hooks; hook='3' syslog-ng starting up; version='3.0.3' ^Z [5]+ Stopped /usr/local/sbin/syslog-ng -p /var/run/syslog.pid -tevdF [root@cfi-db-corp ~]# bg [root@cfi-db-corp ~]# echo test | logger -p auth.info Incoming log entry; line='<38>Mar 18 15:57:40 bseklecki: test' Filter rule evaluation begins; filter_rule='f_ams' Filter node evaluation result; filter_result='not-match', filter_type='level' Filter node evaluation result; filter_result='not-match', filter_type='AND' Filter node evaluation result; filter_result='not-match', filter_type='AND' Filter rule evaluation result; filter_result='not-match', filter_rule='f_ams' Filter rule evaluation begins; filter_rule='f_auth' Filter node evaluation result; filter_result='match', filter_type='facility' Filter rule evaluation result; filter_result='match', filter_rule='f_auth' Filter rule evaluation begins; filter_rule='f_ams' Filter node evaluation result; filter_result='not-match', filter_type='level' Filter node evaluation result; filter_result='not-match', filter_type='AND' Filter node evaluation result; filter_result='not-match', filter_type='AND' Filter rule evaluation result; filter_result='not-match', filter_rule='f_ams' Filter rule evaluation begins; filter_rule='f_authpriv' Filter node evaluation result; filter_result='not-match', filter_type='facility' Filter rule evaluation result; filter_result='not-match', filter_rule='f_authpriv' Filter rule evaluation begins; filter_rule='f_kern' Filter node evaluation result; filter_result='not-match', filter_type='facility' Filter rule evaluation result; filter_result='not-match', filter_rule='f_kern' Filter rule evaluation begins; filter_rule='f_user' Filter node evaluation result; filter_result='not-match', filter_type='facility' Filter rule evaluation result; filter_result='not-match', filter_rule='f_user' Filter rule evaluation begins; filter_rule='f_mail' Filter node evaluation result; filter_result='not-match', filter_type='facility' Filter rule evaluation result; filter_result='not-match', filter_rule='f_mail' Filter rule evaluation begins; filter_rule='f_security' Filter node evaluation result; filter_result='match', filter_type='facility' Filter rule evaluation result; filter_result='match', filter_rule='f_security' Initializing destination file writer; template='/var/log/security', filename='/var/log/security' The two fixes I can find are: - Move: "log { source(src); filter(f_security);.." below "log { source(src); filter(f_auth);..." ...wait, what?! - Remove reference to LOG_SECURITY /usr/include/syslog.h defines for Facility->index mappings haven't changed since RELENG_6, so I'm not sure what to make of this. Very strange, ~BAS

2 2

lists.balabit.com / lists.balabit.hu SSL Cert
by Brian A. Seklecki 23 Jun '10

23 Jun '10

All: The list headers have balabit.hu, but the SSL cert is for *.com. Sounds like a nightmare. I suspect Mailman domain name change is a nightmare. Maybe a good chance to experiment with RFC3546 / SNI (SSL Vhosting) :) ~BAS

1 0

Syslog-ng 3.0.5 TLS support
by Prashant Vaidya 16 Jun '10

16 Jun '10

Hi Guys, I think I'm at the right forum and asking right question:- I have installed Syslog-ng 3.0.5 on my linux box. Can some body please guide me how I can use RFC 5426 (UDP support) and RFC5425 (TLS encryption support) between two servers. My current setup is having 4 linux mahcines, I want to configure and use the TLS feature. Can some body please answer to my quaries:- 1:- What could be the corrective steps to test whether TLS is working or not. 2. So I have to configure Syslog-ng with TCP wrapper, SSL , IPV6 and spoofing support. I would be very grateful If someone can answer to above question. Regards, Prashva

3 3

assigning structured data from IETF syslog messages to macros
by Risto Vaarandi 14 Jun '10

14 Jun '10

hi all, I have run into an issue when trying to process structured data from IETF syslog messages. According to syslog-ng administrators guide (I am using syslog-ng 3.0.7), the SDATA.SDID.SDNAME macro can be used for referring structured data. However, when I tried to test the example from the administrator guide, I found that only the $SDATA macro works fine. The example is somewhat confusing, since the SDATA.EXAMPLE.EVENTSOURCE macro allegedly points to the SD ID "exampleSDID@0" (which contains the name eventSource). In order to test syslog-ng with a clearer case, I created the following messages with nc: echo -n '<165>1 2010-06-09T18:00:00.000000+02:00 192.168.1.1 myproc - - [timeQuality tzKnown="0" isSynced="0"] This is a test' | nc -w 1 -u localhost 8514 The syslog-ng configuration looks like this: template format { template("$DATE $HOST [$FACILITY.$LEVEL] $MSG StructuredData = ${SDATA.TIMEQUALITY.TZKNOWN}\n"); }; #template format { template("$DATE $HOST [$FACILITY.$LEVEL] $MSG StructuredData = ${SDATA}\n"); }; source ietfmsg { syslog(ip(0.0.0.0) port(8514) transport(udp)); }; destination netlog { file("/var/log/syslog-ng.log" template(format)); }; log { source(ietfmsg); destination(netlog); }; If the second template definition is active (i.e., the first definition is commented out), everything works fine and I have the following messages in the log: Jun 9 18:00:00 localhost [local4.notice] This is a test StructuredData = [timeQuality tzKnown="0" isSynced="0"] However, when the first template definition is activated, I will get the following output from syslog-ng: Incoming log entry; line='<165>1 2010-06-09T18:00:00.000000+02:00 192.168.1.1 myproc - - [timeQuality tzKnown="0" isSynced="0"] This is a test' No such value known; value='SDATA.TIMEQUALITY.TZKNOWN' Also, the log messages look like this: Jun 9 18:00:00 localhost [local4.notice] This is a test StructuredData = What am I doing wrong? The example from the manual does not seem to work, neither does my simple test case. kind regards, risto

2 2

syslog-ng and network problem
by Giovanni Mancuso 11 Jun '10

11 Jun '10

Hello guys I've installed two syslog-ng server instances behind a load balancer (LB) and configured N syslog-ng clients pointing to my virtual ip address. The clients doesn't send too much messages to the servers at this time. For some reason the LB closes the connections and when my client have to send a message to the server it realize that the connection was closed and it reopen a new one. The problem is that the messages even if the connection is stablished once again doesn't arrive to the server. I tryed also to enable the mark_freq(10) option but nothing changed. It starts to work if I restart the client or if I generate a lot of messages ( echo "MESSAGE" >> FILE.log ) I realize that the problem is in the network, but is someone have another idea to workaround this issue i'll be thankfull! Here is the config of one of my clients: @version: 3.0 #Default configuration file for syslog-ng. # # For a description of syslog-ng configuration file directives, please read # the syslog-ng Administrator's guide at: # # http://www.balabit.com/dl/html/syslog-ng-admin-guide_en.html/bk01-toc.html ######### # options options { time_reopen(10); keep_hostname(yes); log_msg_size(65535); flush_lines(0); }; ######### # sources source s_balabit { internal(); }; source s_source { file("FILE.log"); }; filter f_filter { match ("OUTGOING" value("MESSAGE")); }; ######### # destinations destination d_internal { file("/var/log/balabit.log"); }; destination d_network { tcp("vip" port(10001) tls( ca_dir("/etc/ssl/syslog-ng/ca.d")) log_fifo_size(4194304) ); }; ######### # logs log { source(s_source); filter(f_filter); destination(d_network);}; log { source(s_balabit); destination(d_internal); };

2 2

syslog-ng 3.0.7 has been released
by devel＠balabit.hu 11 Jun '10

11 Jun '10

------------------------------------------------------------------------------ PACKAGE : syslog-ng VERSION : 3.0.7 SUMMARY : new stable release DATE : Jun 7, 2010 ------------------------------------------------------------------------------ DESCRIPTION: A new stable version of syslog-ng Open Source Edition (3.0.7) has been released. For latest fixes in the 3.0.x branch you are recommended to upgrade to this version. CHANGES: 3.0.7 Sun, 06 Jun 2010 13:27:38 +0200 Bugfixes: * When syslog-ng is reloaded, the local hostname value was not refreshed, causing syslog-ng to remember the hostname until the next restart. This may not play nice with DHCP configured hostnames, which may change dynamically. * When several SIGHUP signals are received in quick succession, the last one may have been dropped. This problem is fixed. * Make sure that numbers never get resolved using getpwnam/getgrnam (e.g. libnss functions), because this may cause deadlocks if the NSS provider is LDAP and the LDAP server is trying to log a message to syslog about invalid usernames. * Fixed flush_timeout() handling in for non-file destinations. Enabling flush_lines()/flush_timeout() for a non-file destination, either using a global option, or a per-destination option could cause excess CPU usage as long as the destination wasn't writable. Other changes: * Introduced a new, cleaner syntax to specify that the permissions of a file are not to be changed. Use owner/group/perm options of a file without any parameters instead of "-1". The old mechanism had the problem that both per-file and global settings had to use "-1" to effectively stop syslog-ng from changing the uid/gid/perm values. The new syntax will make it possible to specify defaults in the global options section and customize those on a per-file basis. * syslog-ng will not go into the background if either the --syntax-only or --debug options are specified. DOWNLOAD: You can download the source or binary packages from: http://www.balabit.com/network-security/syslog-ng/opensource-logging-system…

1 0

Message uniqueness
by Dragan Zubac 09 Jun '10

09 Jun '10

Hello I'm using syslog-ng to log messages into PostgreSQL via file logging. I have a problem,because syslog-ng actually sends two messages from log file,instead one at a time. In attachment,You may see two messages that has the same ID number and were sent together,as 'one' message into database procedure. Is it possible to configure syslog-ng to send one message at a time only ? Sincerely Dragan Zubac

3 4

[Bug 85] New: wrong $_YEAR after logrotate
by bugzilla＠bugzilla.balabit.com 09 Jun '10

09 Jun '10

https://bugzilla.balabit.com/show_bug.cgi?id=85 Summary: wrong $_YEAR after logrotate Product: syslog-ng Version: 2.0.x Platform: PC OS/Version: Linux Status: NEW Severity: major Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: cedric.jeanneret(a)camptocamp.com Type of the Report: --- Estimated Hours: 0.0 Hello, We're using syslog-ng on debian lenny. We use macro like this : destination df_haproxy { file("/srv/syslog/archive/haproxy/$S_YEAR-$S_MONTH-$S_DAY-haproxy.log" create_dirs(yes) group(toto) ); }; We see there's an error when date changes, syslog recreate a new file with the wrong year (i.e. one year too old...) This happens each 1st of month: -rw-r----- 1 root toto 15782746 2010-05-31 23:59 2009-06-01-haproxy.log -rw-r----- 1 root toto 1756029229 2010-06-01 23:39 2010-06-01-haproxy.log More over, syslog-ng seems to use the wrong file for about 20 minutes, then "see" it's the wrong one, and create the good one just later... As said, this behavior is seen only on the 1st of each month... It's really strange. I have to confess, I really don't know if it's a problem for debian, or upstream. Here are informations about version and package : apt-cache show syslog-ng Package: syslog-ng Priority: extra Section: admin Installed-Size: 448 Maintainer: SZALAY Attila <sasa(a)debian.org> Architecture: amd64 Version: 2.0.9-4.1 If you think it's a debian problem, let me know so that I'll open a bug on debian's tracker. Thank you. Best regards, C. Jeanneret -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 2