- syslog-ng - lists.balabit.hu

no compatibility between syslog and mysql log priority more
by gatfi sami 04 Jun '09

04 Jun '09

Hi i a successfully configured a centralised syslog server using syslog-ng 2.09 here is the structure: syslog-client(read mysql error log) -------> *send* to a syslog-relay --------->* forword* to a syslog server ----> centralise in a mysql tabe 'logs' CREATE TABLE `syslogtopnet`.`log` ( `seq` int(10) unsigned NOT NULL auto_increment, `host` varchar(60) NOT NULL, `facility` varchar(40) NOT NULL, `*priority*` varchar(40) NOT NULL, `*level*` varchar(40) NOT NULL, `date_log` date NOT NULL, `time_log` time NOT NULL, `program` varchar(40) NOT NULL, `*message*` text NOT NULL, PRIMARY KEY (`seq`), KEY `index_log` (`host`) ) ENGINE=InnoDB AUTO_INCREMENT=684 DEFAULT CHARSET=latin1; my problème is : i find for exemple priority ='*Notice*' and level='*Notic*e' but in the message i find *[warn]* here is an exemple: INSERT INTO `syslogtopnet`.`log` VALUES (320,'192.168.9.195','user',* 'notice'*,*'notice'*,'2009-05-19','19:33:24','','[Tue May 19 19:33:24 2009] *[warn]* Init: Session Cache is not configured [hint: SSLSessionCache]'); INSERT INTO `syslogtopnet`.`log` VALUES (24445,'192.168.9.197','user','notice','notice','2009-06-04','17:21:02','090604','090604 17:21:02 mysqld started'), (24446,'192.168.9.197','user','notice','notice','2009-06-04','17:21:02','090604','090604 17:21:02 mysqld started'), (24447,'192.168.9.197','user',*'notice'*,*'notice'*,'2009-06-04','17:21:02','InnoDB','InnoDB: Unable to lock ./ibdata1, *error: 11*'), i discovered that apache log level are different then syslog's so !!!!! 1 wired isnt it lease is there a solution to keep the same level in the original priority? 2 beside in more than 20589 line in my table priority and level have the same values. is it normal that the level and priority are the same how can this be? thank you

1 0

Beta-testers needed for documentation draft
by Robert Fekete 03 Jun '09

03 Jun '09

Dear syslog-ng users, I am Robert Fekete, the writer of the syslog-ng Administrator Guide (and of the documentation of other BalaBit products as well). First of all, thank you very much for all of you who took the time to send me feedback and corrections about the admin guide. In the last few days, I have prepared an update for the admin guide that covers the news of the recent syslog-ng 3.0.2 OSE release, and also includes documentation for the pattern database feature. However, it seems that currently Bazsi and the other developers and testers will not have time to review the docs for a few weeks. So if you have some time and would line to test the pattern database, the draft is available at http://people.balabit.hu/frobert/syslog-ng-v3.0-guide-admin-en-6ed-draft.pdf If you find anything that is unclear, not accurate, or otherwise wrong (especially in the config examples), please let me know either here on the list, or via the documentation(a)balabit.com address. Among others, the following sections need review: Section 2.12, “Classifying log messages” (p. 15), Section 3.9, “Classifying messages” (p. 61), and Section 8.6.2, “Pattern databases” (p. 201). Thank you very much for your help. Regards, Robert Fekete

2 1

syslog-ng 3.0.2 issues: Extraneous colon (:) in forwarded log; two instances of syslog-ng
by Marvin Nipper 03 Jun '09

03 Jun '09

Hi. I'm been running syslog-ng 2.0.9 for eons, and wanted to get 3.0.2 in place, but have run into two issues. First, the prerequisite info: This is on Solaris 10 x86, and I'm running the eventlog-0.2.9 component. It has all compiled cleanly. I'm using the same script to initialize syslog-ng as I was using before. In the conf file, added in some value() parameters on my filters, in accordance with the 3.0 "warnings", and added in the @version directive, but other than that, it's essentially the same (reasonably simple) conf file I was using with my 2.0.9 version of the code. For reference, I've appended my conf file at the end of the email. "Generally" syslog-ng, seems to start-up fine, and does feed the output files that I've indicated in the conf file. If I didn't "look too closely", I would have thought it was all fine, but... My problems: As part of my test environment, I'm forwarding data from another (production 2.0.9) server, to my test server, so that I can see how things are working. Here are some (truncated) messages, showing both the original message (as it looks in the output file on the 2.0.9 server), and the second message being what gets written to the file by the 3.0.2 component: Original: May 29 09:15:05 10.132.240.25 id=firewall sn=0006B10B2E88 <truncated> Received and written by 3.0.2: May 29 09:15:05 10.132.240.25 id=firewall: sn=0006B10B2E88 <truncated> So the "bug" (I assume it to be a bug) is that "colon" that is now sitting just behind the word "firewall". When my test server is running a 2.0.9 daemon, these two lines match, but the 3.0.2 daemon has inserted that colon. I can only assume that this is tied to a bug in the filtering activity, because if you look at my conf file, you will see these lines: filter SonicWallNoise { match("id=firewall" value("MSG")) and filter(SonicWallMsgs); }; Bottom line, I am doing a filter that happens to "match on" that exact "firewall" string, and it seems like more than just coincidence that this is the exact location where that unwanted colon now appears. It seems like that filtering process has somehow injected that character into the output. (Maybe I'm just missing some "new functionality or feature" in the 3.0 docs, that would intentionally cause this?) Anyway... I'm obviously looking for any input as to why this is different, and/or how to fix it (or even just work-around the problem until it is fixed). And secondly, as my subject line implies, when I run my "/etc/init.d/syslog-ng start" scripting (the same one I've always used), I find, right away, that I now have two instances of syslog-ng running. I saw a bug report from back in 2006, mentioning a similar behavior that was related to (I think) syslog-ng restarting itself, but that was supposedly fixed. I'm ass-u-me-ing that something is now causing the daemon to restart itself, but again, when I run my 2.0.9 code on place of the 3.0.2 code, I get just the one copy running. Again, I'm looking for any guidance as to something that might be driving that behavior (and as I said, maybe it's just something stupid on my part, because I'm missing something "new" in the 3.0 behavior). Thanks, in advance, for any help and/or guidance as to what these issues might be. I'm glad to provide further information if it will help. Marvin Nipper syslog-ng.conf: @version:3.0 options { dir_perm(0755); perm(0600); chain_hostnames(no); keep_hostname(yes); log_fifo_size(1000); dns_cache_size(5000); dns_cache_expire(86400); dns_cache_expire_failed(86400); }; source any_udp { udp(); }; source any_tcp { tcp(port(601)); }; destination SEC {pipe("/tmp/sec"); }; destination routers_log { file("/var/adm/log/routers.log" create_dirs(yes)); }; destination ravlin_log { file("/var/adm/log/ravlin.log" create_dirs(yes)); }; destination windows_log { file("/var/adm/log/windows.log" create_dirs(yes)); }; destination workstation_log { file("/var/adm/log/workstation.log" create_dirs(yes)); }; destination catch-all_log { file("/var/adm/log/catch-all.log" create_dirs(yes)); }; destination test { file("/var/adm/log/test.log" create_dirs(yes)); }; destination dev_null {}; filter f_4 { facility(syslog) and level(info..emerg); }; log { source(any_udp); filter(f_4); destination(windows_log); flags(final); }; filter f_1 { facility(local5) and level(debug..emerg); }; log { source(any_udp); filter(f_1); destination(routers_log); flags(final); }; filter SonicWallNoise { match("id=firewall" value("MSG")) and filter(SonicWallMsgs); }; filter SonicWallMsgs { match("m=97" value("MSG")) or match("m=98" value("MSG")) or match("m=537" value("MSG")); }; log { source(any_udp); filter(SonicWallNoise); destination(dev_null); flags(final); }; filter f_3 { facility(local0) and level(debug..emerg); }; log { source(any_udp); filter(f_3); destination(ravlin_log); flags(final); }; log { source(any_tcp); destination(workstation_log); flags(final); }; log { source(any_udp); destination(catch-all_log); flags(final); }; This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.

2 2

Help on trying to build syslog-ng 3.0.2 on RHEL AS 4
by Joe Wood 03 Jun '09

03 Jun '09

Hello. I'm having troubles getting syslog-ng to compile on RHEL AS 4, with all appropriate libs installed. The configure script errors out with: checking pkg-config is at least version 0.14... yes checking for GLIB - version >= 2.10.1... no *** Could not run GLIB test program, checking why... *** The test program failed to compile or link. See the file config.log for the *** exact error that occured. This usually means GLIB is incorrectly installed. configure: error: Cannot find GLIB version >= 2.10.1: is pkg-config in path? When I examine config.log, I see the following: configure:8496: gcc -o conftest -g -O2 -Wall conftest.c >&5 conftest.c:42:18: glib.h: No such file or directory conftest.c: In function `main': conftest.c:48: error: `glib_major_version' undeclared (first use in this function) conftest.c:48: error: (Each undeclared identifier is reported only once conftest.c:48: error: for each function it appears in.) conftest.c:48: error: `glib_minor_version' undeclared (first use in this function) conftest.c:48: error: `glib_micro_version' undeclared (first use in this function) configure:8502: $? = 1 configure: failed program was: However, glib.h and all appropriate glib headers are installed in /usr/local/include, which even when specified to configure (through --includedir) it doesn't seem to pick up on.... Anyone encountered this before? Joe

2 1

intentionally dropping messages
by Jan Schaumann 03 Jun '09

03 Jun '09

Hello, I'm trying to drop a certain set of messages. Following http://www.balabit.com/dl/html/syslog-ng-admin-guide_en.html/ch07s08.html I notice that if I do *not* include a 'destination' in the 'log' statement, then a very large number of messages that should *not* be dropped are in fact dropped. As soon as I add a 'destination' to the log statement, it filters the correct messages. That is: destination d_dev_null { file("/dev/null" perm(0666) ); }; filter demo_debugfilter { (level(debug..notice) and facility(local0); }; log { source(s_udp); filter(demo_debugfilter); flags(final); }; Appears to falsely drop a number of messages that were of local1.info. As soon as I change the 'log' directive to be: log { source(s_udp); filter(demo_debugfilter); destination(d_dev_null); flags(final); }; it correctly filters only level0.{debug,info,notice}. This happens on a host with a large number of messages per second, but not on a host that receives very little traffic. I can't make much sense of this and was hoping somebody on this list could... -Jan

2 2

Re: [syslog-ng] db-parser issues
by Jakub Jankowski 03 Jun '09

03 Jun '09

On 2009-06-02, Robert Fekete wrote: > I think the problem is that you define a pattern in your xml for the program > as well: >> <program name='logger'> >> <pattern>param</pattern> > The <pattern> tag overrides the <program name>, so instead of logger, > syslog-ng thinks this rule applies to messages that have "param" in their > PROGRAM field. In addition to what you've quoted above, I also tried: 1. <program name='logger'> <pattern>logger</pattern> 2. <program name='somethingirrelevant'> <pattern>logger</pattern> and even without <pattern>: 3. <program name='logger'> All of these give me the same result - no match. Any other suggestions? Cheers, Jakub. -- Jakub Jankowski|shasta@toxcorp.com|http://toxcorp.com/ GPG: FCBF F03D 9ADB B768 8B92 BB52 0341 9037 A875 942D

2 1

Roadmap for syslog-ng OSE 4.0
by Balazs Scheidler 03 Jun '09

03 Jun '09

Dear syslog-ng users, The roadmap for syslog-ng OSE has been updated and published at: http://www.balabit.com/network-security/syslog-ng/opensource-logging-system… The same page also describes important release policy changes, thus it is recommended reading for all syslog-ng users. A related blog post can be found here: http://bazsi.blogs.balabit.com/2009/05/syslog-ng-40-roadmap-plus-release.ht… -- Bazsi

1 0

plz help
by dreamywoman 02 Jun '09

02 Jun '09

Hi All last week i have installed php-syslog-ng-2.5.1 and it works and i can see all logs in my web but after week i have decided to change the version of php-syslog-ng to php-syslog-ng-2.9.8j.tgz but after web installation i decided to change the perivious version (php-syslog-ng-2.5.1 ) after that i have problems i can get logs in other file like /var/log/catchall but i cant get anything in my php-syslog-ng in my web i use tcpdump i can get logs but i cant see anything in my web what should i do? source s_remote { unix-stream("/dev/log"); internal(); udp(); }; destination d_mysql { pipe("/tmp/mysql.pipe " template("INSERT INTO logs (host, facility, priority, level, tag, date, time, program, msg) VALUES ('$HOST', '$FACILITY', '$PRIORITY', '$LEVEL', '$TAG', '$YEAR-$MONTH-$DAY', '$HOUR:$MIN:$SEC', '$PROGRAM', '$MSG' );\n") template-escape(yes)); }; log { source(s_remote); destination(d_mysql); }; what is my problem in installation?

1 0

iptables logging in a separate logfile works not properly
by Ralf Heidenreich 02 Jun '09

02 Jun '09

Hello, I have the following line in iptables: iptables -A INPUT -j LOG --log-prefix "INPUT: " In syslog-ng (/etc/syslog-ng/syslog-ng.conf) I have the following config: ############################################# options { sync (0); time_reopen (10); log_fifo_size (1000); long_hostnames (off); use_dns (no); use_fqdn (no); create_dirs (no); keep_hostname (yes); }; source s_sys { file ("/proc/kmsg" log_prefix("kernel: ")); unix-stream ("/dev/log"); internal(); # udp(ip(0.0.0.0) port(514)); }; destination d_cons { file("/dev/console"); }; destination d_mesg { file("/var/log/messages"); }; destination d_auth { file("/var/log/secure"); }; destination d_mail { file("/var/log/maillog" sync(10)); }; destination d_spol { file("/var/log/spooler"); }; destination d_boot { file("/var/log/boot.log"); }; destination d_cron { file("/var/log/cron"); }; destination d_kern { file("/var/log/messages"); }; destination d_mlal { usertty("*"); }; # direct iptables logs into /var/log/firewall.log destination iptables { file("/var/log/firewall.log"); }; filter iptables { match("INPUT:"); }; log { source(s_sys); filter(iptables); destination(iptables); }; filter noiptables { not match("INPUT:"); }; log { source(s_sys); filter(noiptables); destination(d_kern); }; filter f_kernel { facility(kern); }; filter f_default { level(info..emerg) and not (facility(mail) or facility(authpriv) or facility(cron)); }; filter f_auth { facility(authpriv); }; filter f_mail { facility(mail); }; filter f_emergency { level(emerg); }; filter f_news { facility(uucp) or (facility(news) and level(crit..emerg)); }; filter f_boot { facility(local7); }; filter f_cron { facility(cron); }; #log { source(s_sys); filter(f_kernel); destination(d_cons); }; log { source(s_sys); filter(f_kernel); destination(d_kern); }; log { source(s_sys); filter(f_default); destination(d_mesg); }; log { source(s_sys); filter(f_auth); destination(d_auth); }; log { source(s_sys); filter(f_mail); destination(d_mail); }; log { source(s_sys); filter(f_emergency); destination(d_mlal); }; log { source(s_sys); filter(f_news); destination(d_spol); }; log { source(s_sys); filter(f_boot); destination(d_boot); }; log { source(s_sys); filter(f_cron); destination(d_cron); }; ################################################################ iptables loggs to /var/log/firewall.log, but also to /var/log/messages. A line in messages is: May 29 14:13:55 hostname kernel: INPUT: IN=eth0 OUT= MAC=00:16:17:6b:34:bf:00:02:85:12:c9:40:08:00 SRC=source-ip DST=server-ip LEN=104 TOS=0x00 PREC=0x00 TTL=56 ID=38612 PROTO=TCP SPT=26787 DPT=122 WINDOW=1032 RES=0x00 ACK PSH URGP=0 What I am doing wrong. with kind regards Ralf

2 3

rotate logs in recursive directories
by Reaky Rok 01 Jun '09

01 Jun '09

Hi friends I have a problem with syslog-ng in my server, I setup it successfuly in RHEL5 and It waorking without any problems, I make it a central log server for all servers in my company, The problem is that I orgnized it for each server bu date and facility like that $HOST/$YEAR.$MONTH.$DAY/$FACILITY.log which makes a lot of recursive directory for each server, Now I'm facing a problem to rotate it as I don't know how to make the logrotate rotate all this files in the recursive directories, Could you help me please, This is my /etc/logrotate.d/syslog-ng /var/log/syslog-ng { daily rotate 30 compress missingok notifempty create 0600 root root } Thanks & Best Regards Bassam Muhammad _________________________________________________________________ Windows Live™: Keep your life in sync. Check it out! http://windowslive.com/explore?ocid=TXT_TAGLM_WL_t1_allup_explore_012009

1 0

db-parser issues
by Jakub Jankowski 29 May '09

29 May '09

Hi, I'm having hard time getting db-parser functionality to work. This is my syslog-ng config (omitted irrelevant stuff; some lines may be split due to line wrappings in my MUA): @version: 3.0 source s_sys { file ("/proc/kmsg"); unix-stream ("/dev/log" flags(no-multi-line)); internal(); }; destination d_test { file("/var/log/test.log" template("$HOUR:$MIN:$SEC $TZ $HOST [$LEVEL] ${.classifier.class} ${MSGONLY} ${FOO.BAR}\n") template_escape(no)); }; parser p_test { db-parser(file(/etc/syslog-ng/test.xml)); }; filter f_test { program("logger" type(string)); }; log { source(s_sys); filter(f_test); parser(p_test); destination(d_test); }; My pattern db (test.xml) is as simple as: <patterndb version='1' pub_date='2009-05-16'> <program name='logger'> <pattern>param</pattern> <rules> <rule provider='tester' id='666' class='security'> <description>Test rule</description> <pattern>param1=@IPV4:FOO.BAR@</pattern> </rule> </rules> </program> </patterndb> Syslog-ng has no trouble loading it on startup, as suggested by these lines in /var/log/messages: May 30 00:17:37 atest1 syslog-ng[8681]: Log pattern database reloaded; file='/etc/syslog-ng/test.xml', version='1', pub_date='2009-05-16' May 30 00:17:37 atest1 syslog-ng[8681]: syslog-ng starting up; version='3.0.2' I'm testing db-parser, by issuing: $ echo param1=1.2.3.4 param2=abcde | logger -i $ And this is what I get in /var/log/test.log: 00:17:38 +02:00 atest1 [notice] unknown param1=1.2.3.4 param2=abcde Clearly not what I wanted. Seems like my parser doesn't match anything, as ${FOO.BAR} is empty. Can you shed some light on my problem? Am I doing something obviously wrong? Db-parser functionality is neat, but it lacks documentation - I base on Balázs' presentation and some blog posts only. Thanks in advance, Jakub -- Jakub Jankowski|shasta@toxcorp.com|http://toxcorp.com/ GPG: FCBF F03D 9ADB B768 8B92 BB52 0341 9037 A875 942D

1 0

Re: [syslog-ng] syslog-ng 3.0.2 issues: Extraneous colon (:) in forwarded log; two instances of syslog-ng
by Marvin Nipper 29 May '09

29 May '09

Sorry for a second message, but I've just noticed a third issue with 3.0.2. Same test scenario as already described. Here's a messaged logged on my production 2.0.9 server, and also forwarded to my test server (I've removed some of the trailing noise): May 29 16:15:09 fwmil01 [LOG_WARNING] ipsrd[213]: igmp_recv_leave_group: <truncated.....> When it arrives on my test server, and a 2.0.9 daemon catches it (and writes it to a file), it looks exactly as above, but when 3.0.2 catches it, this is the result: May 29 16:15:09 fwmil01 ipsrd[213]: igmp_recv_leave_group: <truncated.....> In short, the Severity information has been stripped. Again, in looking at the 3.0 docs, I'm not clear as to any change that would cause this to be stripped out. (It might not be that big a deal to lose that, except that there are subsequent filters in my system that are looking for that "translated severity" as part of their search.) Anyway, again, I would appreciate any input as to why 3.x would be doing this, and what configuration change might restore the original content. THANKS for your help. _____________________________________________ From: Marvin Nipper Sent: Friday, May 29, 2009 12:08 PM To: 'syslog-ng(a)lists.balabit.hu' Subject: syslog-ng 3.0.2 issues: Extraneous colon (:) in forwarded log; two instances of syslog-ng Hi. I'm been running syslog-ng 2.0.9 for eons, and wanted to get 3.0.2 in place, but have run into two issues. First, the prerequisite info: This is on Solaris 10 x86, and I'm running the eventlog-0.2.9 component. It has all compiled cleanly. I'm using the same script to initialize syslog-ng as I was using before. In the conf file, added in some value() parameters on my filters, in accordance with the 3.0 "warnings", and added in the @version directive, but other than that, it's essentially the same (reasonably simple) conf file I was using with my 2.0.9 version of the code. For reference, I've appended my conf file at the end of the email. "Generally" syslog-ng, seems to start-up fine, and does feed the output files that I've indicated in the conf file. If I didn't "look too closely", I would have thought it was all fine, but... My problems: As part of my test environment, I'm forwarding data from another (production 2.0.9) server, to my test server, so that I can see how things are working. Here are some (truncated) messages, showing both the original message (as it looks in the output file on the 2.0.9 server), and the second message being what gets written to the file by the 3.0.2 component: Original: May 29 09:15:05 10.132.240.25 id=firewall sn=0006B10B2E88 <truncated> Received and written by 3.0.2: May 29 09:15:05 10.132.240.25 id=firewall: sn=0006B10B2E88 <truncated> So the "bug" (I assume it to be a bug) is that "colon" that is now sitting just behind the word "firewall". When my test server is running a 2.0.9 daemon, these two lines match, but the 3.0.2 daemon has inserted that colon. I can only assume that this is tied to a bug in the filtering activity, because if you look at my conf file, you will see these lines: filter SonicWallNoise { match("id=firewall" value("MSG")) and filter(SonicWallMsgs); }; Bottom line, I am doing a filter that happens to "match on" that exact "firewall" string, and it seems like more than just coincidence that this is the exact location where that unwanted colon now appears. It seems like that filtering process has somehow injected that character into the output. (Maybe I'm just missing some "new functionality or feature" in the 3.0 docs, that would intentionally cause this?) Anyway... I'm obviously looking for any input as to why this is different, and/or how to fix it (or even just work-around the problem until it is fixed). And secondly, as my subject line implies, when I run my "/etc/init.d/syslog-ng start" scripting (the same one I've always used), I find, right away, that I now have two instances of syslog-ng running. I saw a bug report from back in 2006, mentioning a similar behavior that was related to (I think) syslog-ng restarting itself, but that was supposedly fixed. I'm ass-u-me-ing that something is now causing the daemon to restart itself, but again, when I run my 2.0.9 code on place of the 3.0.2 code, I get just the one copy running. Again, I'm looking for any guidance as to something that might be driving that behavior (and as I said, maybe it's just something stupid on my part, because I'm missing something "new" in the 3.0 behavior). Thanks, in advance, for any help and/or guidance as to what these issues might be. I'm glad to provide further information if it will help. Marvin Nipper syslog-ng.conf: @version:3.0 options { dir_perm(0755); perm(0600); chain_hostnames(no); keep_hostname(yes); log_fifo_size(1000); dns_cache_size(5000); dns_cache_expire(86400); dns_cache_expire_failed(86400); }; source any_udp { udp(); }; source any_tcp { tcp(port(601)); }; destination SEC {pipe("/tmp/sec"); }; destination routers_log { file("/var/adm/log/routers.log" create_dirs(yes)); }; destination ravlin_log { file("/var/adm/log/ravlin.log" create_dirs(yes)); }; destination windows_log { file("/var/adm/log/windows.log" create_dirs(yes)); }; destination workstation_log { file("/var/adm/log/workstation.log" create_dirs(yes)); }; destination catch-all_log { file("/var/adm/log/catch-all.log" create_dirs(yes)); }; destination test { file("/var/adm/log/test.log" create_dirs(yes)); }; destination dev_null {}; filter f_4 { facility(syslog) and level(info..emerg); }; log { source(any_udp); filter(f_4); destination(windows_log); flags(final); }; filter f_1 { facility(local5) and level(debug..emerg); }; log { source(any_udp); filter(f_1); destination(routers_log); flags(final); }; filter SonicWallNoise { match("id=firewall" value("MSG")) and filter(SonicWallMsgs); }; filter SonicWallMsgs { match("m=97" value("MSG")) or match("m=98" value("MSG")) or match("m=537" value("MSG")); }; log { source(any_udp); filter(SonicWallNoise); destination(dev_null); flags(final); }; filter f_3 { facility(local0) and level(debug..emerg); }; log { source(any_udp); filter(f_3); destination(ravlin_log); flags(final); }; log { source(any_tcp); destination(workstation_log); flags(final); }; log { source(any_udp); destination(catch-all_log); flags(final); }; This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.

1 0

RFC 5424 compliant syslog library?
by Daniel Mahu 28 May '09

28 May '09

Hello, I'm looking for a linux library that would implement the syslog functions using the new standard format in RFC 5424. Of all the new things in the standard, I'm particularly interested in the timestamp resolution - I need to log timestamps with fractions of a second. Already tried searching google to no avail (only found a NetBSD libc patch, but I need linux). Since syslog-ng can now parse these messages, I thought maybe someone here may have heard about it if such a library existed. thanks, Daniel

2 1

How to name dir or logfile with the time by the message timestamp itself
by Roc Zhou 22 May '09

22 May '09

Hi, I'm now have a syslog-ng to collect the syslog of 2 sites, 1 is in China and another is in US, the host *vip-syslog* is in China, and US send the the messages to *vip-syslog* through a special tunnel, (for reliability, I can build a proxy with disk buffering to relay the syslog messages). Now * vip-syslog* can recevied the messages and works well, but I still suffer a problem. This is the syslog-ng.conf: #cat /etc/syslog-ng/syslog-ng.conf # # configuration file for syslog-ng, customized for remote logging # source s_internal { internal(); }; destination d_syslognglog { file("/var/log/syslog-ng.log"); }; log { source(s_internal); destination(d_syslognglog); }; options { ts_format(rfc3164); chain_hostnames(no); use_dns(yes); dns_cache(yes); dns_cache_hosts(/etc/syslog-ng/hosts); use_fqdn(no); perm(0644); dir_perm(0700); flush_lines(128); flush_timeout(500); log_msg_size(16384); use_time_recvd(no); # recv_time_zone(-07:00); # send_time_zone(-07:00); }; # --------------------------------------------------------------------- # Local sources, filters and destinations are commented out # If you want to replace sysklogd simply uncomment the following # parts and disable sysklogd # # Local sources # source s_local { unix-dgram("/dev/log"); file("/proc/kmsg" log_prefix("kernel:")); }; # # --------------------------------------------------------------------- # Local filters # filter f_messages { level(info..emerg); }; filter f_secure { facility(authpriv); }; filter f_mail { facility(mail); }; filter f_cron { facility(cron); }; filter f_emerg { level(emerg); }; filter f_spooler { level(crit..emerg) and facility(uucp, news); }; filter f_local7 { facility(local7); }; # # --------------------------------------------------------------------- # Local destinations # destination d_messages { file("/var/log/messages"); }; destination d_secure { file("/var/log/secure"); }; destination d_maillog { file("/var/log/maillog"); }; destination d_cron { file("/var/log/cron"); }; destination d_console { usertty("root"); }; destination d_spooler { file("/var/log/spooler"); }; destination d_bootlog { file("/var/log/boot.log"); }; # # --------------------------------------------------------------------- # Local logs - order DOES matter ! # log { source(s_local); filter(f_emerg); destination(d_console); }; log { source(s_local); filter(f_secure); destination(d_secure); flags(final); }; log { source(s_local); filter(f_mail); destination(d_maillog); flags(final); }; log { source(s_local); filter(f_cron); destination(d_cron); flags(final); }; log { source(s_local); filter(f_spooler); destination(d_spooler); }; log { source(s_local); filter(f_local7); destination(d_bootlog); }; log { source(s_local); filter(f_messages); destination(d_messages); }; # --------------------------------------------------------------------- # Remote logging # # Remote sources # source s_remote { tcp(ip(0.0.0.0) port(514)); udp(ip(0.0.0.0) port(514)); }; # --------------------------------------------------------------------- # Remote destinations # template t_message { # template("$STAMP $HOST $PROGRAM[$PID] $MSG\n"); template_escape(no); template("$S_STAMP $HOST $MSG\n"); template_escape(no); }; template t_apache { template("$MSGONLY\n"); template_escape(no); }; # --------------------------------- # Global site with US timezone: destination d_en_alarm { file("/var/log/syslog-ng/$S_YEAR-$S_MONTH-$S_DAY/$HOST/alarm" owner("root") group("root") perm(0640) dir_perm(0750) create_dirs(yes) time_zone(-07:00) template(t_message)); # program("sec.pl") }; destination d_en_necessary { file("/var/log/syslog-ng/$S_YEAR-$S_MONTH-$S_DAY/$HOST/necessary" owner("root") group("root") perm(0640) dir_perm(0750) create_dirs(yes) time_zone(-07:00) template(t_message)); }; destination d_en_cookie_log { file("/var/log/syslog-ng/$S_YEAR-$S_MONTH-$S_DAY/$HOST/cookie_log" owner("root") group("root") perm(0640) dir_perm(0750) create_dirs(yes) time_zone(-07:00) template(t_apache)); }; # --------------------------------- # China site with Beijing timezone: destination d_cn_alarm { file("/var/log/syslog-ng/$S_YEAR-$S_MONTH-$S_DAY/$HOST/alarm" owner("root") group("root") perm(0640) dir_perm(0750) create_dirs(yes) template(t_message)); }; destination d_cn_necessary { file("/var/log/syslog-ng/$S_YEAR-$S_MONTH-$S_DAY/$HOST/necessary" owner("root") group("root") perm(0640) dir_perm(0750) create_dirs(yes) template(t_message)); }; destination d_cn_cookie_log { file("/var/log/syslog-ng/$S_YEAR-$S_MONTH-$S_DAY/$HOST/cookie_log" owner("root") group("root") perm(0640) dir_perm(0750) create_dirs(yes) template(t_apache)); }; destination d_null { file("/dev/null"); }; destination d_sec { program("/etc/syslog-ng/sec.pl -input=\"-\" -conf=/etc/syslog-ng/sec.conf"); }; # --------------------------------------------------------------------- # Remote filters # filter f_en_host { host("(^hz|^us|^hk)_"); }; # # Used by data warehouse(dw): filter f_cn_local1 { facility(local1); }; filter f_en_local2 { facility(local2); }; filter f_en_dw { filter(f_en_host) and filter(f_en_local2); }; filter f_cn_dw { not filter(f_en_host) and filter(f_cn_local1); }; # Used by system admins and security admins: filter f_en_sys { filter(f_en_host) and not filter(f_en_local2); }; filter f_cn_sys { not filter(f_en_host) and not filter(f_cn_local1); }; filter f_necessary { match("necessary") or match("important"); ...... }; filter f_alarm { match("(?i)(error|fail)"); ...... }; ...... # --------------------------------------------------------------------- # Remote log path # log { source(s_remote); filter(f_en_sys); filter(f_necessary); destination(d_en_necessary); }; log { source(s_remote); filter(f_cn_sys); filter(f_necessary); destination(d_cn_necessary); }; log { source(s_remote); filter(f_en_sys); filter(f_alarm); destination(d_en_alarm); }; # log { source(s_remote); filter(f_en_sys); filter(f_necessary); destination(d_en_alarm); }; # --> This will not generate duplicated message! log { source(s_remote); filter(f_cn_sys); filter(f_alarm); destination(d_cn_alarm); }; log { source(s_remote); filter(f_cn_dw); destination(d_cn_cookie_log); }; log { source(s_remote); filter(f_en_dw); destination(d_en_cookie_log); }; log { source(s_remote); destination(d_null); }; NOW as you can see, I want the message to be logged to the dir: /var/log/syslog-ng/$S_YEAR-$S_MONTH-$S_DAY/$HOST/ The logs of China is OK, but the US is not since a timezone translating must be performed. Now the message timestamp in the logfile is right, but the dir name is not, for example: #head /var/log/syslog-ng/2009-05-22/us_search63/alarm May 21 10:50:27 us_search63 kernel: isupdate[7157]: segfault at 0000000000000010 rip 000000000041adad rsp 00007fffaa3cf560 error 4 May 21 18:09:55 us_search63 snmpd[26753]: /etc/snmp/snmpd.conf: line 433: Error: WARNING: This output format is being deprecated - Please use the 'extend' directive instead May 21 18:09:55 us_search63 snmpd[26753]: /etc/snmp/snmpd.conf: line 434: Error: WARNING: This output format is being deprecated - Please use the 'extend' directive instead May 21 18:09:55 us_search63 snmpd[26753]: /etc/snmp/snmpd.conf: line 435: Error: WARNING: This output format is being deprecated - Please use the 'extend' directive instead May 21 18:09:56 us_search63 snmpd[26753]: /etc/snmp/snmpd.conf: line 436: Error: WARNING: This output format is being deprecated - Please use the 'extend' directive instead As you can see, the dir of 2009-05-22 has the records of May 21, which is not what I want. I have tried the MACROS of $DAY, $S_DAY and $R_DAY, seems can not solve this problem. Any suggestions? Thanks.

1 0

Space in tag causes parsing problems - syslog-ng 2.0.8
by Fegan, Joe 21 May '09

21 May '09

Syslog-ng 2.0.8 I found that logging a message with a space in the tag causes the first word of the tag to be used as the hostname. This does not happen with the standard syslogd. Best illustrated by an example: [root@kudos8 ~]# logger -t "hello" "here is a message" [root@kudos8 ~]# tail -4 /var/log/messages May 18 16:43:53 kudos8 info snmpd[5296]: Connection from UDP: [127.0.0.1]:41757 May 18 16:43:53 kudos8 info snmpd[5296]: Connection from UDP: [127.0.0.1]:44965 May 18 16:43:53 kudos8 info snmpd[5296]: Received SNMP packet(s) from UDP: [127.0.0.1]:44965 May 18 16:43:55 kudos8 notice hello: this is a message [root@kudos8 ~]# This is expected behaviour. Now see what happens when I put a space in the tag: [root@kudos8 ~]# logger -t "hello world" "second message" [root@kudos8 ~]# tail -5 /var/log/messages May 18 16:43:53 kudos8 info snmpd[5296]: Connection from UDP: [127.0.0.1]:41757 May 18 16:43:53 kudos8 info snmpd[5296]: Connection from UDP: [127.0.0.1]:44965 May 18 16:43:53 kudos8 info snmpd[5296]: Received SNMP packet(s) from UDP: [127.0.0.1]:44965 May 18 16:43:55 kudos8 notice hello: this is a message May 18 16:44:04 hello notice world: second message [root@kudos8 ~]# See that "hello" was incorrectly used as the hostname. This is 100% reproducible. I stopped syslog-ng and started the standard syslogd instead. Standard syslogd does not have this problem: [root@kudos8 ~]# logger -t "hello" "here is a message" [root@kudos8 ~]# logger -t "hello world" "second message" [root@kudos8 ~]# service syslog-ng stop Stopping syslog-ng: [ OK ] [root@kudos8 ~]# service syslog start Starting system logger: [ OK ] Starting kernel logger: [ OK ] [root@kudos8 ~]# logger -t "hello world" "third message" [root@kudos8 ~]# tail -10 /var/log/messages May 18 16:54:25 kudos8 info snmpd[5296]: Connection from UDP: [127.0.0.1]:33587 May 18 16:54:25 kudos8 info snmpd[5296]: Connection from UDP: [127.0.0.1]:46177 May 18 16:54:25 kudos8 info snmpd[5296]: Received SNMP packet(s) from UDP: [127.0.0.1]:46177 May 18 16:54:30 kudos8 notice hello: here is a message May 18 16:54:31 hello notice world: second message May 18 16:54:34 kudos8 info syslog-ng[8737]: Termination requested via signal, terminating; May 18 16:54:34 kudos8 notice syslog-ng[8737]: syslog-ng shutting down; version=\'2.0.8\' May 18 16:54:38 kudos8 syslogd 1.4.1: restart. May 18 16:54:38 kudos8 kernel: klogd 1.4.1, log source = /proc/kmsg started. May 18 16:54:49 kudos8 hello world: third message [root@kudos8 ~]#

2 3

parsing netapp syslog (2)
by David - DCPC 21 May '09

21 May '09

Hello, I'm new on syslog-ng. I use v3.0.2 on a CentOS 4.4. I send syslog logs from a netapp FAS 2020 (*.*) to the syslog-ng server via udp. I have problem to analyse thoses logs. the lines have a different format than the standard syslog from another unix by example and are different than the BSD or IETF standard described in the syslog-ng documentation : (...) Tue May 19 11:27:30 CEST [netapp-L: wafl.quota.qtree.exceeded:notice]: tid 10: tree quota exceeded on volume linusers. Additional warnings will be suppressed for approximately 60 minutes or until a 'quota resize' is performed. (...) 1°) When i use it in syslog-ng i get this line in a sample extraction without filter : (...) May 19 11:27:30 netapp-b wafl.quota.qtree.exceeded:notice]: tid 10: tree quota exceeded on volume linusers. Additional warnings will be suppressed for approximately 60 minutes or until a 'quota resize' is performed. (...) (difference between host named should come from the renaming of hosts during tests, and dns aliases, don't think this is the problem). Is it possible to correct the processus that parse the original line ? it seems that the parser block can be used only for the $MESSAGE part of the line. 2°) It want to use other macro to make some filter and destination, like $PROGRAM. Is it possible to 'debug' the processing and have a details of macros and their contents, line by line ? Thx (sorry for the first miss) -- Salutations, David CHALON

2 1

NULL byte and Syslog-NG
by Jon Fredrickson 21 May '09

21 May '09

Hello, I am importing events from the Oracle sys.aud$ table via JDBC, then forwarding these events to a syslog-ng server. Every time a user makes an ODBC connection from a windows machine, a NULL byte appears directly after the host name in the USERHOST field. When syslog-ng gets this raw event it changes the NULL byte from 00 to a line feed 0a. Does anyone know how to stop this from happening, or allow syslog-ng to trim NULL characters? My syslog-ng version is 2.0.1 Thanks in advance, Jon

2 1

parsing netapp syslog
by David - DCPC 19 May '09

19 May '09

Hello, I'm new on syslog-ng. I use v3.0.2 on a CentOS 4.4. I send syslog logs from a netapp FAS 2020 (*.*) to the syslog-ng server via udp. I have problem to analyse thoses logs. the lines have a different format than the standard syslog from another unix by example. and different tha -- Salutations, David CHALON

1 0

Logging with db-parser issue
by Nate Hausrath 14 May '09

14 May '09

Hey everyone, Sorry to ask what is probably an easy question. I'm attempting to use the new db-parser feature in syslog-ng to parse Windows events generated by Snare. What I've attempted to do is write a simple XML file that will match one simple rule. This way I can test that everything is functioning properly and I can move on from there. Unfortunately, it's not working, so I'm unsure as to whether it is my rule or my installation. So I have a few basic questions that will hopefully clear up my confusion! - I did not install syslog-ng from source. I downloaded the .deb for version 3.01 and used that. Does this even include db-parser? (I'm assuming yes because it does not cause an error from the config file alone.) - My installation is in /opt/syslog-ng, so I've put the example XML files (http://www.balabit.com/downloads/files/patterndb/) in /opt/syslog-ng/var. Is this the right location? Other than that, here is my syslog-ng.conf file: @version: 3.0 #Default configuration file for syslog-ng. # # For a description of syslog-ng configuration file directives, please read # the syslog-ng Administrator's guide at: # # http://www.balabit.com/dl/html/syslog-ng-admin-guide_en.html/bk01-toc.html # options { }; ###### # sources source s_local { # message generated by Syslog-NG internal(); # standard Linux log source (this is the default place for the syslog() # function to send logs to) unix-stream("/dev/log"); # messages from the kernel file("/proc/kmsg" program_override("kernel: ")); }; source s_net { udp(); tcp(); }; parser p_db { db-parser(); }; ###### # destinations destination d_messages { file("/var/log/messages"); }; destination d_parsed { file("/var/log/remote/parsed.log"); }; log { source(s_local); destination(d_messages); }; log { source(s_net); destination(d_parsed); parser(p_db); }; And here is my windows.xml file: <?xml version='1.0' encoding='UTF-8'?> <patterndb version='2' pub_date='2009-05-07'> <ruleset name='windows'> <pattern>MSWinEventLog</pattern> <rules> <rule provider='nate' id='1' class='system'> <patterns> <pattern>540</pattern> </patterns> <description>This is a terrible terrible message to receive. Game over man! Game over!</description> </rule> </rules> </ruleset> </patterndb> As you see, to test I just want to match on the number '540' and put that log message in /var/log/remote/parsed.log. I have verified that messages that should match this are arriving at the machine when I send them. If i turn off the db-parser, the messages appear in the parsed.log file. If there is any documentation that explains the basics of this, sorry that I missed it. I've only been able to find some of the other messages on this mailing list and the blog entries on balabit. Thanks for any help! Nate

4 10

Compile error on Ubuntu 8.04
by Nate Hausrath 12 May '09

12 May '09

Hello everyone, I'm attempting to compile and install the latest syslog-ng 3.0 from source (downloaded from the git repository). I believe I have installed all the dependencies (configure does not complain). However, I am receiving the following error when running make. /bin/bash ../ylwrap cfg-lex.l .c cfg-lex.c -- /bin/bash /home/nhausrath/packages/syslog-ng.git/missing --run flex make[2]: *** [cfg-lex.c] Error 1 make[2]: Leaving directory `/home/nhausrath/packages/syslog-ng.git/src' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/home/nhausrath/packages/syslog-ng.git' make: *** [all] Error 2 It seems the error is in cfg-lex.l, but I can't tell what exactly is causing the problem. Of course, it may be something completely different that I am not familiar with. Thanks for any help! -Nate

3 4

Re: [syslog-ng] PATCH: Fix some small memory leaks and a SEGV
by mark tomlinson 10 May '09

10 May '09

On Fri, 2009-05-08 at 15:16 +0200, Balazs Scheidler wrote: > Hi, > > Thanks for your contribution. Although some of these might be genuine > leaks, they mostly occur of you use the same option multiple times in > the configuration file. > ... deletions ... > > > > diff --git a/src/dgroup.c b/src/dgroup.c > > index 7919a54..be02899 100644 > > --- a/src/dgroup.c > > +++ b/src/dgroup.c > > @@ -76,6 +76,16 @@ log_dest_group_deinit(LogPipe *s) > > NULL); > > success = FALSE; > > } > > + if (p->group) > > + { > > + g_free(p->group); > > + p->group = NULL; > > + } > > + if (p->id) > > + { > > + g_free(p->id); > > + p->id = NULL; > > + } > > } > > return success; > > } > > these should be freed when the driver in question gets freed in the > log_drv_free() function > This was a definite leak with our configuration and use. I believed the problem to be due to log_dest_group_deinit() being called, and then log_dest_group_init() being called, without log_drv_free() ever being called inbetween. I will have a look at our configuration files. We do change configuration between logging to a file and logging to/from network on the fly (and while many log messages are being generated). > > diff --git a/src/logsource.c b/src/logsource.c > > index 311e61f..93a8747 100644 > > --- a/src/logsource.c > > +++ b/src/logsource.c > > @@ -181,6 +181,10 @@ log_source_set_options(LogSource *self, LogSourceOptions *options, gint stats_le > > g_atomic_counter_set(&self->options->window_size, current_window); > > self->stats_level = stats_level; > > self->stats_source = stats_source; > > + if (self->stats_id) > > + g_free(self->stats_id); > > + if (self->stats_instance) > > + g_free(self->stats_instance); > > self->stats_id = stats_id ? g_strdup(stats_id) : NULL; > > self->stats_instance = stats_instance ? g_strdup(stats_instance): NULL; > > } > > > > > diff --git a/src/logwriter.c b/src/logwriter.c > > index 4c895cc..1d1e00e 100644 > > --- a/src/logwriter.c > > +++ b/src/logwriter.c > > @@ -792,6 +792,10 @@ log_writer_set_options(LogWriter *self, LogPipe *control, LogWriterOptions *opti > > > > self->stats_level = stats_level; > > self->stats_source = stats_source; > > + if (self->stats_id) > > + g_free(self->stats_id); > > + if (self->stats_instance) > > + g_free(self->stats_instance); > > self->stats_id = stats_id ? g_strdup(stats_id) : NULL; > > self->stats_instance = stats_instance ? g_strdup(stats_instance) : NULL; > > these again should only be called once, but better if the function is > idempotent. Again, I was seeing deinit()/init() happening, without any calls to free the driver from af_socket_dd. > > > > > diff --git a/src/sgroup.c b/src/sgroup.c > > index 904237c..0fdf8fb 100644 > > --- a/src/sgroup.c > > +++ b/src/sgroup.c > > @@ -79,7 +79,18 @@ log_source_group_deinit(LogPipe *s) > > evt_tag_str("id", p->id), > > NULL); > > success = FALSE; > > - } > > + } > > + if (p->group) > > + { > > + g_free(p->group); > > + p->group = NULL; > > + } > > + if (p->id) > > + { > > + g_free(p->id); > > + p->id = NULL; > > + } > > + log_pipe_append(&p->super, NULL); > > > hmm... can you show me the backtrace of the crash that this is supposed > to fix? when the configuration is deinitialized the old LogSource > instances should never be referenced again, thus this dangling pointer > will never be referenced again. > > Also, group/id is freed whenever the driver instance is freed, thus no > need to open code it. I see the possibility of a leak if syslog-ng falls > back to the old configuration as we coulnd't initialize the new one (in > which case log_sgroup_init is called multiple times on the same config), > but otherwise it should not be a leak. > The memory leaks again occured due to seeing deinit/init pairs being called. The fix for the crash is the log_pipe_append(), which prevents the driver using the sgroup again after it has been freed. I don't know the full reason for this happening, only that it did (with HUPs, changing configs, and lots of log messages all at the same time...) The stack trace was as follows: Thread Backtraces: Thread 1 (process 862): #0 log_source_group_queue (s=0x100795d8, msg=0x100aa598, path_options=0x7fa70ba4) at sgroup.c:98 #1 0x1000a238 in log_pipe_forward_msg (self=<value optimized out>, msg=0x1, path_options=0x3025134c) at logpipe.h:121 #2 0x1000a238 in log_pipe_forward_msg (self=<value optimized out>, msg=0x1, path_options=0x3025134c) at logpipe.h:121 #3 0x10026148 in log_source_queue (s=0x100bacf0, msg=0x100aa598, path_options=<value optimized out>) at logpipe.h:121 #4 0x1001b43c in log_reader_fd_dispatch (source=<value optimized out>, callback=<value optimized out>, user_data=<value optimized out>) at logpipe.h:121 #5 0x30070f9c in IA__g_main_context_dispatch (context=0x100507f0) at gmain.c:1916 #6 0x300732c4 in g_main_context_iterate (context=0x0, block=0, dispatch=1, self=<value optimized out>) at gmain.c:2547 #7 0x300733c8 in IA__g_main_context_iteration (context=0x0, may_block=268765168) at gmain.c:2606 #8 0x10005324 in main_loop_run (cfg=0x0) at main.c:175 #9 0x1000584c in main (argc=1, argv=0x7fa70de4) at main.c:428 The point it crashed at (sgroup.c:98) was the line: (*self->processed_messages)++; in log_source_group_queue(). But self has already been freed, so the memory address it was incrementing was bogus, causing the SEGV. The fix above means that the log_pipe_forward_msg() call will drop the message in this case.

2 1

syslog-ng 3.0.2 has been released
by devel＠balabit.hu 08 May '09

08 May '09

------------------------------------------------------------------------------ PACKAGE : syslog-ng VERSION : 3.0.2 SUMMARY : new stable release DATE : May 8, 2009 ------------------------------------------------------------------------------ DESCRIPTION: A new stable version of syslog-ng Open Source Edition (3.0.2) has been released. For latest fixes in the 3.0.x branch you are recommended to upgrade to this version. CHANGES: 3.0.2 Thu, 07 May 2009 08:30:06 +0200 Highlights: * This release is the first to feature binary packages, installation packages for various platforms. Bugfixes: * Only assume that a source logfile was rotated if the new log file has a non-zero size. This fix solves a minor race which might cause some messages to be lost from the end of the old logfile. * Fixed the processing of the -u and -g command line options to set the user/group of the syslog-ng process. Previously all usernames were rejected as non-existant. * Resolve usernames before chrooting, to avoid the need to have an /etc/passwd & /etc/group inside the chroot. * Fixed the read rate of regular files, with the change in this release it is now possible to fully saturate a destination pipe by using plain text files. * Fixed a synchronization problem in the SQL destination on AIX causing random crashes when the SQL destination is used. * Make the "syslog" reserved word usable as a facility name, previously syslog-ng indicated a syntax error if syslog as a facility was not enclosed in quotation marks. * Don't write the pidfile in case only a syntax check was requested. * Fixed compilation problems if SSL support is not enabled. * Set the default fd limit to 256 on cygwin as that platform does not support the current default of 4096 used on all other platforms. * Fixed template expansion for non-inline templates defined using the template statement. * Fixed the handling of zero-sized packets in the udp() source. * Fixed the storing of regexp/db-parser() matches on big-endian, 32 bit CPUs. * Fixed character conversion in case one chunk of data has an incomplete sequence at the end. * Don't let the character conversion eat up unlimited memory to avoid an easy DoS condition. * Fixed SSL certificate validation, added X.509 policy check, fixed required-untrusted verify-mode and added basicConstraints validation. * Fixed /dev/klog handling on FreeBSD, syslog-ng attempted to open this device multiple times which is not permitted. * Fixed $TZOFS expansion in templates in case the target does not specify a conversion timezone (which is the default and is true in most cases). * Fixed the use of multiple SQL destinations, previously due to an initialization problem only the first SQL destination would ever start writing messages to the target database. * If you elected to use a host-file only name resolution, the check to see if the hosts file was changed was performed for each DNS resolution, causing a major performance drop. This was fixed. * Due to a typo in the configuration file parser, using SSL destinations always resulted in a segmentation fault. This was fixed. * Fixed counting the number of concurrent connections in the SSL enabled tcp() source, previously failing handshakes were counted, thus it is quite easy to reach the max-connection limit. * Clarified logging about connection rejected by /etc/hosts.deny & /etc/hosts.allow * HP-UX 11v3 compilation fixes * Fixed SOURCEIP expansion for IPv6 addresses. * Fixed match(type(string)) not to match the message in case only a prefix match was found. * Fixed capability setup code around file opens. Other changes: * Implemented 'store-legacy-msghdr' source flag which keeps the original value of MSGHDR intact, instead of parsing/rebuilding it at the cost of some performance penalty. * Implemented 'empty-lines' source flag to avoid dropping empty log messages which is the default behaviour. * Added support for file includes in the configuration file. * Allow follow_freq() to be a floating point number. * Cygwin packaging fixes. * Make it possible to create named pipes automatically in the pipe() destination. * Allow the use of macros in the filename of pipe() destinations. * Added support for setting the default priority/facility value for each source that get applied to messages that lack one. * Recognize "mssql" as an alias for "freetds" in the SQL destination. * Added the "greedy" flags to the csv-parser(): it makes the last column eat the complete end of the line without caring about column separation characters * Added the "drop-invalid" flag to the csv-parser() which makes the parser drop anything that has less columns than it expects to have. * Make it possible to use fractions of a second even in non-ISO timestamps. * The default values for SQL columns is being deprecated, please supply SQL schema information explicitly in your SQL destinations. A warning message was added in case you don't have a full schema and rely on the default values. * The SQL destination got a null() option that specifies a 'special' value for a column which get replaced by an SQL NULL. * Added local_timezone() option to file/sql destinations that control the timezone used when expanding the filename/tablename templates. It defaults to the local timezone. Configure changes: * This section lists changes that might affect distribution packages, please make sure that these changes do not break the syslog-ng packages in any way. * Changed the default value of pidfiledir to ${localstatedir} to match GNU conventions, you can specify it explicitly with a --with-pidfile-dir configure option. * Changed the default linking mode to "dynamic" to make it easier to compile syslog-ng, as most distributions prefer dynamic libraries over static ones. This means that syslog-ng might not be able to start up if /usr is on NFS. On platforms where syslog-ng is used as a system logger, the previous setting of --enable-mixed-linking is preferred. Loggen changes: * the sequence number and timestamping of generated messages were erroneously commented out, these were reverted. * do not allow message sizes that are larger than the predefined message buffer size. * in addition to reporting the number of messages/sec, also report the effective bandwidth used during testing. Test suite changes: * A lot of enhancements to the functional test program to cover more of the syslog-ng code, like SSL sources & SQL destinations. DOWNLOAD: You can download the latest binaries from: http://www.balabit.com/network-security/syslog-ng/opensource-logging-system… The latest versions of syslog-ng in source format can be found at: http://www.balabit.com/downloads/files/syslog-ng/open-source-edition/

2 1

Question on '$PRIORITY' and '$LEVEL'
by Arnold Wang 08 May '09

08 May '09

I have a question regarding how syslog-ng treats "priority" and "level" value from the logs. According to RFC, "priority" is different than "level". However, it appears to me that syslog-ng use "level" value for "priority" as well. My questions are: 1. Is that true that syslog-ng use "level" value for "priority" as well? 2. Is there a reason behind this behavior? 3. Is there a plan to change/fix this? Thanks.

2 2

PATCH: Fix some small memory leaks and a SEGV
by mark tomlinson 08 May '09

08 May '09

Fix various memory leaks caused by strings not being freed. Also, in log_source_group_deinit(), ensure no drivers are still pointing to this possibly soon-to-be-freed structure. Without this a SIGSEGV was possible when a SIGHUP occured while syslog was busy. diff --git a/src/afinet.c b/src/afinet.c index 28b9d50..144a08a 100644 --- a/src/afinet.c +++ b/src/afinet.c @@ -191,6 +191,8 @@ afinet_sd_set_transport(LogDriver *s, const gchar *transport) { AFInetSourceDriver *self = (AFInetSourceDriver *) s; + if (self->super.transport) + g_free(self->super.transport); self->super.transport = g_strdup(transport); if (strcasecmp(transport, "udp") == 0) { diff --git a/src/afstreams.c b/src/afstreams.c index df4e758..35d4812 100644 --- a/src/afstreams.c +++ b/src/afstreams.c @@ -104,6 +104,8 @@ afstreams_sd_set_sundoor(LogDriver *s, gchar *filename) { AFStreamsSourceDriver *self = (AFStreamsSourceDriver *) s; + if (self->door_filename) + g_string_free(self->door_filename, TRUE); self->door_filename = g_string_new(filename); } diff --git a/src/dgroup.c b/src/dgroup.c index 7919a54..be02899 100644 --- a/src/dgroup.c +++ b/src/dgroup.c @@ -76,6 +76,16 @@ log_dest_group_deinit(LogPipe *s) NULL); success = FALSE; } + if (p->group) + { + g_free(p->group); + p->group = NULL; + } + if (p->id) + { + g_free(p->id); + p->id = NULL; + } } return success; } diff --git a/src/logsource.c b/src/logsource.c index 311e61f..93a8747 100644 --- a/src/logsource.c +++ b/src/logsource.c @@ -181,6 +181,10 @@ log_source_set_options(LogSource *self, LogSourceOptions *options, gint stats_le g_atomic_counter_set(&self->options->window_size, current_window); self->stats_level = stats_level; self->stats_source = stats_source; + if (self->stats_id) + g_free(self->stats_id); + if (self->stats_instance) + g_free(self->stats_instance); self->stats_id = stats_id ? g_strdup(stats_id) : NULL; self->stats_instance = stats_instance ? g_strdup(stats_instance): NULL; } diff --git a/src/logwriter.c b/src/logwriter.c index 4c895cc..1d1e00e 100644 --- a/src/logwriter.c +++ b/src/logwriter.c @@ -792,6 +792,10 @@ log_writer_set_options(LogWriter *self, LogPipe *control, LogWriterOptions *opti self->stats_level = stats_level; self->stats_source = stats_source; + if (self->stats_id) + g_free(self->stats_id); + if (self->stats_instance) + g_free(self->stats_instance); self->stats_id = stats_id ? g_strdup(stats_id) : NULL; self->stats_instance = stats_instance ? g_strdup(stats_instance) : NULL; diff --git a/src/sgroup.c b/src/sgroup.c index 904237c..0fdf8fb 100644 --- a/src/sgroup.c +++ b/src/sgroup.c @@ -79,7 +79,18 @@ log_source_group_deinit(LogPipe *s) evt_tag_str("id", p->id), NULL); success = FALSE; - } + } + if (p->group) + { + g_free(p->group); + p->group = NULL; + } + if (p->id) + { + g_free(p->id); + p->id = NULL; + } + log_pipe_append(&p->super, NULL); } return success; }

3 2

suppressing pidfile and control socket for syslog-ng 3.0
by Denis Laplante 08 May '09

08 May '09

To clarify my question about suppressing PID and control socket in syslog-ng version 3.0: I don't want the error message to be suppressed by having the file successfully created in syslog-ng's favourite location, I want to suppress opening the 2 files . I might want multiple syslog-ng processes running simultaneously, for example a test instance to run without interfering with production instances. In the case of PID file, I could perhaps specify "--pidfile /dev/null" ; is there a chance that the target file might get deleted or have permissions changed? A couple of minor complaints for this option: Option --pidfile does not handle relative directories (unlike syslog-ng.conf) . Option -P responds with "Error parsing command line arguments: Unknown option -P". > Message: 3 Date: Mon, 4 May 2009 10:51:20 +0200 (CEST) > From: bugzilla(a)bugzilla.balabit.com > Subject: [syslog-ng] [Bug 43] syslog-ng 2.1.4 consumes all CPU when processing "source file" > Comment #4 from Balazs Scheidler <bazsi(a)balabit.hu> 2009-05-04 10:51:21 > [...] >>> > > can you try 3.0 if it fixes your problem? >> > Yes, 3.0.1 on Solaris-8 fixes this problem. > [...] > > How do I suppress pidfile and/or control socket? >> > > Error opening control socket, bind() failed; error='Permission denied (13)', socket='/local/var/run/syslog-ng.ctl' > This is a different issue, you should probably create /local/var/run/ directory with appropriate permissions.

3 2