May 2009 - syslog-ng - lists.balabit.hu

syslog-ng 3.0.2 issues: Extraneous colon (:) in forwarded log; two instances of syslog-ng
by Marvin Nipper 03 Jun '09

03 Jun '09

Hi. I'm been running syslog-ng 2.0.9 for eons, and wanted to get 3.0.2 in place, but have run into two issues. First, the prerequisite info: This is on Solaris 10 x86, and I'm running the eventlog-0.2.9 component. It has all compiled cleanly. I'm using the same script to initialize syslog-ng as I was using before. In the conf file, added in some value() parameters on my filters, in accordance with the 3.0 "warnings", and added in the @version directive, but other than that, it's essentially the same (reasonably simple) conf file I was using with my 2.0.9 version of the code. For reference, I've appended my conf file at the end of the email. "Generally" syslog-ng, seems to start-up fine, and does feed the output files that I've indicated in the conf file. If I didn't "look too closely", I would have thought it was all fine, but... My problems: As part of my test environment, I'm forwarding data from another (production 2.0.9) server, to my test server, so that I can see how things are working. Here are some (truncated) messages, showing both the original message (as it looks in the output file on the 2.0.9 server), and the second message being what gets written to the file by the 3.0.2 component: Original: May 29 09:15:05 10.132.240.25 id=firewall sn=0006B10B2E88 <truncated> Received and written by 3.0.2: May 29 09:15:05 10.132.240.25 id=firewall: sn=0006B10B2E88 <truncated> So the "bug" (I assume it to be a bug) is that "colon" that is now sitting just behind the word "firewall". When my test server is running a 2.0.9 daemon, these two lines match, but the 3.0.2 daemon has inserted that colon. I can only assume that this is tied to a bug in the filtering activity, because if you look at my conf file, you will see these lines: filter SonicWallNoise { match("id=firewall" value("MSG")) and filter(SonicWallMsgs); }; Bottom line, I am doing a filter that happens to "match on" that exact "firewall" string, and it seems like more than just coincidence that this is the exact location where that unwanted colon now appears. It seems like that filtering process has somehow injected that character into the output. (Maybe I'm just missing some "new functionality or feature" in the 3.0 docs, that would intentionally cause this?) Anyway... I'm obviously looking for any input as to why this is different, and/or how to fix it (or even just work-around the problem until it is fixed). And secondly, as my subject line implies, when I run my "/etc/init.d/syslog-ng start" scripting (the same one I've always used), I find, right away, that I now have two instances of syslog-ng running. I saw a bug report from back in 2006, mentioning a similar behavior that was related to (I think) syslog-ng restarting itself, but that was supposedly fixed. I'm ass-u-me-ing that something is now causing the daemon to restart itself, but again, when I run my 2.0.9 code on place of the 3.0.2 code, I get just the one copy running. Again, I'm looking for any guidance as to something that might be driving that behavior (and as I said, maybe it's just something stupid on my part, because I'm missing something "new" in the 3.0 behavior). Thanks, in advance, for any help and/or guidance as to what these issues might be. I'm glad to provide further information if it will help. Marvin Nipper syslog-ng.conf: @version:3.0 options { dir_perm(0755); perm(0600); chain_hostnames(no); keep_hostname(yes); log_fifo_size(1000); dns_cache_size(5000); dns_cache_expire(86400); dns_cache_expire_failed(86400); }; source any_udp { udp(); }; source any_tcp { tcp(port(601)); }; destination SEC {pipe("/tmp/sec"); }; destination routers_log { file("/var/adm/log/routers.log" create_dirs(yes)); }; destination ravlin_log { file("/var/adm/log/ravlin.log" create_dirs(yes)); }; destination windows_log { file("/var/adm/log/windows.log" create_dirs(yes)); }; destination workstation_log { file("/var/adm/log/workstation.log" create_dirs(yes)); }; destination catch-all_log { file("/var/adm/log/catch-all.log" create_dirs(yes)); }; destination test { file("/var/adm/log/test.log" create_dirs(yes)); }; destination dev_null {}; filter f_4 { facility(syslog) and level(info..emerg); }; log { source(any_udp); filter(f_4); destination(windows_log); flags(final); }; filter f_1 { facility(local5) and level(debug..emerg); }; log { source(any_udp); filter(f_1); destination(routers_log); flags(final); }; filter SonicWallNoise { match("id=firewall" value("MSG")) and filter(SonicWallMsgs); }; filter SonicWallMsgs { match("m=97" value("MSG")) or match("m=98" value("MSG")) or match("m=537" value("MSG")); }; log { source(any_udp); filter(SonicWallNoise); destination(dev_null); flags(final); }; filter f_3 { facility(local0) and level(debug..emerg); }; log { source(any_udp); filter(f_3); destination(ravlin_log); flags(final); }; log { source(any_tcp); destination(workstation_log); flags(final); }; log { source(any_udp); destination(catch-all_log); flags(final); }; This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.

2 2

iptables logging in a separate logfile works not properly
by Ralf Heidenreich 02 Jun '09

02 Jun '09

Hello, I have the following line in iptables: iptables -A INPUT -j LOG --log-prefix "INPUT: " In syslog-ng (/etc/syslog-ng/syslog-ng.conf) I have the following config: ############################################# options { sync (0); time_reopen (10); log_fifo_size (1000); long_hostnames (off); use_dns (no); use_fqdn (no); create_dirs (no); keep_hostname (yes); }; source s_sys { file ("/proc/kmsg" log_prefix("kernel: ")); unix-stream ("/dev/log"); internal(); # udp(ip(0.0.0.0) port(514)); }; destination d_cons { file("/dev/console"); }; destination d_mesg { file("/var/log/messages"); }; destination d_auth { file("/var/log/secure"); }; destination d_mail { file("/var/log/maillog" sync(10)); }; destination d_spol { file("/var/log/spooler"); }; destination d_boot { file("/var/log/boot.log"); }; destination d_cron { file("/var/log/cron"); }; destination d_kern { file("/var/log/messages"); }; destination d_mlal { usertty("*"); }; # direct iptables logs into /var/log/firewall.log destination iptables { file("/var/log/firewall.log"); }; filter iptables { match("INPUT:"); }; log { source(s_sys); filter(iptables); destination(iptables); }; filter noiptables { not match("INPUT:"); }; log { source(s_sys); filter(noiptables); destination(d_kern); }; filter f_kernel { facility(kern); }; filter f_default { level(info..emerg) and not (facility(mail) or facility(authpriv) or facility(cron)); }; filter f_auth { facility(authpriv); }; filter f_mail { facility(mail); }; filter f_emergency { level(emerg); }; filter f_news { facility(uucp) or (facility(news) and level(crit..emerg)); }; filter f_boot { facility(local7); }; filter f_cron { facility(cron); }; #log { source(s_sys); filter(f_kernel); destination(d_cons); }; log { source(s_sys); filter(f_kernel); destination(d_kern); }; log { source(s_sys); filter(f_default); destination(d_mesg); }; log { source(s_sys); filter(f_auth); destination(d_auth); }; log { source(s_sys); filter(f_mail); destination(d_mail); }; log { source(s_sys); filter(f_emergency); destination(d_mlal); }; log { source(s_sys); filter(f_news); destination(d_spol); }; log { source(s_sys); filter(f_boot); destination(d_boot); }; log { source(s_sys); filter(f_cron); destination(d_cron); }; ################################################################ iptables loggs to /var/log/firewall.log, but also to /var/log/messages. A line in messages is: May 29 14:13:55 hostname kernel: INPUT: IN=eth0 OUT= MAC=00:16:17:6b:34:bf:00:02:85:12:c9:40:08:00 SRC=source-ip DST=server-ip LEN=104 TOS=0x00 PREC=0x00 TTL=56 ID=38612 PROTO=TCP SPT=26787 DPT=122 WINDOW=1032 RES=0x00 ACK PSH URGP=0 What I am doing wrong. with kind regards Ralf

2 3

db-parser issues
by Jakub Jankowski 29 May '09

29 May '09

Hi, I'm having hard time getting db-parser functionality to work. This is my syslog-ng config (omitted irrelevant stuff; some lines may be split due to line wrappings in my MUA): @version: 3.0 source s_sys { file ("/proc/kmsg"); unix-stream ("/dev/log" flags(no-multi-line)); internal(); }; destination d_test { file("/var/log/test.log" template("$HOUR:$MIN:$SEC $TZ $HOST [$LEVEL] ${.classifier.class} ${MSGONLY} ${FOO.BAR}\n") template_escape(no)); }; parser p_test { db-parser(file(/etc/syslog-ng/test.xml)); }; filter f_test { program("logger" type(string)); }; log { source(s_sys); filter(f_test); parser(p_test); destination(d_test); }; My pattern db (test.xml) is as simple as: <patterndb version='1' pub_date='2009-05-16'> <program name='logger'> <pattern>param</pattern> <rules> <rule provider='tester' id='666' class='security'> <description>Test rule</description> <pattern>param1=@IPV4:FOO.BAR@</pattern> </rule> </rules> </program> </patterndb> Syslog-ng has no trouble loading it on startup, as suggested by these lines in /var/log/messages: May 30 00:17:37 atest1 syslog-ng[8681]: Log pattern database reloaded; file='/etc/syslog-ng/test.xml', version='1', pub_date='2009-05-16' May 30 00:17:37 atest1 syslog-ng[8681]: syslog-ng starting up; version='3.0.2' I'm testing db-parser, by issuing: $ echo param1=1.2.3.4 param2=abcde | logger -i $ And this is what I get in /var/log/test.log: 00:17:38 +02:00 atest1 [notice] unknown param1=1.2.3.4 param2=abcde Clearly not what I wanted. Seems like my parser doesn't match anything, as ${FOO.BAR} is empty. Can you shed some light on my problem? Am I doing something obviously wrong? Db-parser functionality is neat, but it lacks documentation - I base on Balázs' presentation and some blog posts only. Thanks in advance, Jakub -- Jakub Jankowski|shasta@toxcorp.com|http://toxcorp.com/ GPG: FCBF F03D 9ADB B768 8B92 BB52 0341 9037 A875 942D

1 0

Re: [syslog-ng] syslog-ng 3.0.2 issues: Extraneous colon (:) in forwarded log; two instances of syslog-ng
by Marvin Nipper 29 May '09

29 May '09

Sorry for a second message, but I've just noticed a third issue with 3.0.2. Same test scenario as already described. Here's a messaged logged on my production 2.0.9 server, and also forwarded to my test server (I've removed some of the trailing noise): May 29 16:15:09 fwmil01 [LOG_WARNING] ipsrd[213]: igmp_recv_leave_group: <truncated.....> When it arrives on my test server, and a 2.0.9 daemon catches it (and writes it to a file), it looks exactly as above, but when 3.0.2 catches it, this is the result: May 29 16:15:09 fwmil01 ipsrd[213]: igmp_recv_leave_group: <truncated.....> In short, the Severity information has been stripped. Again, in looking at the 3.0 docs, I'm not clear as to any change that would cause this to be stripped out. (It might not be that big a deal to lose that, except that there are subsequent filters in my system that are looking for that "translated severity" as part of their search.) Anyway, again, I would appreciate any input as to why 3.x would be doing this, and what configuration change might restore the original content. THANKS for your help. _____________________________________________ From: Marvin Nipper Sent: Friday, May 29, 2009 12:08 PM To: 'syslog-ng(a)lists.balabit.hu' Subject: syslog-ng 3.0.2 issues: Extraneous colon (:) in forwarded log; two instances of syslog-ng Hi. I'm been running syslog-ng 2.0.9 for eons, and wanted to get 3.0.2 in place, but have run into two issues. First, the prerequisite info: This is on Solaris 10 x86, and I'm running the eventlog-0.2.9 component. It has all compiled cleanly. I'm using the same script to initialize syslog-ng as I was using before. In the conf file, added in some value() parameters on my filters, in accordance with the 3.0 "warnings", and added in the @version directive, but other than that, it's essentially the same (reasonably simple) conf file I was using with my 2.0.9 version of the code. For reference, I've appended my conf file at the end of the email. "Generally" syslog-ng, seems to start-up fine, and does feed the output files that I've indicated in the conf file. If I didn't "look too closely", I would have thought it was all fine, but... My problems: As part of my test environment, I'm forwarding data from another (production 2.0.9) server, to my test server, so that I can see how things are working. Here are some (truncated) messages, showing both the original message (as it looks in the output file on the 2.0.9 server), and the second message being what gets written to the file by the 3.0.2 component: Original: May 29 09:15:05 10.132.240.25 id=firewall sn=0006B10B2E88 <truncated> Received and written by 3.0.2: May 29 09:15:05 10.132.240.25 id=firewall: sn=0006B10B2E88 <truncated> So the "bug" (I assume it to be a bug) is that "colon" that is now sitting just behind the word "firewall". When my test server is running a 2.0.9 daemon, these two lines match, but the 3.0.2 daemon has inserted that colon. I can only assume that this is tied to a bug in the filtering activity, because if you look at my conf file, you will see these lines: filter SonicWallNoise { match("id=firewall" value("MSG")) and filter(SonicWallMsgs); }; Bottom line, I am doing a filter that happens to "match on" that exact "firewall" string, and it seems like more than just coincidence that this is the exact location where that unwanted colon now appears. It seems like that filtering process has somehow injected that character into the output. (Maybe I'm just missing some "new functionality or feature" in the 3.0 docs, that would intentionally cause this?) Anyway... I'm obviously looking for any input as to why this is different, and/or how to fix it (or even just work-around the problem until it is fixed). And secondly, as my subject line implies, when I run my "/etc/init.d/syslog-ng start" scripting (the same one I've always used), I find, right away, that I now have two instances of syslog-ng running. I saw a bug report from back in 2006, mentioning a similar behavior that was related to (I think) syslog-ng restarting itself, but that was supposedly fixed. I'm ass-u-me-ing that something is now causing the daemon to restart itself, but again, when I run my 2.0.9 code on place of the 3.0.2 code, I get just the one copy running. Again, I'm looking for any guidance as to something that might be driving that behavior (and as I said, maybe it's just something stupid on my part, because I'm missing something "new" in the 3.0 behavior). Thanks, in advance, for any help and/or guidance as to what these issues might be. I'm glad to provide further information if it will help. Marvin Nipper syslog-ng.conf: @version:3.0 options { dir_perm(0755); perm(0600); chain_hostnames(no); keep_hostname(yes); log_fifo_size(1000); dns_cache_size(5000); dns_cache_expire(86400); dns_cache_expire_failed(86400); }; source any_udp { udp(); }; source any_tcp { tcp(port(601)); }; destination SEC {pipe("/tmp/sec"); }; destination routers_log { file("/var/adm/log/routers.log" create_dirs(yes)); }; destination ravlin_log { file("/var/adm/log/ravlin.log" create_dirs(yes)); }; destination windows_log { file("/var/adm/log/windows.log" create_dirs(yes)); }; destination workstation_log { file("/var/adm/log/workstation.log" create_dirs(yes)); }; destination catch-all_log { file("/var/adm/log/catch-all.log" create_dirs(yes)); }; destination test { file("/var/adm/log/test.log" create_dirs(yes)); }; destination dev_null {}; filter f_4 { facility(syslog) and level(info..emerg); }; log { source(any_udp); filter(f_4); destination(windows_log); flags(final); }; filter f_1 { facility(local5) and level(debug..emerg); }; log { source(any_udp); filter(f_1); destination(routers_log); flags(final); }; filter SonicWallNoise { match("id=firewall" value("MSG")) and filter(SonicWallMsgs); }; filter SonicWallMsgs { match("m=97" value("MSG")) or match("m=98" value("MSG")) or match("m=537" value("MSG")); }; log { source(any_udp); filter(SonicWallNoise); destination(dev_null); flags(final); }; filter f_3 { facility(local0) and level(debug..emerg); }; log { source(any_udp); filter(f_3); destination(ravlin_log); flags(final); }; log { source(any_tcp); destination(workstation_log); flags(final); }; log { source(any_udp); destination(catch-all_log); flags(final); }; This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.

1 0

RFC 5424 compliant syslog library?
by Daniel Mahu 28 May '09

28 May '09

Hello, I'm looking for a linux library that would implement the syslog functions using the new standard format in RFC 5424. Of all the new things in the standard, I'm particularly interested in the timestamp resolution - I need to log timestamps with fractions of a second. Already tried searching google to no avail (only found a NetBSD libc patch, but I need linux). Since syslog-ng can now parse these messages, I thought maybe someone here may have heard about it if such a library existed. thanks, Daniel

2 1

How to name dir or logfile with the time by the message timestamp itself
by Roc Zhou 22 May '09

22 May '09

Hi, I'm now have a syslog-ng to collect the syslog of 2 sites, 1 is in China and another is in US, the host *vip-syslog* is in China, and US send the the messages to *vip-syslog* through a special tunnel, (for reliability, I can build a proxy with disk buffering to relay the syslog messages). Now * vip-syslog* can recevied the messages and works well, but I still suffer a problem. This is the syslog-ng.conf: #cat /etc/syslog-ng/syslog-ng.conf # # configuration file for syslog-ng, customized for remote logging # source s_internal { internal(); }; destination d_syslognglog { file("/var/log/syslog-ng.log"); }; log { source(s_internal); destination(d_syslognglog); }; options { ts_format(rfc3164); chain_hostnames(no); use_dns(yes); dns_cache(yes); dns_cache_hosts(/etc/syslog-ng/hosts); use_fqdn(no); perm(0644); dir_perm(0700); flush_lines(128); flush_timeout(500); log_msg_size(16384); use_time_recvd(no); # recv_time_zone(-07:00); # send_time_zone(-07:00); }; # --------------------------------------------------------------------- # Local sources, filters and destinations are commented out # If you want to replace sysklogd simply uncomment the following # parts and disable sysklogd # # Local sources # source s_local { unix-dgram("/dev/log"); file("/proc/kmsg" log_prefix("kernel:")); }; # # --------------------------------------------------------------------- # Local filters # filter f_messages { level(info..emerg); }; filter f_secure { facility(authpriv); }; filter f_mail { facility(mail); }; filter f_cron { facility(cron); }; filter f_emerg { level(emerg); }; filter f_spooler { level(crit..emerg) and facility(uucp, news); }; filter f_local7 { facility(local7); }; # # --------------------------------------------------------------------- # Local destinations # destination d_messages { file("/var/log/messages"); }; destination d_secure { file("/var/log/secure"); }; destination d_maillog { file("/var/log/maillog"); }; destination d_cron { file("/var/log/cron"); }; destination d_console { usertty("root"); }; destination d_spooler { file("/var/log/spooler"); }; destination d_bootlog { file("/var/log/boot.log"); }; # # --------------------------------------------------------------------- # Local logs - order DOES matter ! # log { source(s_local); filter(f_emerg); destination(d_console); }; log { source(s_local); filter(f_secure); destination(d_secure); flags(final); }; log { source(s_local); filter(f_mail); destination(d_maillog); flags(final); }; log { source(s_local); filter(f_cron); destination(d_cron); flags(final); }; log { source(s_local); filter(f_spooler); destination(d_spooler); }; log { source(s_local); filter(f_local7); destination(d_bootlog); }; log { source(s_local); filter(f_messages); destination(d_messages); }; # --------------------------------------------------------------------- # Remote logging # # Remote sources # source s_remote { tcp(ip(0.0.0.0) port(514)); udp(ip(0.0.0.0) port(514)); }; # --------------------------------------------------------------------- # Remote destinations # template t_message { # template("$STAMP $HOST $PROGRAM[$PID] $MSG\n"); template_escape(no); template("$S_STAMP $HOST $MSG\n"); template_escape(no); }; template t_apache { template("$MSGONLY\n"); template_escape(no); }; # --------------------------------- # Global site with US timezone: destination d_en_alarm { file("/var/log/syslog-ng/$S_YEAR-$S_MONTH-$S_DAY/$HOST/alarm" owner("root") group("root") perm(0640) dir_perm(0750) create_dirs(yes) time_zone(-07:00) template(t_message)); # program("sec.pl") }; destination d_en_necessary { file("/var/log/syslog-ng/$S_YEAR-$S_MONTH-$S_DAY/$HOST/necessary" owner("root") group("root") perm(0640) dir_perm(0750) create_dirs(yes) time_zone(-07:00) template(t_message)); }; destination d_en_cookie_log { file("/var/log/syslog-ng/$S_YEAR-$S_MONTH-$S_DAY/$HOST/cookie_log" owner("root") group("root") perm(0640) dir_perm(0750) create_dirs(yes) time_zone(-07:00) template(t_apache)); }; # --------------------------------- # China site with Beijing timezone: destination d_cn_alarm { file("/var/log/syslog-ng/$S_YEAR-$S_MONTH-$S_DAY/$HOST/alarm" owner("root") group("root") perm(0640) dir_perm(0750) create_dirs(yes) template(t_message)); }; destination d_cn_necessary { file("/var/log/syslog-ng/$S_YEAR-$S_MONTH-$S_DAY/$HOST/necessary" owner("root") group("root") perm(0640) dir_perm(0750) create_dirs(yes) template(t_message)); }; destination d_cn_cookie_log { file("/var/log/syslog-ng/$S_YEAR-$S_MONTH-$S_DAY/$HOST/cookie_log" owner("root") group("root") perm(0640) dir_perm(0750) create_dirs(yes) template(t_apache)); }; destination d_null { file("/dev/null"); }; destination d_sec { program("/etc/syslog-ng/sec.pl -input=\"-\" -conf=/etc/syslog-ng/sec.conf"); }; # --------------------------------------------------------------------- # Remote filters # filter f_en_host { host("(^hz|^us|^hk)_"); }; # # Used by data warehouse(dw): filter f_cn_local1 { facility(local1); }; filter f_en_local2 { facility(local2); }; filter f_en_dw { filter(f_en_host) and filter(f_en_local2); }; filter f_cn_dw { not filter(f_en_host) and filter(f_cn_local1); }; # Used by system admins and security admins: filter f_en_sys { filter(f_en_host) and not filter(f_en_local2); }; filter f_cn_sys { not filter(f_en_host) and not filter(f_cn_local1); }; filter f_necessary { match("necessary") or match("important"); ...... }; filter f_alarm { match("(?i)(error|fail)"); ...... }; ...... # --------------------------------------------------------------------- # Remote log path # log { source(s_remote); filter(f_en_sys); filter(f_necessary); destination(d_en_necessary); }; log { source(s_remote); filter(f_cn_sys); filter(f_necessary); destination(d_cn_necessary); }; log { source(s_remote); filter(f_en_sys); filter(f_alarm); destination(d_en_alarm); }; # log { source(s_remote); filter(f_en_sys); filter(f_necessary); destination(d_en_alarm); }; # --> This will not generate duplicated message! log { source(s_remote); filter(f_cn_sys); filter(f_alarm); destination(d_cn_alarm); }; log { source(s_remote); filter(f_cn_dw); destination(d_cn_cookie_log); }; log { source(s_remote); filter(f_en_dw); destination(d_en_cookie_log); }; log { source(s_remote); destination(d_null); }; NOW as you can see, I want the message to be logged to the dir: /var/log/syslog-ng/$S_YEAR-$S_MONTH-$S_DAY/$HOST/ The logs of China is OK, but the US is not since a timezone translating must be performed. Now the message timestamp in the logfile is right, but the dir name is not, for example: #head /var/log/syslog-ng/2009-05-22/us_search63/alarm May 21 10:50:27 us_search63 kernel: isupdate[7157]: segfault at 0000000000000010 rip 000000000041adad rsp 00007fffaa3cf560 error 4 May 21 18:09:55 us_search63 snmpd[26753]: /etc/snmp/snmpd.conf: line 433: Error: WARNING: This output format is being deprecated - Please use the 'extend' directive instead May 21 18:09:55 us_search63 snmpd[26753]: /etc/snmp/snmpd.conf: line 434: Error: WARNING: This output format is being deprecated - Please use the 'extend' directive instead May 21 18:09:55 us_search63 snmpd[26753]: /etc/snmp/snmpd.conf: line 435: Error: WARNING: This output format is being deprecated - Please use the 'extend' directive instead May 21 18:09:56 us_search63 snmpd[26753]: /etc/snmp/snmpd.conf: line 436: Error: WARNING: This output format is being deprecated - Please use the 'extend' directive instead As you can see, the dir of 2009-05-22 has the records of May 21, which is not what I want. I have tried the MACROS of $DAY, $S_DAY and $R_DAY, seems can not solve this problem. Any suggestions? Thanks.

1 0

Space in tag causes parsing problems - syslog-ng 2.0.8
by Fegan, Joe 21 May '09

21 May '09

Syslog-ng 2.0.8 I found that logging a message with a space in the tag causes the first word of the tag to be used as the hostname. This does not happen with the standard syslogd. Best illustrated by an example: [root@kudos8 ~]# logger -t "hello" "here is a message" [root@kudos8 ~]# tail -4 /var/log/messages May 18 16:43:53 kudos8 info snmpd[5296]: Connection from UDP: [127.0.0.1]:41757 May 18 16:43:53 kudos8 info snmpd[5296]: Connection from UDP: [127.0.0.1]:44965 May 18 16:43:53 kudos8 info snmpd[5296]: Received SNMP packet(s) from UDP: [127.0.0.1]:44965 May 18 16:43:55 kudos8 notice hello: this is a message [root@kudos8 ~]# This is expected behaviour. Now see what happens when I put a space in the tag: [root@kudos8 ~]# logger -t "hello world" "second message" [root@kudos8 ~]# tail -5 /var/log/messages May 18 16:43:53 kudos8 info snmpd[5296]: Connection from UDP: [127.0.0.1]:41757 May 18 16:43:53 kudos8 info snmpd[5296]: Connection from UDP: [127.0.0.1]:44965 May 18 16:43:53 kudos8 info snmpd[5296]: Received SNMP packet(s) from UDP: [127.0.0.1]:44965 May 18 16:43:55 kudos8 notice hello: this is a message May 18 16:44:04 hello notice world: second message [root@kudos8 ~]# See that "hello" was incorrectly used as the hostname. This is 100% reproducible. I stopped syslog-ng and started the standard syslogd instead. Standard syslogd does not have this problem: [root@kudos8 ~]# logger -t "hello" "here is a message" [root@kudos8 ~]# logger -t "hello world" "second message" [root@kudos8 ~]# service syslog-ng stop Stopping syslog-ng: [ OK ] [root@kudos8 ~]# service syslog start Starting system logger: [ OK ] Starting kernel logger: [ OK ] [root@kudos8 ~]# logger -t "hello world" "third message" [root@kudos8 ~]# tail -10 /var/log/messages May 18 16:54:25 kudos8 info snmpd[5296]: Connection from UDP: [127.0.0.1]:33587 May 18 16:54:25 kudos8 info snmpd[5296]: Connection from UDP: [127.0.0.1]:46177 May 18 16:54:25 kudos8 info snmpd[5296]: Received SNMP packet(s) from UDP: [127.0.0.1]:46177 May 18 16:54:30 kudos8 notice hello: here is a message May 18 16:54:31 hello notice world: second message May 18 16:54:34 kudos8 info syslog-ng[8737]: Termination requested via signal, terminating; May 18 16:54:34 kudos8 notice syslog-ng[8737]: syslog-ng shutting down; version=\'2.0.8\' May 18 16:54:38 kudos8 syslogd 1.4.1: restart. May 18 16:54:38 kudos8 kernel: klogd 1.4.1, log source = /proc/kmsg started. May 18 16:54:49 kudos8 hello world: third message [root@kudos8 ~]#

2 3

parsing netapp syslog (2)
by David - DCPC 21 May '09

21 May '09

Hello, I'm new on syslog-ng. I use v3.0.2 on a CentOS 4.4. I send syslog logs from a netapp FAS 2020 (*.*) to the syslog-ng server via udp. I have problem to analyse thoses logs. the lines have a different format than the standard syslog from another unix by example and are different than the BSD or IETF standard described in the syslog-ng documentation : (...) Tue May 19 11:27:30 CEST [netapp-L: wafl.quota.qtree.exceeded:notice]: tid 10: tree quota exceeded on volume linusers. Additional warnings will be suppressed for approximately 60 minutes or until a 'quota resize' is performed. (...) 1°) When i use it in syslog-ng i get this line in a sample extraction without filter : (...) May 19 11:27:30 netapp-b wafl.quota.qtree.exceeded:notice]: tid 10: tree quota exceeded on volume linusers. Additional warnings will be suppressed for approximately 60 minutes or until a 'quota resize' is performed. (...) (difference between host named should come from the renaming of hosts during tests, and dns aliases, don't think this is the problem). Is it possible to correct the processus that parse the original line ? it seems that the parser block can be used only for the $MESSAGE part of the line. 2°) It want to use other macro to make some filter and destination, like $PROGRAM. Is it possible to 'debug' the processing and have a details of macros and their contents, line by line ? Thx (sorry for the first miss) -- Salutations, David CHALON

2 1

NULL byte and Syslog-NG
by Jon Fredrickson 21 May '09

21 May '09

Hello, I am importing events from the Oracle sys.aud$ table via JDBC, then forwarding these events to a syslog-ng server. Every time a user makes an ODBC connection from a windows machine, a NULL byte appears directly after the host name in the USERHOST field. When syslog-ng gets this raw event it changes the NULL byte from 00 to a line feed 0a. Does anyone know how to stop this from happening, or allow syslog-ng to trim NULL characters? My syslog-ng version is 2.0.1 Thanks in advance, Jon

2 1

parsing netapp syslog
by David - DCPC 19 May '09

19 May '09

Hello, I'm new on syslog-ng. I use v3.0.2 on a CentOS 4.4. I send syslog logs from a netapp FAS 2020 (*.*) to the syslog-ng server via udp. I have problem to analyse thoses logs. the lines have a different format than the standard syslog from another unix by example. and different tha -- Salutations, David CHALON

1 0