- syslog-ng - lists.balabit.hu

Re: [syslog-ng] Syslog-NG 3.0.5 incorrectly parsing messages without program names
by Larry Low 15 Dec '09

15 Dec '09

> In the past I used the MSG macro to get the full syslog message. Now > MSG is broken into MSG and MSGHDR. > > If a message comes in without a program name such as "exiting on signal > 15" or "last message repeated 20 times", syslog-ng incorrectly takes > the first word as the program name. Trying to rebuild it with > $MSGHDR$MSG causes an erroneous : to be placed on these messages since > $MSGHDR is filled in with "exiting" and "last". I found a previous post and tried adding store-legacy-msghdr. The problem now is that MSGHDR still includes the first word such as "exiting" and "last" but no colon but on messages like "kernel : Kernel logging (proc) stopped.", "kernel: " is put in MSGHDR but then also placed in MSG and MSGONLY. > > --- > Larry Lo

2 7

Re: [syslog-ng] syntax error in config-file
by Pallagi Zoltán 15 Dec '09

15 Dec '09

andip írta: > Hi, > > You're right i got the syslog filter, but not destination. i changed that to > _syslog, and as i started syslog-ng again, i got syntax-errors on different > lines. i fixed the lines one after the other (see comments in config file in > original mail). the last error that i could not get past was on line 35, > which is the first filter-line in my config-file. > double quote is missing from filter: - filter aplog { host (10.20.); }; + filter aplog { host ("10.20."); }; > so since syslog-ng exited with syntax-error, and it reports it being line 35, > i felt sure it had to do with that line, though ofc i can be wrong as i'm an > amateur. i am ofc aware there are/might be errors further down in the > config-file. > i'd very much like to be certain though, before i re-install version 3.0.4 > again. > > could you please confirm for me that you're aware of this before i try out > your suggested solution? > There is my solution, it worked for me: @@ -28,8 +28,7 @@ destination ipflog { file ("/var/log/ipflog"); }; destination imaplog { file ("/var/log/imaplog"); }; destination console { file ("/dev/console"); }; -destination syslog { file ("/var/log/syslog"); }; ### changed -to _syslog +destination _syslog { file ("/var/log/syslog"); }; ### changedto _syslog destination komm { file ("/var/log/fwlogs/kommunikasjon-context.log"); }; destination asa { file ("/var/log/fwlogs/asa.log"); }; @@ -47,8 +46,7 @@ destination wifi { file ("/var/log/fwlogs/wifi-context.log"); }; destination aplog { file ("/var/log/aplog/aplog.txt"); }; -filter aplog { host (10.20.); }; ### this line caused the -syntax error (line 35) +filter aplog { host ("10.20."); }; filter gje { host (10.8.30.28); }; filter nes { host (10.8.30.20); }; filter ull { host (10.8.30.36); }; @@ -75,7 +73,7 @@ filter mail { facility (mail); }; filter daemon { facility (daemon); }; filter auth { facility (auth); }; -filter syslog { facility (syslog); }; +filter _syslog { facility (syslog); }; filter lpr { facility (lpr); }; filter news { facility (news); }; filter uucp { facility (uucp); }; @@ -103,7 +101,7 @@ log { source (sys); filter (mail); destination (maillog); }; #log { source (sys); filter (local0); destination (ipflog); }; #log { source (sys); filter (local1); destination (imaplog); }; -log { source (sys); filter (syslog); destination (syslog); }; +log { source (sys); filter (_syslog); destination (_syslog); }; log { source (sys); filter (emerg); filter (user_none); destination (console); }; log { source (sys); filter (mesg); filter (user_none); and of course you also have to check the warnings after starting. If everything is all right put "@version: 3.0" line on the top of your config. > cheers > andip > > ----- Original Message ----- > From: Pallagi Zoltán <pzolee(a)balabit.hu> > To: Syslog-ng users' and developers' mailing list > <syslog-ng(a)lists.balabit.hu>, andip(a)syndig.com > Sent: Tue, 15 Dec 2009 10:43:10 +0100 > Subject: Re: [syslog-ng] syntax error in config-file > > >> Hi, >> >> In syslog-ng 3.0, "syslog" became a reserved word and you have a >> destination and a filter called "syslog" in your config. >> Please rename them and it will work >> >> andip írta: >> >>> Hi folks, >>> >>> I just tried updating syslog-ng to version 3.0.4, from 2.1.4. i use the >>> gentoo linux-distro, and the new version came into "stable" gentoo a few >>> >> days >> >>> ago. >>> >>> I'm no syslog-ng expert, but i use it to log quite a few things, so my >>> config-file it not default. >>> I've read a bit in the admin-guide, but did not find any reason to my >>> problem. >>> >>> i've tried converting my 2.1.4 config-file to the new 3.0.4 version, with >>> > > >> no >> >>> success. this is parts of my old config-file. ### are comments made to >>> reflect changes i made for v2 -> v3 conversion >>> >>> > ---------------------------------------------------------------------------- > >>> options { >>> chain_hostnames(off); ### changed this to "no" >>> sync(0); ### changed this to "flush_lines" >>> time_sleep(20); >>> log_fifo_size (2048); >>> create_dirs (yes); >>> perm (0640); >>> dir_perm (0750); >>> stats(43200); ### changed this to "stats_freq" >>> owner (syslogger); >>> group (syslogger); >>> perm (0664); >>> }; >>> >>> source sys { >>> unix-stream("/dev/log" max-connections(256)); >>> internal(); >>> file("/proc/kmsg"); >>> }; >>> >>> source net { udp(); }; >>> >>> destination clients { file ("/var/log/fwlogs/netclients/$HOST"); }; >>> destination console_all { file ("/dev/tty12"); }; >>> destination authlog { file ("/var/log/authlog"); }; >>> destination messages { file ("/var/log/messages"); }; >>> destination maillog { file ("/var/log/maillog"); }; >>> destination ipflog { file ("/var/log/ipflog"); }; >>> destination imaplog { file ("/var/log/imaplog"); }; >>> destination console { file ("/dev/console"); }; >>> destination syslog { file ("/var/log/syslog"); }; ### changed >>> >> to _syslog >> >>> >>> >>> destination komm { file ("/var/log/fwlogs/kommunikasjon-context.log"); >>> > > >>> }; >>> destination asa { file ("/var/log/fwlogs/asa.log"); }; >>> destination edb { file ("/var/log/fwlogs/edb-pix.log"); }; >>> destination gje { file ("/var/log/fwlogs/gjerdrum.log"); }; >>> destination nes { file ("/var/log/fwlogs/nes.log"); }; >>> destination ull { file ("/var/log/fwlogs/ullensaker.log"); }; >>> destination hur { file ("/var/log/fwlogs/hurdal.log"); }; >>> destination eid { file ("/var/log/fwlogs/eidsvoll.log"); }; >>> destination nan { file ("/var/log/fwlogs/nannestad.log"); }; >>> destination datas { file ("/var/log/fwlogs/datasenter-context.log"); }; >>> > > >>> destination mgmt { file ("/var/log/fwlogs/management-context.log"); }; >>> > > >>> destination wifi { file ("/var/log/fwlogs/wifi-context.log"); }; >>> destination aplog { file ("/var/log/aplog/aplog.txt"); }; >>> >>> filter aplog { host (10.20.); }; ### this line caused >>> > the > >>> syntax error (line 35) >>> filter gje { host (10.8.30.28); }; >>> filter nes { host (10.8.30.20); }; >>> filter ull { host (10.8.30.36); }; >>> filter hur { host (10.8.30.12); }; >>> filter eid { host (10.8.30.44); }; >>> filter nan { host (10.8.30.52); }; >>> filter datas { host (10.8.0.1); }; >>> filter mgmt { host (10.8.30.60); }; >>> filter wifi { host (10.8.30.92); }; >>> filter asa { host (ASA-R111); }; >>> filter komm { host (10.8.30.76); }; >>> filter edb { host (edb-pix); }; >>> >>> filter emerg { level (emerg); }; >>> filter alert { level (alert .. emerg); }; >>> filter crit { level (crit .. emerg); }; >>> filter err { level (err .. emerg); }; >>> filter warning { level (warning .. emerg); }; >>> filter notice { level (notice .. emerg); }; >>> filter info { level (info .. emerg); }; >>> filter debug { level (debug .. emerg); }; >>> filter kern { facility (kern); }; >>> filter user { facility (user); }; >>> filter mail { facility (mail); }; >>> filter daemon { facility (daemon); }; >>> filter auth { facility (auth); }; >>> filter syslog { facility (syslog); }; >>> filter lpr { facility (lpr); }; >>> filter news { facility (news); }; >>> filter uucp { facility (uucp); }; >>> filter cron { facility (cron); }; >>> #filter local0 { facility (local0); }; >>> #filter local1 { facility (local1); }; >>> #filter local2 { facility (local2); }; >>> #filter local3 { facility (local3); }; >>> #filter local4 { facility (local4); }; >>> #filter local5 { facility (local5); }; >>> #filter local6 { facility (local6); }; >>> #filter local7 { facility (local7); }; >>> filter user_none { not facility (user); }; >>> filter kern_debug { filter (kern) and filter (debug); }; >>> filter daemon_notice { filter (daemon) and filter (notice); }; >>> filter mail_crit { filter (mail) and filter (crit); }; >>> filter mesg { filter (kern_debug) or >>> filter (daemon_notice) or >>> filter (mail_crit); }; >>> filter authinfo { filter (auth) or program (sudo); }; >>> >>> log { source (sys); destination(messages); }; >>> log { source (sys); destination(console_all); }; >>> log { source (sys); filter (authinfo); destination (authlog); }; >>> log { source (sys); filter (mail); destination (maillog); }; >>> #log { source (sys); filter (local0); destination (ipflog); }; >>> #log { source (sys); filter (local1); destination (imaplog); }; >>> log { source (sys); filter (syslog); destination (syslog); }; >>> log { source (sys); filter (emerg); filter (user_none); >>> destination (console); }; >>> log { source (sys); filter (mesg); filter (user_none); >>> destination (messages); }; >>> >>> log { source (net); destination (clients); }; >>> log { source (net); filter (komm); destination (komm); }; >>> log { source (net); filter (asa); destination (asa); }; >>> log { source (net); filter (mgmt); destination (mgmt); }; >>> log { source (net); filter (wifi); destination (wifi); }; >>> log { source (net); filter (datas); destination (datas); }; >>> log { source (net); filter (wifi); destination (wifi); }; >>> log { source (net); filter (gje); destination (gje); }; >>> log { source (net); filter (nan); destination (nan); }; >>> log { source (net); filter (ull); destination (ull); }; >>> log { source (net); filter (hur); destination (hur); }; >>> log { source (net); filter (eid); destination (eid); }; >>> log { source (net); filter (nes); destination (nes); }; >>> log { source (net); filter (edb); destination (edb); }; >>> log { source (net); filter (aplog); destination (aplog); }; >>> #log { source (net); filter (komm); destination (komm); }; >>> ---------------------------------------------------------------------- >>> This is the error i got : >>> >>> syntax error in /etc/syslog-ng/syslog-ng.conf at line 35. >>> >>> syslog-ng documentation: >>> http://www.balabit.com/support/documentation/?product=syslog-ng >>> mailing list: https://lists.balabit.hu/mailman/listinfo/syslog-ng >>> > > >>> >>> * Configuration error. Please fix your configfile >>> (/etc/syslog-ng/syslog-ng.conf) >>> >>> It is not a very big deal at the moment, as i've reverted back to version >>> > > >>> 2.1.4, but i'd like to keep my system current, and somewhat up to date. >>> >> this >> >>> server's main function is as a syslogger for a few firewalls though, so i >>> > > >>> cannot leave it unable to log for long. >>> >>> hope someone can help. >>> >>> cheers >>> andip >>> >>> > _____________________________________________________________________________ > > >> _ >> >>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng >>> Documentation: >>> >> http://www.balabit.com/support/documentation/?product=syslog-ng >> >>> FAQ: http://www.campin.net/syslog-ng/faq.html >>> >>> >>> >>>

2 1

Licensing of 3.0.2 OSE ?
by Fegan, Joe 15 Dec '09

15 Dec '09

Hi all - A licensing question about syslog-ng was sent to me privately recently and I thought it would be best to forward it to this list for comments. > We have been using syslog-ng-1.6.2 in our product for quite a few years now. > We received a case recently on syslog-ng which prompted us to get the newer > version of syslog-ng. We took the syslog-ng_3.0.2 open source edition and > replaced the older one with this. > > We see that the licensing has changed from GPL to BalaBit Proprietary License > since syslog-ng version 3.0.1. The license remains GPL till syslog-ng version > 2.1.4. > > My question here is, can we build the newer open source syslog-ng which has > the Balabit propriety license and distribute it to our customers as part of > our product? I replied that the COPYING file in the 3.0.2 tarball says GPL, so I don't understand the question. The person's reply was: > Joe, > > I missed out on mentioning that part to you. > The COPYING is GPL. But in the spec file we see the following. > > # > # Specfile used by BalaBit internally. > # > Summary: Next generation system logging daemon > Name: syslog-ng > Version: 3.0.1 > Release: 1 > License: BalaBit Proprietary License > > And this is seen when you do rpm -qi : > > [root@n10 ~]# rpm -qi syslog-ng-xx-3.0.2-1 > Name : syslog-ng-xx Relocations: (not relocatable) > Version : 3.0.2 Vendor: Balabit IT Ltd. > Release : 1 Build Date: Mon 07 Dec 2009 06:40:22 PM IST > Install Date: Thu 10 Dec 2009 02:01:43 PM IST Build Host: builder.mycompany.com > Group : System Environment/Daemons Source RPM: syslog-ng-xx-3.0.2-1.src.rpm > Size : 554028 License: BalaBit Proprietary License > Signature : (none) > Packager : Tamas Pal <folti(a)balabit.com> > URL : http://www.balabit.com > Summary : Next generation system logging daemon > Description : > Syslog-ng is a next generation system logger daemon which provides more > capabilities and it has a more flexible configuration then the traditional > syslog daemon.

3 3

Re: [syslog-ng] syntax error in config-file
by andip 15 Dec '09

15 Dec '09

Hi, You're right i got the syslog filter, but not destination. i changed that to _syslog, and as i started syslog-ng again, i got syntax-errors on different lines. i fixed the lines one after the other (see comments in config file in original mail). the last error that i could not get past was on line 35, which is the first filter-line in my config-file. so since syslog-ng exited with syntax-error, and it reports it being line 35, i felt sure it had to do with that line, though ofc i can be wrong as i'm an amateur. i am ofc aware there are/might be errors further down in the config-file. i'd very much like to be certain though, before i re-install version 3.0.4 again. could you please confirm for me that you're aware of this before i try out your suggested solution? cheers andip ----- Original Message ----- From: Pallagi Zoltán <pzolee(a)balabit.hu> To: Syslog-ng users' and developers' mailing list <syslog-ng(a)lists.balabit.hu>, andip(a)syndig.com Sent: Tue, 15 Dec 2009 10:43:10 +0100 Subject: Re: [syslog-ng] syntax error in config-file > Hi, > > In syslog-ng 3.0, "syslog" became a reserved word and you have a > destination and a filter called "syslog" in your config. > Please rename them and it will work > > andip írta: > > Hi folks, > > > > I just tried updating syslog-ng to version 3.0.4, from 2.1.4. i use the > > gentoo linux-distro, and the new version came into "stable" gentoo a few > days > > ago. > > > > I'm no syslog-ng expert, but i use it to log quite a few things, so my > > config-file it not default. > > I've read a bit in the admin-guide, but did not find any reason to my > > problem. > > > > i've tried converting my 2.1.4 config-file to the new 3.0.4 version, with > no > > success. this is parts of my old config-file. ### are comments made to > > reflect changes i made for v2 -> v3 conversion > > > ---------------------------------------------------------------------------- > > options { > > chain_hostnames(off); ### changed this to "no" > > sync(0); ### changed this to "flush_lines" > > time_sleep(20); > > log_fifo_size (2048); > > create_dirs (yes); > > perm (0640); > > dir_perm (0750); > > stats(43200); ### changed this to "stats_freq" > > owner (syslogger); > > group (syslogger); > > perm (0664); > > }; > > > > source sys { > > unix-stream("/dev/log" max-connections(256)); > > internal(); > > file("/proc/kmsg"); > > }; > > > > source net { udp(); }; > > > > destination clients { file ("/var/log/fwlogs/netclients/$HOST"); }; > > destination console_all { file ("/dev/tty12"); }; > > destination authlog { file ("/var/log/authlog"); }; > > destination messages { file ("/var/log/messages"); }; > > destination maillog { file ("/var/log/maillog"); }; > > destination ipflog { file ("/var/log/ipflog"); }; > > destination imaplog { file ("/var/log/imaplog"); }; > > destination console { file ("/dev/console"); }; > > destination syslog { file ("/var/log/syslog"); }; ### changed > to _syslog > > > > > destination komm { file ("/var/log/fwlogs/kommunikasjon-context.log"); > > > }; > > destination asa { file ("/var/log/fwlogs/asa.log"); }; > > destination edb { file ("/var/log/fwlogs/edb-pix.log"); }; > > destination gje { file ("/var/log/fwlogs/gjerdrum.log"); }; > > destination nes { file ("/var/log/fwlogs/nes.log"); }; > > destination ull { file ("/var/log/fwlogs/ullensaker.log"); }; > > destination hur { file ("/var/log/fwlogs/hurdal.log"); }; > > destination eid { file ("/var/log/fwlogs/eidsvoll.log"); }; > > destination nan { file ("/var/log/fwlogs/nannestad.log"); }; > > destination datas { file ("/var/log/fwlogs/datasenter-context.log"); }; > > > > > destination mgmt { file ("/var/log/fwlogs/management-context.log"); }; > > > > > destination wifi { file ("/var/log/fwlogs/wifi-context.log"); }; > > destination aplog { file ("/var/log/aplog/aplog.txt"); }; > > > > filter aplog { host (10.20.); }; ### this line caused the > > syntax error (line 35) > > filter gje { host (10.8.30.28); }; > > filter nes { host (10.8.30.20); }; > > filter ull { host (10.8.30.36); }; > > filter hur { host (10.8.30.12); }; > > filter eid { host (10.8.30.44); }; > > filter nan { host (10.8.30.52); }; > > filter datas { host (10.8.0.1); }; > > filter mgmt { host (10.8.30.60); }; > > filter wifi { host (10.8.30.92); }; > > filter asa { host (ASA-R111); }; > > filter komm { host (10.8.30.76); }; > > filter edb { host (edb-pix); }; > > > > filter emerg { level (emerg); }; > > filter alert { level (alert .. emerg); }; > > filter crit { level (crit .. emerg); }; > > filter err { level (err .. emerg); }; > > filter warning { level (warning .. emerg); }; > > filter notice { level (notice .. emerg); }; > > filter info { level (info .. emerg); }; > > filter debug { level (debug .. emerg); }; > > filter kern { facility (kern); }; > > filter user { facility (user); }; > > filter mail { facility (mail); }; > > filter daemon { facility (daemon); }; > > filter auth { facility (auth); }; > > filter syslog { facility (syslog); }; > > filter lpr { facility (lpr); }; > > filter news { facility (news); }; > > filter uucp { facility (uucp); }; > > filter cron { facility (cron); }; > > #filter local0 { facility (local0); }; > > #filter local1 { facility (local1); }; > > #filter local2 { facility (local2); }; > > #filter local3 { facility (local3); }; > > #filter local4 { facility (local4); }; > > #filter local5 { facility (local5); }; > > #filter local6 { facility (local6); }; > > #filter local7 { facility (local7); }; > > filter user_none { not facility (user); }; > > filter kern_debug { filter (kern) and filter (debug); }; > > filter daemon_notice { filter (daemon) and filter (notice); }; > > filter mail_crit { filter (mail) and filter (crit); }; > > filter mesg { filter (kern_debug) or > > filter (daemon_notice) or > > filter (mail_crit); }; > > filter authinfo { filter (auth) or program (sudo); }; > > > > log { source (sys); destination(messages); }; > > log { source (sys); destination(console_all); }; > > log { source (sys); filter (authinfo); destination (authlog); }; > > log { source (sys); filter (mail); destination (maillog); }; > > #log { source (sys); filter (local0); destination (ipflog); }; > > #log { source (sys); filter (local1); destination (imaplog); }; > > log { source (sys); filter (syslog); destination (syslog); }; > > log { source (sys); filter (emerg); filter (user_none); > > destination (console); }; > > log { source (sys); filter (mesg); filter (user_none); > > destination (messages); }; > > > > log { source (net); destination (clients); }; > > log { source (net); filter (komm); destination (komm); }; > > log { source (net); filter (asa); destination (asa); }; > > log { source (net); filter (mgmt); destination (mgmt); }; > > log { source (net); filter (wifi); destination (wifi); }; > > log { source (net); filter (datas); destination (datas); }; > > log { source (net); filter (wifi); destination (wifi); }; > > log { source (net); filter (gje); destination (gje); }; > > log { source (net); filter (nan); destination (nan); }; > > log { source (net); filter (ull); destination (ull); }; > > log { source (net); filter (hur); destination (hur); }; > > log { source (net); filter (eid); destination (eid); }; > > log { source (net); filter (nes); destination (nes); }; > > log { source (net); filter (edb); destination (edb); }; > > log { source (net); filter (aplog); destination (aplog); }; > > #log { source (net); filter (komm); destination (komm); }; > > ---------------------------------------------------------------------- > > This is the error i got : > > > > syntax error in /etc/syslog-ng/syslog-ng.conf at line 35. > > > > syslog-ng documentation: > > http://www.balabit.com/support/documentation/?product=syslog-ng > > mailing list: https://lists.balabit.hu/mailman/listinfo/syslog-ng > > > > > * Configuration error. Please fix your configfile > > (/etc/syslog-ng/syslog-ng.conf) > > > > It is not a very big deal at the moment, as i've reverted back to version > > 2.1.4, but i'd like to keep my system current, and somewhat up to date. > this > > server's main function is as a syslogger for a few firewalls though, so i > > cannot leave it unable to log for long. > > > > hope someone can help. > > > > cheers > > andip > > > _____________________________________________________________________________ > _ > > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng > > Documentation: > http://www.balabit.com/support/documentation/?product=syslog-ng > > FAQ: http://www.campin.net/syslog-ng/faq.html > > > > > >

1 0

More verisign questions
by Jimmy McDonald 15 Dec '09

15 Dec '09

Hello, Thank you very much for the information! It got me one step closer. My root CA does not have basicConstraints set, but my intermediate does. Is there any way around this issue? I have spoken with my CA (Verisign) and they cannot provide a different root CA. Any ideas? Thank you, Jimmy Sent from my iPhone... So expect typos.

2 1

syntax error in config-file
by andip 15 Dec '09

15 Dec '09

Hi folks, I just tried updating syslog-ng to version 3.0.4, from 2.1.4. i use the gentoo linux-distro, and the new version came into "stable" gentoo a few days ago. I'm no syslog-ng expert, but i use it to log quite a few things, so my config-file it not default. I've read a bit in the admin-guide, but did not find any reason to my problem. i've tried converting my 2.1.4 config-file to the new 3.0.4 version, with no success. this is parts of my old config-file. ### are comments made to reflect changes i made for v2 -> v3 conversion ---------------------------------------------------------------------------- options { chain_hostnames(off); ### changed this to "no" sync(0); ### changed this to "flush_lines" time_sleep(20); log_fifo_size (2048); create_dirs (yes); perm (0640); dir_perm (0750); stats(43200); ### changed this to "stats_freq" owner (syslogger); group (syslogger); perm (0664); }; source sys { unix-stream("/dev/log" max-connections(256)); internal(); file("/proc/kmsg"); }; source net { udp(); }; destination clients { file ("/var/log/fwlogs/netclients/$HOST"); }; destination console_all { file ("/dev/tty12"); }; destination authlog { file ("/var/log/authlog"); }; destination messages { file ("/var/log/messages"); }; destination maillog { file ("/var/log/maillog"); }; destination ipflog { file ("/var/log/ipflog"); }; destination imaplog { file ("/var/log/imaplog"); }; destination console { file ("/dev/console"); }; destination syslog { file ("/var/log/syslog"); }; ### changed to _syslog destination komm { file ("/var/log/fwlogs/kommunikasjon-context.log"); }; destination asa { file ("/var/log/fwlogs/asa.log"); }; destination edb { file ("/var/log/fwlogs/edb-pix.log"); }; destination gje { file ("/var/log/fwlogs/gjerdrum.log"); }; destination nes { file ("/var/log/fwlogs/nes.log"); }; destination ull { file ("/var/log/fwlogs/ullensaker.log"); }; destination hur { file ("/var/log/fwlogs/hurdal.log"); }; destination eid { file ("/var/log/fwlogs/eidsvoll.log"); }; destination nan { file ("/var/log/fwlogs/nannestad.log"); }; destination datas { file ("/var/log/fwlogs/datasenter-context.log"); }; destination mgmt { file ("/var/log/fwlogs/management-context.log"); }; destination wifi { file ("/var/log/fwlogs/wifi-context.log"); }; destination aplog { file ("/var/log/aplog/aplog.txt"); }; filter aplog { host (10.20.); }; ### this line caused the syntax error (line 35) filter gje { host (10.8.30.28); }; filter nes { host (10.8.30.20); }; filter ull { host (10.8.30.36); }; filter hur { host (10.8.30.12); }; filter eid { host (10.8.30.44); }; filter nan { host (10.8.30.52); }; filter datas { host (10.8.0.1); }; filter mgmt { host (10.8.30.60); }; filter wifi { host (10.8.30.92); }; filter asa { host (ASA-R111); }; filter komm { host (10.8.30.76); }; filter edb { host (edb-pix); }; filter emerg { level (emerg); }; filter alert { level (alert .. emerg); }; filter crit { level (crit .. emerg); }; filter err { level (err .. emerg); }; filter warning { level (warning .. emerg); }; filter notice { level (notice .. emerg); }; filter info { level (info .. emerg); }; filter debug { level (debug .. emerg); }; filter kern { facility (kern); }; filter user { facility (user); }; filter mail { facility (mail); }; filter daemon { facility (daemon); }; filter auth { facility (auth); }; filter syslog { facility (syslog); }; filter lpr { facility (lpr); }; filter news { facility (news); }; filter uucp { facility (uucp); }; filter cron { facility (cron); }; #filter local0 { facility (local0); }; #filter local1 { facility (local1); }; #filter local2 { facility (local2); }; #filter local3 { facility (local3); }; #filter local4 { facility (local4); }; #filter local5 { facility (local5); }; #filter local6 { facility (local6); }; #filter local7 { facility (local7); }; filter user_none { not facility (user); }; filter kern_debug { filter (kern) and filter (debug); }; filter daemon_notice { filter (daemon) and filter (notice); }; filter mail_crit { filter (mail) and filter (crit); }; filter mesg { filter (kern_debug) or filter (daemon_notice) or filter (mail_crit); }; filter authinfo { filter (auth) or program (sudo); }; log { source (sys); destination(messages); }; log { source (sys); destination(console_all); }; log { source (sys); filter (authinfo); destination (authlog); }; log { source (sys); filter (mail); destination (maillog); }; #log { source (sys); filter (local0); destination (ipflog); }; #log { source (sys); filter (local1); destination (imaplog); }; log { source (sys); filter (syslog); destination (syslog); }; log { source (sys); filter (emerg); filter (user_none); destination (console); }; log { source (sys); filter (mesg); filter (user_none); destination (messages); }; log { source (net); destination (clients); }; log { source (net); filter (komm); destination (komm); }; log { source (net); filter (asa); destination (asa); }; log { source (net); filter (mgmt); destination (mgmt); }; log { source (net); filter (wifi); destination (wifi); }; log { source (net); filter (datas); destination (datas); }; log { source (net); filter (wifi); destination (wifi); }; log { source (net); filter (gje); destination (gje); }; log { source (net); filter (nan); destination (nan); }; log { source (net); filter (ull); destination (ull); }; log { source (net); filter (hur); destination (hur); }; log { source (net); filter (eid); destination (eid); }; log { source (net); filter (nes); destination (nes); }; log { source (net); filter (edb); destination (edb); }; log { source (net); filter (aplog); destination (aplog); }; #log { source (net); filter (komm); destination (komm); }; ---------------------------------------------------------------------- This is the error i got : syntax error in /etc/syslog-ng/syslog-ng.conf at line 35. syslog-ng documentation: http://www.balabit.com/support/documentation/?product=syslog-ng mailing list: https://lists.balabit.hu/mailman/listinfo/syslog-ng * Configuration error. Please fix your configfile (/etc/syslog-ng/syslog-ng.conf) It is not a very big deal at the moment, as i've reverted back to version 2.1.4, but i'd like to keep my system current, and somewhat up to date. this server's main function is as a syslogger for a few firewalls though, so i cannot leave it unable to log for long. hope someone can help. cheers andip

2 1

multiple destinations
by Ulrich David 14 Dec '09

14 Dec '09

Hi, If I use multiple destinations like : log { source(src); filter(f_52); destination(mysql_52); destination(file_52); flags(final); }; If my first destination is down (a mysql database), does the second destination works correctly ? There is 2 fifo, one for each destinations, but are the 2 destinations working totally independantly? Regards David

2 1

Syslog-NG 3.0.5 incorrectly parsing messages without program names
by Larry Low 11 Dec '09

11 Dec '09

In the past I used the MSG macro to get the full syslog message. Now MSG is broken into MSG and MSGHDR. If a message comes in without a program name such as "exiting on signal 15" or "last message repeated 20 times", syslog-ng incorrectly takes the first word as the program name. Trying to rebuild it with $MSGHDR$MSG causes an erroneous : to be placed on these messages since $MSGHDR is filled in with "exiting" and "last". --- Larry Low 480.385.7045 (w) | llow(a)telesphere.com | www.telesphere.com 9237 E Via de Ventura | Suite 250 | Scottsdale, AZ 85258

1 0

udp packet receive errors
by Paras Fadte 11 Dec '09

11 Dec '09

Hi, When logging to a remote syslog server , I see a lot of udp "packet receive errors" on the remote syslog server . The source machines don't show any such packet loss. I tried increasing net.core.wmem_max to 16MB and changed the log_fifo_size value to 2000 from default 100 on remote syslog server but still I continue to see udp packet receive errors . What could be the issue here ? The incoming request rate is inorder of 50k/sec during peak time. What can be done to minimize these errors? Syslog-ng version is syslog-ng 1.6.8 Thank you. -Paras

3 2

Chroot Occuring Too Early
by Larry Low 11 Dec '09

11 Dec '09

Building out a new syslog-ng server and it seems that in syslog-ng 3.0.5 is chrooting before reading the config file and setting up /dev/log. I ran the same configuration in syslog 2.0.7 without this problem. /opt/syslog-ng/sbin/syslog-ng --chroot /var/log Error opening configuration file; filename='/opt/syslog-ng/etc/syslog-ng.conf', error='No such file or directory (2)' --- Larry Low

2 2

colored messages
by Gaydov Victor 11 Dec '09

11 Dec '09

Hello. I wrote a small patch to add colorization to syslog-ng. The idea is to apply different color schemes to messages (that are going to a tty, for example) depending on their type and importance and using standard syslog-ng filters. IMHO this makes tty log much more readable. There is a new keyword: 'scheme'. "scheme { ... }" construction binds colors to message fields and "scheme()" field of "log {...}" adds scheme filter that colorizes the output (before it will be sent to destination). The filter uses ANSI escape sequences but it's easy to extend it to anything else. Color support is enabled with defining 'ENABLE_COLOR' or using '--enable-colors' in configure script. If you'll find it useful you can use it :) The patch and example (draft) syslog-ng.conf are attached. -- Gaydov Victor. Practically perfect people never permit sentiment to muddle their thinking. -- Mary Poppins

5 6

Syslogd and syslog-ng
by Rocco Scappatura 11 Dec '09

11 Dec '09

Hello, I'm configuring a centralized loghost (SLES 10 SP3) based on syslog-ng, for collecting mail log of different clients. Several machine which are entitled to log its mail logs on the central loghost (Debian), still use old syslogd as logger system. I configured one of this to send its mail logs to the loghost putting: mail.* @syslogmi01.local in /etc/syslog.conf file and restarted the service. Moreover on the loghost I set: source mail_gateway_src { udp(ip("0.0.0.0") port(514)); }; filter ernesto_mail_f { facility(mail); }; destination d_ernesto_mail { file("/var/log/ernesto_mail"); }; log { source(mail_gateway_src); filter(ernesto_mail_f); destination(d_ernesto_mail); }; And restarted it. But no file are created and so no log are saved on the loghost. With tcpdump I see the the log comes on the loghost, but syslog doesn't want to catch them. # tcpdump -vvv host 192.168.252.107 and port 514 tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes 18:04:57.046952 IP (tos 0x0, ttl 64, id 3157, offset 0, flags [DF], proto: UDP (17), length: 358) 192.168.252.107.syslog > syslogmi01.local.syslog: SYSLOG, length: 330 Facility mail (2), Severity info (6) Msg: postfix/smtpd[30946]: NOQUEUE: reject: RCPT from u[|syslog] 18:04:57.046962 IP (tos 0x0, ttl 64, id 3158, offset 0, flags [DF], proto: UDP (17), length: 360) 192.168.252.107.syslog > syslogmi01.local.syslog: SYSLOG, length: 332 I saw that the logs coming from Debian machine (so using syslogd) miss of timestamp and host name that generates host. Could be this a source of incompatibility between syslogd and syslog-ng? Or there is something that I'm wrong? Thanks in advance, rocsca

5 10

re-opening it's logs
by Tim Rupp 11 Dec '09

11 Dec '09

will syslog-ng re-open it's log files if it finds that the size of the log has suddenly decreased or the inode has changed or something similar that may be caused by a logrotate? Or should I just get in the habit of sighup'ing the process? I have a log file on AFS space that seems to have stopped being written to, but log files on the local system are still chugging along just fine, so I think an errant rotate script is taking place on that remote system and moving the file and syslog-ng isn't noticing. My last log entry in the syslog-ng log is this Dec 7 15:56:02 clogger syslog-ng[23479]: Initializing destination file writer; template='/afs/files/data/k5logs/tmp/kdc.log', filename='/afs/files/data/k5logs/tmp/kdc.log' but, and this is my next question, that timestamp is ~8 hours off. It's 9:47 am here and syslog-ng is writing Dec 8 01:38:01 into the syslog-ng.log file. `date` command on the system is reporting [root@clogger tarupp]# date Tue Dec 8 09:43:16 CST 2009 and other log files have a correct date in them. The related log config entries are source s_internal { internal(); }; destination d_syslog { file("/logging/syslog-ng/syslog-ng.log"); }; log { source(s_internal); destination(d_syslog); }; a snippet of the syslog-ng.log file is Dec 8 01:44:01 clogger syslog-ng[23479]: Closing log transport fd; fd='1043' Dec 8 01:44:01 clogger syslog-ng[23479]: Destination timed out, reaping; template='/logging/syslog-ng/$HOST/secure', filename='/logging/syslog-ng/fnd0763/secure' and there are a bunch of these going into the file. The "seconds" in the timestamp also do not change even though time is. For example, that 01 seconds could stay there for a minute or so for each log entry and then a Dec 8 01:45:31 clogger syslog-ng[23479]: Reaping unused destination files; template='/logging/syslog-ng/$HOST/messages' will come along and increment the seconds by 1; not to the new time though. If I had to venture a guess, maybe the time_reap value is too low? (ours is set to the default) and maybe we have such a large number of one-shot hosts that syslog-ng is getting backed up writing to that log file? Any ideas? Thanks, Tim

2 1

Re: [syslog-ng] Macro expansion in program destination
by chris packham 11 Dec '09

11 Dec '09

This is intentional behaviour. Syslog-ng starts any "program()" when syslog-ng is started and it expects that the program will perform read from stdin and block waiting for data. This protects syslog-ng againts the overhead of forking a new process everytime a matching log message is seen. You can still use filters so I'd suggest modifying your program so that it can have the $S_DAY fed to it at the start of each log message. >>> Evan Rempel <erempel(a)uvic.ca> 12/04/09 4:07 PM >>> Is it possible to use macro expansion in the program destination. I wanted to write something like destination mydest{ program("/some/path/to/filter/program -options >> /path/log.$S_DAY"));}; I was hoping that syslog-ng would not open this destination until the first message was ready for this destination. I was hoping that syslog-ng would open a new one when the $S_DAY changed, allowing me to have each day of logs processed by my program with outputs based on day. Does anyone know if macro expantion can be used on program destinations? If not, can someone explain what I am overlooking that makes this a bad thing? Thanks, -- Evan Rempel ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html

4 7

Undefined symbol: yylex, 3.1beta1
by Marvin Nipper 11 Dec '09

11 Dec '09

I decided to try to at least build the 3.1beta1 package, to be sure that there were no new "compilation oddities" introduced. It seems to have fallen over while trying to compile the pdbtool component: gcc -DHAVE_CONFIG_H -I. -I.. -I/usr/local/include/glib-2.0 -I/usr/local/lib/glib-2.0/include -I/usr/local/include/eventlog -I/usr/local/ssl/include -DLIBNET_LIL_ENDIAN -D_GNU_SOURCE -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -g -O2 -Wall -MT pdbtool.o -MD -MP -MF .deps/pdbtool.Tpo -c -o pdbtool.o pdbtool.c mv -f .deps/pdbtool.Tpo .deps/pdbtool.Po gcc -g -O2 -Wall -o pdbtool pdbtool.o libsyslog-ng.a -lpthread -ldoor -lsocket -lrt -lnsl -lfl -L/usr/local/lib -lglib-2.0 -L/usr/local/lib -levtlog -L/usr/local/ssl/lib -lssl -lcrypto -lsocket -lnsl -ldl -lz -lsocket -lnsl -lnet -lresolv -lpthread Undefined first referenced symbol in file yylex /usr/sfw/lib/gcc/i386-pc-solaris2.10/3.4.3/../../../libfl.so ld: fatal: Symbol referencing errors. No output written to pdbtool collect2: ld returned 1 exit status *** Error code 1 make: Fatal error: Command failed for target `pdbtool' Current working directory /var/opt/packages/syslog-ng-3.1beta1/src *** Error code 1 The following command caused the error: failcom='exit 1'; \ for f in x $MAKEFLAGS; do \ case $f in \ *=* | --[!k]*);; \ *k*) failcom='fail=yes';; \ esac; \ done; \ dot_seen=no; \ target=`echo all-recursive | sed s/-recursive//`; \ list='src tests doc contrib debian tgz2build'; for subdir in $list; do \ echo "Making $target in $subdir"; \ if test "$subdir" = "."; then \ dot_seen=yes; \ local_target="$target-am"; \ else \ local_target="$target"; \ fi; \ (CDPATH="${ZSH_VERSION+.}:" && cd $subdir && make $local_target) \ || eval $failcom; \ done; \ if test "$dot_seen" = "no"; then \ make "$target-am" || exit 1; \ fi; test -z "$fail" make: Fatal error: Command failed for target `all-recursive' Current working directory /var/opt/packages/syslog-ng-3.1beta1 *** Error code 1 make: Fatal error: Command failed for target `all' Note that I just built 3.0.5 (today, as well), with no problems at all. I'm not done googling, but at least one site, seems to indicate that this might be a make-related issue: http://mail.opensolaris.org/pipermail/tools-gcc/2006-May/000121.html This is Solaris 10 x86, running the original (albeit patched) gcc/make components. Here's the summary from the configure that preceded the make activities: syslog-ng Open Source Edition 3.1beta1 configured Compiler options: compiler : gcc compiler options : -g -O2 -Wall -I/usr/local/include/glib-2.0 -I/usr/local/lib/glib-2.0/include -I/usr/local/include/eventlog -I/usr/local/ssl/include -DLIBNET_LIL_ENDIAN -D_GNU_SOURCE -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 linker flags : -lpthread prefix : /usr/local linking mode : dynamic Features: Sun STREAMS support : yes Sun Door support : yes Debug symbols : no GCC profiling : no Memtrace : no IPV6 support : yes spoof-source support : yes tcp-wrapper support : no SSL support : yes SQL support : no Linux capability support : no PCRE support : no Env wrapper support : no Not dying (or needing) to get 3.1 running. Just submitting this to try to sort out any compilation problems while this is all still in beta. APOLOGIES (in advance) if I've just done something moronically stupid! Let me know if there's some other input that it would be useful to provide, and/or if you have any thoughts/suggestions as to what might be happening. THANKS! Marvin This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.

2 1

Syslog UDP extra packet?
by Syslog Beginner 11 Dec '09

11 Dec '09

Hello Syslog Experts, I have a question on the syslog-ng. I use syslog-ng 3.0.2. I tried to setup syslog-ng to forward syslog messages to other host. I tested 2 destination driver, udp() and syslog() as per below. test 1: udp("10.x.x.x" port(514)) test 2: syslog("10.x.x.x" transport("udp") port(514)); I found that with udp() driver, syslog-ng just forwards the incoming log messages to external host. No problem. However, with the syslog() driver, I found that syslog-ng generate 2 udp packets packet1... contains only 4 bytes in payload, I think this is message length?? packet2... is the actual syslog udp packet. Is it possible to disable the first packet? this just create the overhead unnecessarily? Please advise. Thanks. Cheers,

2 1

Re: [syslog-ng] Question about verisign certs
by Jimmy McDonald 11 Dec '09

11 Dec '09

The verify works fine. The client gives the following error now that things have been configured to get the verify to return OK tss4s003 syslog-ng[9739]: Syslog connection accepted; fd='8', client='AF_INET(10.139.64.126:3766)', local='AF_INET(xx.xx.xx.xx:8514)' Dec 9 16:24:25 tss4s003 syslog-ng[9739]: Certificate validation failed; subject='OU=Class 3 Public Primary Certification Authority, O=VeriSign\, Inc., C=US', issuer='OU=Class 3 Public Primary Certification Authority, O=VeriSign\, Inc., C=US', error='invalid CA certificate', depth='2' Dec 9 16:24:25 tss4s003 syslog-ng[9739]: SSL error while reading stream; tls_error='SSL routines:SSL3_GET_CLIENT_CERTIFICATE:no certificate returned' Dec 9 16:24:25 tss4s003 syslog-ng[9739]: I/O error occurred while reading; fd='8', error='Connection reset by peer (131)' Dec 9 16:24:25 tss4s003 syslog-ng[9739]: Syslog connection closed; fd='8', client='AF_INET(10.139.64.126:3766)', local='AF_4:27:03 PM: Chris: INET(xx.xx.xx.xx:8514)' Sent from my iPhone... So expect typos. On Dec 6, 2009, at 10:24 AM, Balazs Scheidler <bazsi(a)balabit.hu> wrote: > On Fri, 2009-12-04 at 11:20 -0500, Jimmy McDonald wrote: >> I have syslog-ng installed and configured for mutual authentication. >> When the client was using a self signed cert it worked. The client >> now >> has a verisign cert. >> >> Imported the CA and an intermediate cert and the public cert so the >> chain is complete but the error on authentication says: invalid CA >> certificate, depth=2 >> >> I have tried putting the pub cert in cert.d with the ca and >> intermediate in ca.d. I also tried putting the intermediate in cert.d >> >> I made a hash for the ca and the intermediate. >> >> I'm not really sure what else to try. Any help would be greatly >> appreciated. > > Can you ensure that "openssl verify" validates your cert in case > syslog-ng doesn't? > > -- > Bazsi > > > ______________________________________________________________________________ > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng > Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng > FAQ: http://www.campin.net/syslog-ng/faq.html >

2 1

syslog-ng 3.1 administrator guide draft
by Robert Fekete 09 Dec '09

09 Dec '09

Dear syslog-ng users, A draft version of the 3.1 administrator guide is available at http://people.balabit.hu/frobert/syslog-ng-v3.1admin-guide-en-1ed-draft.pdf Its mainly interesting if you are giving pattern databases a try: Section 8.6.2.3. Creating pattern databases now describes the latest V3 format, and in Appendix 1 you will find a man page for pdbtool. Also, some experimental/unsupported/unchecked/for-testing-only pattern databases are now available at http://www.balabit.com/downloads/files/patterndb-snapshot/, but Marci will write in more detail about that. Have fun! Robert

1 0

syslog-ng OSE 3.0.5 released
by Balazs Scheidler 09 Dec '09

09 Dec '09

Dear syslog-ng users, I've uploaded syslog-ng OSE 3.0.5 to our webservers recently which has some quite important bugfixes. I'd recommend any serious deployment of syslog-ng 3.x to upgrade to this release. The results of the last four months were accumulated into this release. Some statistics for those who like them: 37 files changed, 885 insertions(+), 468 deletions(-) In this timespan, altogether 36 patches were produced by the following people: Anthony Lineham (1) Balazs Scheidler (25) Marton Illes (2) Tamas Pal (3) Tevesz Andras (4) Viktor Tusa (1) Some of the bugreporters and testers also deserve credit, who were: Tamas Monos Sandor Geller Imre Lazar Arkadiusz Miśkiewicz Elan Ruusamäe Thank you for your contribution. Also please note that a new feature release of syslog-ng was published in recent weeks, numbered version 3.1beta1. That beta version has improvements in various areas you might want to check out. We'll only include bugfixes in syslog-ng 3.0.x in the future. The list of changes: 3.0.5 Thu, 03 Dec 2009 18:02:56 +0100 Bugfixes: * Fixed a possible hang that caused syslog-ng to completely stop processing messages, in case SQL destination is enabled in configure. The bug does not need a working SQL destination. * Fixed a possible 100% CPU usage in case an udp destination went down. * Fixed host_override/program_override options that were broken in 3.0.4. * Fixed the SQL destination to avoid dropping the output queue in case a HUP was received. * Fixed a possible segmentation fault when parsing syntactically incorrect patterndb files. * Fixed ESTRING parser to include the ending quote character in the result. * Report an error in case there's no terminating brace in template macro references, e.g. no closing brace in ${macro} format. * Fixed FreeBSD init script to add $prefix/sbin to the PATH environment variable. This problem caused syslog-ng not to start up on boot. * Fixed FreeBSD /dev/klog message reading, follow_freq() needs to default to zero in this case (just like Linux /proc/kmsg). * Fixed a possible segfault in SIGHUP processing if an included file contains a syntax error. * RFC5424 over UDP erroneously required frame length, just like TCP, this is now fixed. * Fixed HP-UX 11iv2 compilation issue related to pread/pwrite compile errors. * The flow-control flag was erroneously enabled if any of the log statements contained either "final" or "fallback". * Fixed rewrite processing, in some cases a rewritten value could be propagated to wrong branches in the log processing tree. Other changes: * Added support for Cisco ASA timestamps. * Added version checks for PCRE in the configure script. * Integrate with system LSB init scripts a bit better, with 3.0.5 the init script status reports should nicely integrate with the host system. Happy logging, -- Bazsi

1 0

Lock up problems when using unix-dgram("/dev/log") type on
by Lance Laursen 09 Dec '09

09 Dec '09

Hello, I'm having problems with machines eventually hanging on all processes that write to /dev/log when using unix-dgram("/dev/log") with Syslog-NG 3.0.4. The servers run fine for a while and hum along as expected. Unfortunately the success does not last, with various programs completely hanging after an undetermined time. Having an existing root shell as this happens allows me to kill syslog-ng, freeing up all locks. Repro'ing this is...well, annoying. I have 300+ servers running this build of syslog-ng fine, all using unix-stream(). The 4 servers that are locking up are the only ones I have running unix-dgram(). Completely fresh ubuntu 8.04 installs with syslog-ng 3.0.4, identical to all other boxes aside from the one syslog-ng option. I've got strace output that is hanging after programs try to write to /dev/log as well. I'm currently doing a repro by running "while true ; do logger -p local0.info ...longest_message_possible... ; sleep 1s ; done" in non-exact science and have managed to pile things up after just over 120 messages, or two minutes. I can still hop around as root, but all programs that try to write to /dev/log pile up. The pile up seems to be log-size/throughput based, not time-based after some rudimentary tests - though it could be something random that is triggering it while my crappy tests are running. My next test plans to have small log messages in very rapid succession. I'm running: # uname -a Linux tny0032 2.6.24-24-generic #1 SMP Tue Jul 7 19:10:36 UTC 2009 x86_64 GNU/Linux # cat /etc/debian_version lenny/sid (ubuntu 8.04) Here's my source definition: # all known message sources source s_all { internal(); unix-dgram("/dev/log"); file("/proc/kmsg" program_override("kernel: ")); }; Here's some strace output that locks after trying to write to /dev/log: # strace su - lance ... ... stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=118, ...}) = 0 socket(PF_FILE, SOCK_DGRAM, 0) = 3 fcntl(3, F_SETFD, FD_CLOEXEC) = 0 connect(3, {sa_family=AF_FILE, path="/dev/log"}, 110) = 0 sendto(3, "<85>Dec 4 02:12:43 su[7086]: pa"..., 148, MSG_NOSIGNAL, NULL, 0 # strace logger -p local0.info lalala produces the same lock-point as above. I thought dgram should be connectionless? I'm not sure how syslog-ng could be locking up resources. Has anyone seen this before? I will continue looking for a better repro case, if anyone has any ideas though shout. I am using unix-dgram solely because it does not break to a new log entry on NewLines. I was encountering a problem where, using unix-stream, lighttpd's multi-line log output was getting broken up into multiple syslog lines. This would have been fine, except when the new line is broken out and made into new log entries, the $hostname and $program fields get stripped out, leaving me with just $date $msg. This basically negated the ability to filter and relay logs effectively. I can elaborate further here if requested, but making unix-stream behave the same as unix-dgram with regards to multi-line log messages would solve all my problems. Thanks, -Lance

3 3

Re: [syslog-ng] Macro expansion in program destination
by chris packham 07 Dec '09

07 Dec '09

> No, this is *not* what syslog-ng does. Syslog-ng does *not* pass the macro value > for $S_DAY do the command line, and thus the 30 does *not* appear at all. I think a barrier to implementing this kind of functionality is that the program is started long before any log messages are received so there is nothing intelligent that the program macro could be expanded to. Starting the program when the first log message is received is a bad idea because 1) there may be an overhead in starting any given program which may mean log messages are lost and 2) if there is anything wrong with your program you won't know until the log message is received (instead of getting some indication when syslog-ng is started/sighup'd) But... I just had a thought on how this could be implemented (in a rather hack-ish manner) assuming the macro expansion _could_ be applied to the program destination (which I realize is not the case right now). One handy behaviour of syslog-ng is that if the program() exits it is restarted (I'm going on memory here so I reserve the right to be completely wrong). Normally program destinations _shouldn't_ exit, but if that program were to exit when syslog-ng comes to restart the program could perform the macro expansion again. I'd still favour and alternative (like parsing in the destination program) but if its an itch you feel like scratching grab the code and go for it. Also just another random thought for a Monday (to be taken with symptoms of caffeine withdrawal in mind). You could perform some trickery with named pipes (see mkfifo(1)). Lets say you wanted program x to receive log messages with some knowledge of the day without needing to do any parsing. You could setup 31 named pipes called /tmp/pipe-1 /tmp/pipe-2.... /tmp/pipe-31 then you could ask syslog-ng to write to a file destination d_pipe ("/tmp/pipe-$S_DAY") external to all this you could start 31 instances of program x each reading from one of the named pipes.

2 1

pipe causing lockup?
by Tim Rupp 07 Dec '09

07 Dec '09

Hi list, I'm hoping someone can explain this to me because it's taken several days of strace'ing and debugging for me to draw this conclusion. I have a source that right now is receiving a relay of syslog from another syslog-ng server, version 1.6.11. The new syslog-ng server is 3.0.4 using the RHEL 5 amd64 rpm. The new server has the following source to receive from the old server source s_net { udp( ip(0.0.0.0) port(514) ); }; I was seeing similar problems as described by Mike in this discussion https://lists.balabit.hu/pipermail/syslog-ng/2009-September/013371.html I strace'd the syslog-ng process, would wait for it to die, and then attempted to see if there were any patterns. In each strace syslog-ng appeared to hand right after processing a write to a fifo; the relevant strace output is below. 25461 write(340, "Dec 3 22:32:32 cmsstor52 sshd[28272]: Authorized to root, krb5 principal host/cmssrv28.fnal.gov(a)FNAL.GOV (krb5_kuserok)\n", 121) = 121 25461 poll([{fd=39, events=POLLOUT}, {fd=6, events=POLLIN}, {fd=3, events=POLLIN}, {fd=10, events=POLLIN}, {fd=5, events=POLLIN}, {fd=8, events=POLLIN}, {fd= 9, events=POLLIN}], 7, 29155) = 1 ([{fd=9, revents=POLLIN}]) 25461 recvfrom(9, "<14>Dec 3 22:32:30 fndhcp1 dhcp_checkd[1679]: [ID 702911 user.info] -- DHCP Watchdog reports: All DHCP registration processes are runni ng --\n", 8192, 0, {sa_family=AF_INET, sin_port=htons(46911), sin_addr=inet_addr("131.225.110.121")}, [16]) = 144 25461 stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=3543, ...}) = 0 25461 poll([{fd=39, events=POLLOUT}], 1, 0) = 0 (Timeout) 25461 lseek(14, 0, SEEK_END) = 1212423178 25461 write(14, "Dec 3 22:32:30 fndhcp1 dhcp_checkd[1679]: [ID 702911 user.info] -- DHCP Watchdog reports: All DHCP registration processes are running --\ n", 140) = 140 25461 lseek(38, 0, SEEK_END) = 641655 25461 write(38, "Dec 3 22:32:30 fndhcp1 dhcp_checkd[1679]: [ID 702911 user.info] -- DHCP Watchdog reports: All DHCP registration processes are running --\ n", 140) = 140 25461 lseek(40, 0, SEEK_END) = 641655 25461 write(40, "Dec 3 22:32:30 fndhcp1 dhcp_checkd[1679]: [ID 702911 user.info] -- DHCP Watchdog reports: All DHCP registration processes are running --\ n", 140) = 140 25461 lseek(41, 0, SEEK_END) = 641655 25461 write(41, "Dec 3 22:32:30 fndhcp1 dhcp_checkd[1679]: [ID 702911 user.info] -- DHCP Watchdog reports: All DHCP registration processes are running --\ n", 140) = 140 fd 9 is the UDP socket. Right after those messages, it would do a bunch of close() calls ... 25461 close(203) = 0 25461 close(150) = 0 25461 close(565) = 0 25461 close(637) = 0 25461 close(517) = 0 ... followed by a poll that, this time, didn't include the UDP socket 25461 poll([{fd=39, events=POLLOUT}, {fd=6, events=POLLIN}, {fd=3, events=POLLIN}, {fd=10, events=POLLIN}, {fd=5, events=POLLIN}, {fd=8, events=POLLIN}], 6, 0) = 0 (Timeout) 25461 poll([{fd=39, events=POLLOUT}], 1, 0) = 0 (Timeout) 25461 lseek(11, 0, SEEK_END) = 107767627 25461 write(11, "Dec 3 22:33:01 clogged syslog-ng[25461]: Reaping unused destination files; template='/logging/syslog-ng/$HOST/messages'\n", 121) = 121 25461 lseek(12, 0, SEEK_END) = 105497311 I don't have flow-control enabled in my config. After several days of head scratching and reading, I narrowed it down to having something to do with my DHCP destinations/filters/logs I have a pipe() destination that points to a fifo destination d_cstdb { pipe("/opt/dhcp_syslog_cst_db"); }; a filter filter f_dhcp { facility(local4,user) and host("fndhcp1*"); }; and the corresponding log statement log { source(s_net); filter(f_dhcp); destination(d_cstdb); }; I've concluded that the problem is caused by the pipe/fifo. On the old server I have something reading from the fifo. On the new server I hadn't stood that process up yet. syslog-ng would start just fine, would even write several entries to that fifo, and then would reach a point where things would hang. It would stop reading from the UDP socket, and just log things like destinations expiring. I looked online at the documentation for FIFOs and linux and noted that 3.0.4 source appears to specify that fifo's be opened non-blocking. So forgive the silly question, but if the fifo is opened non-blocking, why is syslog-ng appearing to block when writing to a fifo that isn't being read from? Also, why would the UDP socket file descriptor be removed from the poll list? An lsof of the syslog-ng process clearly showed that it was still bound to *:syslog Is this is bug or normal operation? Maybe I don't understand linux's behavior in regards to fifos. Thanks for any clarification, Tim

2 4

Question about verisign certs
by Jimmy McDonald 06 Dec '09

06 Dec '09

I have syslog-ng installed and configured for mutual authentication. When the client was using a self signed cert it worked. The client now has a verisign cert. Imported the CA and an intermediate cert and the public cert so the chain is complete but the error on authentication says: invalid CA certificate, depth=2 I have tried putting the pub cert in cert.d with the ca and intermediate in ca.d. I also tried putting the intermediate in cert.d I made a hash for the ca and the intermediate. I'm not really sure what else to try. Any help would be greatly appreciated. Thanks, Jimmy Sent from my iPhone... So expect typos.

2 1

Expected performance
by Christopher Bland 04 Dec '09

04 Dec '09

Hey guys, I know it's hard to compare apples to apples when you start talking about performance and hardware but I would appreciate some feedback. I am currently have a 32Bit HP Proliant DL380 with 2 2.8Ghz cpus and 4G of memory running Fedora 10. At present I have 150 hosts generating between 10-15G worth of logs per day. I dump all of my logs to a Mysql database so that I can use php-syslog-ng. The OS is build on a 1.2T raid 5 disk array. To cut down on I/O I have the database writing to a ext3 filesystem while the rest of the box uses LVM. I have mounted the database partition with noatime and implemented all of the mysqltuner suggestions. My box is performing slow like molasses. My config looks like this: options { sync (0); time_reopen (10); log_fifo_size (1000); chain_hostnames(no); long_hostnames (off); keep_hostname(no); use_dns (yes); dns_cache(yes); use_fqdn (no); create_dirs (no); keep_hostname (yes); }; source s_sys { file ("/proc/kmsg" log_prefix("kernel: ")); unix-stream ("/dev/log"); internal(); }; source s_everything { file ("/proc/kmsg" log_prefix("kernel: ")); unix-stream ("/dev/log"); internal(); udp(); }; destination d_database { pipe("/tmp/mysql.pipe" template("INSERT INTO logs (host, facility, priority, level, tag, datetime, program, msg) VALUES ( '$HOST', '$FACILITY', '$PRIORITY', '$LEVEL', '$TAG', '$YEAR-$MONTH-$DAY $HOUR:$MIN:$SEC', '$PROGRAM', '$MSG' );\n") template-escape(yes)); }; destination d_cons { file("/dev/console"); }; destination d_mesg { file("/var/log/messages"); }; destination d_auth { file("/var/log/secure"); }; destination d_mail { file("/var/log/maillog" sync(10)); }; destination d_spol { file("/var/log/spooler"); }; destination d_boot { file("/var/log/boot.log"); }; destination d_cron { file("/var/log/cron"); }; destination d_mlal { usertty("*"); }; destination d_asa { file("/var/log/syslog-ng/network/asa/$YEAR/$MONTH/$YEAR-$MONTH-$DAY" owner(root) group(staff) perm(0650) dir_perm(0750) create_dirs(yes)); }; #filter f_filter1 { facility(kern); }; filter f_filter2 { level(info..emerg) and not facility(mail,authpriv,cron); }; filter f_filter3 { facility(authpriv); }; filter f_filter4 { facility(mail); }; filter f_filter5 { level(emerg); }; filter f_filter6 { facility(uucp) or (facility(news) and level(crit..emerg)); }; filter f_filter7 { facility(local7); }; filter f_filter8 { facility(cron); }; filter f_asa { host("asain-temp1"); }; log { source(s_sys); filter(f_filter2); destination(d_mesg); }; log { source(s_sys); filter(f_filter3); destination(d_auth); }; log { source(s_sys); filter(f_filter4); destination(d_mail); }; log { source(s_sys); filter(f_filter5); destination(d_mlal); }; log { source(s_sys); filter(f_filter6); destination(d_spol); }; log { source(s_sys); filter(f_filter7); destination(d_boot); }; log { source(s_sys); filter(f_filter8); destination(d_cron); }; log { source(s_everything); filter(f_asa); destination(d_asa); }; log { source(s_everything); destination(d_database); }; my.cnf looks like this: [mysqld] datadir=/var/lib/mysql socket=/var/lib/mysql/mysql.sock user=mysql old_passwords=1 skip-innodb skip-bdb skip-name-resolve table_cache = 128 tmp_table_size = 256M max_heap_table_size = 256M query_cache_size = 128M query_cache_limit = 4M read_rnd_buffer_size = 1M thread_cache_size = 8 [mysqld_safe] log-error=/var/log/mysqld.log pid-file=/var/run/mysqld/mysqld.pid [ndbd] connect-string="nodeid=2;host=localhost:1186" [ndb_mgm] connect-string="host=localhost:1186" My load looks like this while inserting data: Tasks: 177 total, 1 running, 176 sleeping, 0 stopped, 0 zombie Cpu(s): 20.4%us, 2.3%sy, 0.1%ni, 75.9%id, 0.7%wa, 0.0%hi, 0.6%si, 0.0%st Mem: 3634412k total, 3507692k used, 126720k free, 20224k buffers Swap: 8388600k total, 64k used, 8388536k free, 3246292k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 6868 mysql 20 0 165m 20m 5088 S 91.9 0.6 1488:41 mysqld 7679 root 20 0 4140 1932 948 S 15.6 0.1 158:18.24 syslog-ng 7705 root 20 0 8456 1756 1384 S 5.9 0.0 79:18.84 mysql 21296 root 20 0 2556 996 740 R 2.0 0.0 0:00.01 top 1 root 20 0 2012 780 568 S 0.0 0.0 0:01.62 init 2 root 15 -5 0 0 0 S 0.0 0.0 0:00.00 kthreadd 3 root RT -5 0 0 0 S 0.0 0.0 0:00.14 migration/0 4 root 15 -5 0 0 0 S 0.0 0.0 0:17.46 ksoftirqd/0 5 root RT -5 0 0 0 S 0.0 0.0 0:00.00 watchdog/0 6 root RT -5 0 0 0 S 0.0 0.0 0:00.18 migration/1 7 root 15 -5 0 0 0 S 0.0 0.0 0:26.76 ksoftirqd/1 8 root RT -5 0 0 0 S 0.0 0.0 0:00.00 watchdog/1 9 root RT -5 0 0 0 S 0.0 0.0 0:00.13 migration/2 10 root 15 -5 0 0 0 S 0.0 0.0 0:22.02 ksoftirqd/2 11 root RT -5 0 0 0 S 0.0 0.0 0:00.00 watchdog/2 12 root RT -5 0 0 0 S 0.0 0.0 0:00.11 migration/3 13 root 15 -5 0 0 0 S 0.0 0.0 0:11.79 ksoftirqd/3 14 root RT -5 0 0 0 S 0.0 0.0 0:00.00 watchdog/3 15 root 15 -5 0 0 0 S 0.0 0.0 0:04.69 events/0 16 root 15 -5 0 0 0 S 0.0 0.0 0:02.86 events/1 17 root 15 -5 0 0 0 S 0.0 0.0 0:02.80 events/2 18 root 15 -5 0 0 0 S 0.0 0.0 0:02.91 events/3 19 root 15 -5 0 0 0 S 0.0 0.0 0:00.00 khelper 95 root 15 -5 0 0 0 S 0.0 0.0 0:00.00 kintegrityd/0 96 root 15 -5 0 0 0 S 0.0 0.0 0:00.00 kintegrityd/1 97 root 15 -5 0 0 0 S 0.0 0.0 0:00.00 kintegrityd/2 98 root 15 -5 0 0 0 S 0.0 0.0 0:00.00 kintegrityd/3 100 root 15 -5 0 0 0 S 0.0 0.0 0:00.20 kblockd/0 101 root 15 -5 0 0 0 S 0.0 0.0 0:00.14 kblockd/1 102 root 15 -5 0 0 0 S 0.0 0.0 0:00.33 kblockd/2 103 root 15 -5 0 0 0 S 0.0 0.0 0:12.47 kblockd/3 105 root 15 -5 0 0 0 S 0.0 0.0 0:00.00 kacpid 106 root 15 -5 0 0 0 S 0.0 0.0 0:00.00 kacpi_notify The box gets almost unusable when I do a query to retrieve data from the database? Again, I would appreciate any thoughts or suggestions . -Chris Bland

7 8

[patch] syslog-ng 3.0.4 init script doesn't remove stale PID file on startup
by Gordon McAllister 04 Dec '09

04 Dec '09

Hello, I'm running the syslog-ng 3.0.4 RPM on CentOS 5.4 and came across a problem where syslog-ng would not start properly after an unclean shutdown, since the old PID file was still in place. Attached is a patch to the init script supplied with the RPM that attempts to correct this. Regards, ---Gordon

3 3