- syslog-ng - lists.balabit.hu

[buffering log messages]
by Jürgen Heil 20 Dec '05

20 Dec '05

hi everybody, we are using a central log (debian sarge) server running syslog-ng. all the hosts logging to the log server are running syslog-ng as well. does anybody know wheter syslog-ng on the hosts is able to buffer log messages to the local disk in case the logging server is not available? thank you for your help! juergen

2 1

match(regex) problem
by Staszek Pitucha 20 Dec '05

20 Dec '05

Hello I've tried to setup some filters lately (v1.9.7). There's a problem with regex character ^. For example I want to filter out "Accepted password" and my home ip from sshd logs, so I setup: filter f_sshd { (program("sshd") and not ( (match("Accepted password") and match("from 123\.456\.789\.012")) or // other rules )) or not program("sshd"); }; Works ok, but I want to filter only messages starting with "Accepted password", but "^Accepted password" doesn't work. Yes - I know - don't use ^, so it's faster. Anyway - "^Accepted" should work as a match pattern and it doesn't. (it should match only the "text" part of message, am I right?) TIA

3 4

Syslog-ng shutting down
by Ed Benckert 17 Dec '05

17 Dec '05

Recently, on a semi reliable schedule, our syslog-ng has started shutting down for no apparent reason. 3 nights in a row between 8:12pm and 8:20pm it shuts down. No, it doesnt crash, we get the 'syslog-ng version 1.6.9 going down' then it terminates. Before this, we had a few other times it just shut itself down... Now we havent changed anything on the system, theres no cron job running anywhere near that time... Has anyone else expeienced this? We've be running syslog-ng for 18 months now and this just started a few weeks ago. It seems very intermittent, but then this weekend, almost like clockwork.

2 2

netmask
by Antonio Brown 15 Dec '05

15 Dec '05

Hello All, I was wondering which of the two formats below would you use when filtering using syslog-ng: filter f_pix { match(PIX) and not match(" 1.2.3.4|1.2.3.4|1.2.3.4|1.2.3.4|netmask(" 1.2.3.4/28")|netmask("1.2.3.4/20")|netmask("1.2.3.4/22")|netmask("1.2.3.4/28") ") }; or...... filter f_pix { match(PIX) and not match(1.2.3.4) and not match(1.2.3.4) and not match(1.2.3.4) and not match(1.2.3.4) and not match(netmask("1.2.3.4/28")) and not match(netmask("1.2.3.4/20")) and not match(netmask("1.2.3.4/22")) and not match(netmask("1.2.3.4/28")) }; I am currently using the latter, but it seems as if the individual ips and subnets would only be filtered if ALL is true. Would using "|", like in the first format, check for each individual ip or subnet and filter accordingly? When I say filter I mean, I would like everything except for the individual IPs and subnets specified. I am not certain that this is appropriate format for filtering subnets, please advise.... Thank You, in advance for your assistance!!!

2 2

Re:[syslog-ng] netmask
by Fred Turner 14 Dec '05

14 Dec '05

I will be away from the office from Dec 14, 2005 and will return to the office on Dec 15, 2005. Please call 905-426-4229 (BBi's IT Support Line) for any System related Issues. If you have any urgent issue that needs my attention please contact me on my cell phone. Thank you. ================================================================== NOTE: This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the sender. Please note that any views or opinions presented in this email are solely those of the author and do not necessarily represent those of BBi Enterprises Inc. Finally, the recipient should check this email and any attachments for the presence of viruses. BBi Enterprises Inc. accepts no liability for any damage caused by any virus transmitted by this email.

1 0

DNS
by Kumaran Babu 09 Dec '05

09 Dec '05

Hi, I've planned to setup a Central Logging system using the great Syslog-NG with MySQL. All is fine but just need a couple of clarifications to optimize performance. Basically, I look at logging close to 40 Network devices which can potentially log as much 10Gigs or more of data. I want to use the DNS hostnames to be displayed when viewed in php-syslog-ng interface so I enabled DNS on syslog-ng conf file. I've changed the nsswitch.conf to look at hosts file and then dns server so that I can populate the individual device details into the hosts file so this server doesnt have to query the DNS server for the host details everytime it receives a log entry. Am I doing the right thing or is there a better way of accomplishing this? Also, planning to archive all of the logs onto files and not to simple files so that I can rotate the old logs DB every month. Again, am I doing the right thing or is there an alternate way to do this? Any suggestions and recommendations are gratefully accepted. -- Kumaran Find your IP @ www.itsyourip.com

2 2

logline corruption
by Erik Williamson 06 Dec '05

06 Dec '05

Hi All, I just 'inherited' a large syslog-ng setup that involves on average 1.5 Terabytes of data generated per day. Logs come from many hundreds of hosts, to one of 10 or more syslog-ng servers (selected via DNS round-robin). From there, it goes straight to an NFS server, without being written to a local disk. There are many destination files on the server, but it is possible that more than one logging server may be attempting to write to the same destination file at the same time. We're seeing corruption where a logline will have another spliced in the middle, parts here, parts there, etc. Not constantly, but enough to give us a headache. Stats on the NFS server show that the cpu can get maxed out, but the netwok pipe is not full. The individual logger servers show that syslog-ng is using about 10% CPU on dual 3gHz P4's I''m thinking that it's the nfs server that cannot cope with multiple sources trying to write to the same file, but would like to throw this out to the masses - do you have any hints as to how I could verify where the source of the problem is? I do realize that this setup is less-than-optimal, and will be taking steps to repair this as time goes on. Alas, I must get corruption down first... then re-architect second. Any hints or suggestions would be greatly appreciated! Thanks Very Much, Erik.

3 3

syslog-ng 1.9.7 & eventlog 0.2.4 released
by Balazs Scheidler 03 Dec '05

03 Dec '05

Hi, I have released the next version of the syslog-ng development releases, this time version 1.9.7. The release can be found at the usual place at http://www.balabit.com/products/syslog-ng/ The NEWS entry is quoted below: 1.9.7 Sat, 03 Dec 2005 16:41:58 +0100 * Reworded documentation on flush_lines & flush_timeout. * Fixed a possible segmentation fault on SIGHUP. * Fixed a timezone normalization problem. * Fixed Solaris support. * Fixed MARK support to emit a single mark message only (it was previously implemented for all sources) * Added Debian packaging files. -- Bazsi

1 0

(2nd attempt) syslog-ng and FreeBSD jails
by markzero 30 Nov '05

30 Nov '05

Hi, Still having problems getting logs out of my FreeBSD jails. No errors are recorded, no clue as to why there are no logs. My config looks like this: --- # $Id: syslog-ng.conf,v 1.3 2005/11/25 11:24:02 mzero Exp $ # CUSTOM # shell config options { owner("syslogng"); group("syslogng"); dir_owner("syslogng"); dir_group("syslogng"); dir_perm(0750); perm(0640); sync(0); stats(1200); create_dirs(yes); keep_hostname(yes); chain_hostnames(no); use_time_recvd(yes); time_reopen(3); use_dns(no); }; #---------------------------------------------------------------------- source log_main { unix-dgram("/var/run/log" owner("syslogng") group("syslogng") perm(0600)); unix-dgram("/var/spool/postfix/dev/log" owner("syslogng") group("syslogng") perm(0600)); unix-dgram("/usr/jail/hosts/ssh/root/var/run/log" owner("syslogng") group("syslogng") perm(0600)); unix-dgram("/usr/jail/hosts/comms/root/var/run/log" owner("syslogng") group("syslogng") perm(0600)); }; source log_internal { internal(); }; #---------------------------------------------------------------------- # chrooted, relative paths destination local_main { file("/local/$YEAR/$MONTH/$DAY-main.log"); }; destination local_internal { file("/local/$YEAR/$MONTH/$DAY-syslog.log"); }; destination tcp_stunnel { tcp("127.0.0.1" localip(127.0.0.1) port(10514) tcp-keep-alive(yes)); }; #---------------------------------------------------------------------- log { source(log_main); destination(local_main); destination(tcp_stunnel); }; log { source(log_internal); destination(local_internal); destination(tcp_stunnel); }; --- The process runs chrooted to /var/log and gives no errors. Logs appear correctly from the host machine, but not from any of the jails. Anybody know what's wrong, is this a bug? M -- pgp: http://www.darklogik.org/pub/pgp/pgp.txt 0160 A46A 9A48 D3B0 C92F B690 17FB 4B72 0207 ED43

2 2

How to uninstall libol and syslog-ng
by Krishnakumar Menon 30 Nov '05

30 Nov '05

Hi, I have installed libol and syslog-ng. Now I would like to know the roll back procedure ie unistalling it. Thanks Krishnakumar __________________________________ Yahoo! Mail - PC Magazine Editors' Choice 2005 http://mail.yahoo.com

2 1

Re: syslog-ng in a freeBSD jail
by markzero 30 Nov '05

30 Nov '05

> >I am running FreeBSD 6.0-RELEASE. I have syslog-ng running on the > >host system and have specified in the config that it should create > >a log socket within the jail (an actual FreeBSD jail(), not a plain > >chroot): > > > > unix-dgram("/usr/jail/hosts/ssh/root/var/run/log" > > owner("syslogng") > > group("syslogng") > > perm(0600)); > > > >The socket is created successfully, I recieve no errors but I also > >recieve no logs from processes running in the jail. Do I also need > >a syslog-ng process running inside the jail? > > No, but you do need to configure your syslog-ng process to read from > "/usr/jail/hosts/ssh/root/var/run/log". Sorry, should have given a bit more detail there really. My config looks like this: --- # $Id: syslog-ng.conf,v 1.3 2005/11/25 11:24:02 mzero Exp $ # CUSTOM # shell config options { owner("syslogng"); group("syslogng"); dir_owner("syslogng"); dir_group("syslogng"); dir_perm(0750); perm(0640); sync(0); stats(1200); create_dirs(yes); keep_hostname(yes); chain_hostnames(no); use_time_recvd(yes); time_reopen(3); use_dns(no); }; #---------------------------------------------------------------------- source log_main { unix-dgram("/var/run/log" owner("syslogng") group("syslogng") perm(0600)); unix-dgram("/var/spool/postfix/dev/log" owner("syslogng") group("syslogng") perm(0600)); unix-dgram("/usr/jail/hosts/ssh/root/var/run/log" owner("syslogng") group("syslogng") perm(0600)); unix-dgram("/usr/jail/hosts/comms/root/var/run/log" owner("syslogng") group("syslogng") perm(0600)); }; source log_internal { internal(); }; #---------------------------------------------------------------------- # chrooted, relative paths destination local_main { file("/local/$YEAR/$MONTH/$DAY-main.log"); }; destination local_internal { file("/local/$YEAR/$MONTH/$DAY-syslog.log"); }; destination tcp_stunnel { tcp("127.0.0.1" localip(127.0.0.1) port(10514) tcp-keep-alive(yes)); }; #---------------------------------------------------------------------- log { source(log_main); destination(local_main); destination(tcp_stunnel); }; log { source(log_internal); destination(local_internal); destination(tcp_stunnel); }; --- Is there some glaring omission here that I'm not aware of? The above looks to me like it should work properly. thanks, mark -- pgp: http://www.darklogik.org/pub/pgp/pgp.txt 0160 A46A 9A48 D3B0 C92F B690 17FB 4B72 0207 ED43

1 0

syslog-ng in a freeBSD jail
by markzero 29 Nov '05

29 Nov '05

Hello. I am running FreeBSD 6.0-RELEASE. I have syslog-ng running on the host system and have specified in the config that it should create a log socket within the jail (an actual FreeBSD jail(), not a plain chroot): unix-dgram("/usr/jail/hosts/ssh/root/var/run/log" owner("syslogng") group("syslogng") perm(0600)); The socket is created successfully, I recieve no errors but I also recieve no logs from processes running in the jail. Do I also need a syslog-ng process running inside the jail? Is there a better solution to this problem? thanks, mark -- pgp: http://www.darklogik.org/pub/pgp/pgp.txt 0160 A46A 9A48 D3B0 C92F B690 17FB 4B72 0207 ED43

2 1

hostname disappear (L4 switch log)
by 김유균 29 Nov '05

29 Nov '05

Hi~ when I use syslog for collecting logs (from network device Route, L2 switch, L4 switch, ) there are no prolbems. => raw log(which is shown in network device) Nov 22 18:46:48 [11.114.116.15.1.2] NOTICE WebOS : real server 211.11.11.111 operational =>log that syslog get Nov 22 18:46:48 [11.114.116.15.1.2] NOTICE WebOS : real server 211.11.11.111 operational nochange!!!!!!!!!!!!!!!!!!!!!! --------------------------- however when I use syslog-ng for collecting logs, the raw log(above) has changed like this~ Nov 22 18:46:48 NOTICE WebOS : real server 211.11.11.111 operational [11.114.116.15.1.2] gone!!!!!!!!!! I think syslog-ng ommit "[11.114.116.15.1.2]" area. Between network device and syslog-ng server , there are no syslog, syslog-ng servers. Do you have same problem? plz help me i use Linux EL4, syslog-ng 1.6.9 <http://www.korea.com> <http://event.korea.com/event/2005/10/fallbooks/>

1 0

syslog-ng, hostname disappear (L4 switch log)
by 김유균 29 Nov '05

29 Nov '05

Hi~ when I use syslog for collecting logs (from network device Route, L2 switch<Cisco>, L4 switch<Alteon>, ) there are no prolbems. => raw log(which is shown in network device) Nov 22 18:46:48 [11.114.116.15.1.2] NOTICE WebOS <slb>: real server 211.11.11.111 operational =>log that syslog get Nov 22 18:46:48 [11.114.116.15.1.2] NOTICE WebOS <slb>: real server 211.11.11.111 operational nochange!!!!!!!!!!!!!!!!!!!!!! --------------------------- however when I use syslog-ng for collecting logs, the raw log(above) has changed like this~ Nov 22 18:46:48 NOTICE WebOS <slb>: real server 211.11.11.111 operational [11.114.116.15.1.2] gone!!!!!!!!!! I think syslog-ng ommit "[11.114.116.15.1.2]" area. Between network device and syslog-ng server , there are no syslog, syslog-ng servers. Do you have same problem? plz help me i use Linux EL4, syslog-ng 1.6.9 <http://www.korea.com> <http://event.korea.com/event/2005/11/dite/>

1 0

Max TCP Connections
by Philip Webster 27 Nov '05

27 Nov '05

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 G'day, I'm currently running syslog-ng 1.6.5 on Red Hat 2.1AS (syslog-ng will be upgraded to 1.6.9 shortly) as a central log server, and I'm having a problem with a growing number of TCP connections. Most hosts log to the central log server via UDP, but there are 7 remote syslog-ng servers using TCP. I currently allow up to 250 connections: source src_tcp { tcp(ip(x.x.x.x) port(514) max-connections(250)); }; but this is reached within about a fortnight. I restart the daemon each week to ensure that it doesn't max out, but figure that there must be a better way. I haven't seen any specific fixes in the last few releases, so is there a particular option I need? Our network is reasonably reliable so I don't think that this is the problem, but there's always a chance ... Any thoughts? Phil -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.7 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFDhoegT2yvGjk/8+MRAoeGAJ42YH4BtcjnIxoggrynxesEeHpp4gCeJT19 gOe3dRE69P3IlPI7BdV39m8= =Pbix -----END PGP SIGNATURE-----

2 2

Solaris functionality fixed in the 1.9.x branch
by Balazs Scheidler 25 Nov '05

25 Nov '05

Hi, I'm writing this email to inform you that I've fixed up the Solaris STREAMS based logging in the development series. Now it should work on Solaris again. The snapshot for tomorrow will contain the fix. -- Bazsi

1 0

I want to avoid some messages...please suggest.
by Atif Munir 25 Nov '05

25 Nov '05

There are some unwanted messages on my syslog-php web page........Is there a "19510 syslog-ng notice 2005-11-24 11:53:24 syslog-ng[24047]: STATS: dropped 0 " way that i can avoid them... Thanks in advance. atif

3 2

how to avoid such messages
by Atif Munir 25 Nov '05

25 Nov '05

Please help me i am in great trouble......I am getting such "syslog-ng[24047]: STATS: dropped 0" messages in the syslog-ng server and i want to avoid them....please let me know any way to avoid them...thax in advance. atif

1 0

log-rotation: how to write rotated files to another directory
by Oliver Lang 25 Nov '05

25 Nov '05

Hi list, using the macros $MONTH, $WEEK and $DAY it is possible to rotate the logfiles. The rotated files are written to the defined working directory. Question: Is it possible to define another directory for these rotated logs ?? The working directory should contain the current logfiles only. Hope you can help. Kind Regards Oliver Environemnt: syslog-ng 1.6.8 on RHEL ES 4 __________________________________ Yahoo! Music Unlimited Access over 1 million songs. Try it free. http://music.yahoo.com/unlimited/

2 1

syslog-ng 1.6.9 released
by Balazs Scheidler 23 Nov '05

23 Nov '05

Hi, I have released a new maintenance release from the syslog-ng 1.6.x branch, with version number 1.6.9. The list of user-visible changes is below. The most important change is probably the added workaround for lose-a-message when the TCP connection is broken. The updated version can be found on the syslog-ng homepage at: http://www.balabit.com/products/syslog-ng/ News for the 1.6.9 release Wed, 23 Nov 2005 09:52:30 +0100 * Fixed usertty destination, names in utmp records are not NUL terminated. This might cause some logged in users not to be found. * Fixed timezone calculation during daylight saving. * Fixed a warning message in file destinations which indicated that permissions would be changed even if no permission change was requested. * Added a workaround for lose-one-message when connection is broken. * Owner and group information on files is set independently, so one can work while the other fails, this fixes the behaviour when syslog-ng is running as non-root and only the group information is specified. * Documentation updates. * Updated bundled libol to 0.3.17 -- Bazsi

1 1

IPv6 support
by Frederic Beck 23 Nov '05

23 Nov '05

Hello I'm looking for a syslog daemon with IPv6 support. By asking google, it seemed that msyslog had this support, but it's not the case, and it's the same for syslog-ng, in the ML it is said to being working, but i didn't manage to make it work. Does anyone know such a daemon or how to use syslog-ng with IPv6 ? I tried several statements in the configuration file, using DNS names, "...", "[....]", but none of these did work, when syslog-ng starts it only uses IPv4 addresses Thanks a lot Fred

2 1

Compiling libol on Solaris 10 SPARC
by Dave Knoch 23 Nov '05

23 Nov '05

Attempting to compile libol 3.16 on Solaris 10, I receive the following error: *** Error code 1 make: Fatal error: Command failed for target `libol.la' Current working directory /tmp/libol-0.3.16/src *** Error code 1 make: Fatal error: Command failed for target `all' Current working directory /tmp/libol-0.3.16/src *** Error code 1 The following command caused the error: set fnord $MAKEFLAGS; amf=$2; \ dot_seen=no; \ target=`echo all-recursive | sed s/-recursive//`; \ list='utils src'; for subdir in $list; do \ echo "Making $target in $subdir"; \ if test "$subdir" = "."; then \ dot_seen=yes; \ local_target="$target-am"; \ else \ local_target="$target"; \ fi; \ (cd $subdir && make $local_target) \ || case "$amf" in *=*) exit 1;; *k*) fail=yes;; *) exit 1;; esac; \ done; \ if test "$dot_seen" = "no"; then \ make "$target-am" || exit 1; \ fi; test -z "$fail" make: Fatal error: Command failed for target `all-recursive' Any suggestions? Dave

2 1

syslogng+logcheck
by Metal Gear 22 Nov '05

22 Nov '05

hi guys currently i am testing logwatch, logcheck and newlogcheck to generated reports from log files generated by syslog-ng. The log files are generated in the format /var/log/syslog-ng/host1/date1/messages /var/log/syslog-ng/host1/date2/messages /var/log/syslog-ng/host1/date3/messages /var/log/syslog-ng/host2/date1/messages /var/log/syslog-ng/host2/date2/messages * * /var/log/syslog-ng/host*/date*/messages so i played around with logcheck and newlogcheck and it worked fine for single file. According to the README provided by logcheck it will work fine for a single logfile. So is there a way to run it against multiple logfiles according to dates and hosts. Regards

4 5

Full hostname available when tunnelling through stunnel?
by Alexi Papaleonardos 22 Nov '05

22 Nov '05

Hi folks. We have some of our more important hosts reporting our central syslog-ng servers via stunnel. The downside seems to be that only the first "word" (unqualified) of the hostname makes its way to the loghost. Because of an well-intentioned but annoying host naming policy, we have a number of hosts named things like web002.abc and web002.xyz. It would seem that this causes log files to become intermingled. Here is the line we are using in one of the destination statements: file("/var/log/HOSTS/$HOST/$YEAR/$MONTH/$DAY/$FACILITY$YEAR$MONTH$DAY" so both web002.abc and web002.xyz end up going to /var/log/HOSTS/web002/ . We tried using $FULLHOST but instead of getting the value of hostname -f (at least on linux), like we hoped, we got something very strange, like s_sys@hostname, which evidently is trying to describe the chain of hosts the log message passed through (though we have no hosts named s or sys). Does anyone have experience or suggestions for getting around this problem? Or perhaps there's a solution in a newer version of syslog-ng. We are using syslog-ng 1.6.5 and libol 0.3.14. Thanks! Alexi

2 1

sun streams in 1.9.6
by John.Dickinson＠nominet.org.uk 22 Nov '05

22 Nov '05

Hi, Can anyone tell me the current state of sun streams support in 1.9.6? There is an old mention of it being missing in the NEWS file and I can not get it to work - but that might just be me :) I configured like this ./configure --prefix=/opt/syslog-ng2 --with-glib=/opt/glib --with-eventlog=/opt/eventlog --enable-sun-streams --enable-sun-door --enable-dynamic-linking When I try to run it I get STREAMS support not compiled in; Segmentation Fault Thanks John

3 6