- syslog-ng - lists.balabit.hu

DNS
by Kumaran Babu 09 Dec '05

09 Dec '05

Hi, I've planned to setup a Central Logging system using the great Syslog-NG with MySQL. All is fine but just need a couple of clarifications to optimize performance. Basically, I look at logging close to 40 Network devices which can potentially log as much 10Gigs or more of data. I want to use the DNS hostnames to be displayed when viewed in php-syslog-ng interface so I enabled DNS on syslog-ng conf file. I've changed the nsswitch.conf to look at hosts file and then dns server so that I can populate the individual device details into the hosts file so this server doesnt have to query the DNS server for the host details everytime it receives a log entry. Am I doing the right thing or is there a better way of accomplishing this? Also, planning to archive all of the logs onto files and not to simple files so that I can rotate the old logs DB every month. Again, am I doing the right thing or is there an alternate way to do this? Any suggestions and recommendations are gratefully accepted. -- Kumaran Find your IP @ www.itsyourip.com

2 2

logline corruption
by Erik Williamson 06 Dec '05

06 Dec '05

Hi All, I just 'inherited' a large syslog-ng setup that involves on average 1.5 Terabytes of data generated per day. Logs come from many hundreds of hosts, to one of 10 or more syslog-ng servers (selected via DNS round-robin). From there, it goes straight to an NFS server, without being written to a local disk. There are many destination files on the server, but it is possible that more than one logging server may be attempting to write to the same destination file at the same time. We're seeing corruption where a logline will have another spliced in the middle, parts here, parts there, etc. Not constantly, but enough to give us a headache. Stats on the NFS server show that the cpu can get maxed out, but the netwok pipe is not full. The individual logger servers show that syslog-ng is using about 10% CPU on dual 3gHz P4's I''m thinking that it's the nfs server that cannot cope with multiple sources trying to write to the same file, but would like to throw this out to the masses - do you have any hints as to how I could verify where the source of the problem is? I do realize that this setup is less-than-optimal, and will be taking steps to repair this as time goes on. Alas, I must get corruption down first... then re-architect second. Any hints or suggestions would be greatly appreciated! Thanks Very Much, Erik.

3 3

syslog-ng 1.9.7 & eventlog 0.2.4 released
by Balazs Scheidler 03 Dec '05

03 Dec '05

Hi, I have released the next version of the syslog-ng development releases, this time version 1.9.7. The release can be found at the usual place at http://www.balabit.com/products/syslog-ng/ The NEWS entry is quoted below: 1.9.7 Sat, 03 Dec 2005 16:41:58 +0100 * Reworded documentation on flush_lines & flush_timeout. * Fixed a possible segmentation fault on SIGHUP. * Fixed a timezone normalization problem. * Fixed Solaris support. * Fixed MARK support to emit a single mark message only (it was previously implemented for all sources) * Added Debian packaging files. -- Bazsi

1 0

(2nd attempt) syslog-ng and FreeBSD jails
by markzero 30 Nov '05

30 Nov '05

Hi, Still having problems getting logs out of my FreeBSD jails. No errors are recorded, no clue as to why there are no logs. My config looks like this: --- # $Id: syslog-ng.conf,v 1.3 2005/11/25 11:24:02 mzero Exp $ # CUSTOM # shell config options { owner("syslogng"); group("syslogng"); dir_owner("syslogng"); dir_group("syslogng"); dir_perm(0750); perm(0640); sync(0); stats(1200); create_dirs(yes); keep_hostname(yes); chain_hostnames(no); use_time_recvd(yes); time_reopen(3); use_dns(no); }; #---------------------------------------------------------------------- source log_main { unix-dgram("/var/run/log" owner("syslogng") group("syslogng") perm(0600)); unix-dgram("/var/spool/postfix/dev/log" owner("syslogng") group("syslogng") perm(0600)); unix-dgram("/usr/jail/hosts/ssh/root/var/run/log" owner("syslogng") group("syslogng") perm(0600)); unix-dgram("/usr/jail/hosts/comms/root/var/run/log" owner("syslogng") group("syslogng") perm(0600)); }; source log_internal { internal(); }; #---------------------------------------------------------------------- # chrooted, relative paths destination local_main { file("/local/$YEAR/$MONTH/$DAY-main.log"); }; destination local_internal { file("/local/$YEAR/$MONTH/$DAY-syslog.log"); }; destination tcp_stunnel { tcp("127.0.0.1" localip(127.0.0.1) port(10514) tcp-keep-alive(yes)); }; #---------------------------------------------------------------------- log { source(log_main); destination(local_main); destination(tcp_stunnel); }; log { source(log_internal); destination(local_internal); destination(tcp_stunnel); }; --- The process runs chrooted to /var/log and gives no errors. Logs appear correctly from the host machine, but not from any of the jails. Anybody know what's wrong, is this a bug? M -- pgp: http://www.darklogik.org/pub/pgp/pgp.txt 0160 A46A 9A48 D3B0 C92F B690 17FB 4B72 0207 ED43

2 2

How to uninstall libol and syslog-ng
by Krishnakumar Menon 30 Nov '05

30 Nov '05

Hi, I have installed libol and syslog-ng. Now I would like to know the roll back procedure ie unistalling it. Thanks Krishnakumar __________________________________ Yahoo! Mail - PC Magazine Editors' Choice 2005 http://mail.yahoo.com

2 1

Re: syslog-ng in a freeBSD jail
by markzero 30 Nov '05

30 Nov '05

> >I am running FreeBSD 6.0-RELEASE. I have syslog-ng running on the > >host system and have specified in the config that it should create > >a log socket within the jail (an actual FreeBSD jail(), not a plain > >chroot): > > > > unix-dgram("/usr/jail/hosts/ssh/root/var/run/log" > > owner("syslogng") > > group("syslogng") > > perm(0600)); > > > >The socket is created successfully, I recieve no errors but I also > >recieve no logs from processes running in the jail. Do I also need > >a syslog-ng process running inside the jail? > > No, but you do need to configure your syslog-ng process to read from > "/usr/jail/hosts/ssh/root/var/run/log". Sorry, should have given a bit more detail there really. My config looks like this: --- # $Id: syslog-ng.conf,v 1.3 2005/11/25 11:24:02 mzero Exp $ # CUSTOM # shell config options { owner("syslogng"); group("syslogng"); dir_owner("syslogng"); dir_group("syslogng"); dir_perm(0750); perm(0640); sync(0); stats(1200); create_dirs(yes); keep_hostname(yes); chain_hostnames(no); use_time_recvd(yes); time_reopen(3); use_dns(no); }; #---------------------------------------------------------------------- source log_main { unix-dgram("/var/run/log" owner("syslogng") group("syslogng") perm(0600)); unix-dgram("/var/spool/postfix/dev/log" owner("syslogng") group("syslogng") perm(0600)); unix-dgram("/usr/jail/hosts/ssh/root/var/run/log" owner("syslogng") group("syslogng") perm(0600)); unix-dgram("/usr/jail/hosts/comms/root/var/run/log" owner("syslogng") group("syslogng") perm(0600)); }; source log_internal { internal(); }; #---------------------------------------------------------------------- # chrooted, relative paths destination local_main { file("/local/$YEAR/$MONTH/$DAY-main.log"); }; destination local_internal { file("/local/$YEAR/$MONTH/$DAY-syslog.log"); }; destination tcp_stunnel { tcp("127.0.0.1" localip(127.0.0.1) port(10514) tcp-keep-alive(yes)); }; #---------------------------------------------------------------------- log { source(log_main); destination(local_main); destination(tcp_stunnel); }; log { source(log_internal); destination(local_internal); destination(tcp_stunnel); }; --- Is there some glaring omission here that I'm not aware of? The above looks to me like it should work properly. thanks, mark -- pgp: http://www.darklogik.org/pub/pgp/pgp.txt 0160 A46A 9A48 D3B0 C92F B690 17FB 4B72 0207 ED43

1 0

syslog-ng in a freeBSD jail
by markzero 29 Nov '05

29 Nov '05

Hello. I am running FreeBSD 6.0-RELEASE. I have syslog-ng running on the host system and have specified in the config that it should create a log socket within the jail (an actual FreeBSD jail(), not a plain chroot): unix-dgram("/usr/jail/hosts/ssh/root/var/run/log" owner("syslogng") group("syslogng") perm(0600)); The socket is created successfully, I recieve no errors but I also recieve no logs from processes running in the jail. Do I also need a syslog-ng process running inside the jail? Is there a better solution to this problem? thanks, mark -- pgp: http://www.darklogik.org/pub/pgp/pgp.txt 0160 A46A 9A48 D3B0 C92F B690 17FB 4B72 0207 ED43

2 1

hostname disappear (L4 switch log)
by 김유균 29 Nov '05

29 Nov '05

Hi~ when I use syslog for collecting logs (from network device Route, L2 switch, L4 switch, ) there are no prolbems. => raw log(which is shown in network device) Nov 22 18:46:48 [11.114.116.15.1.2] NOTICE WebOS : real server 211.11.11.111 operational =>log that syslog get Nov 22 18:46:48 [11.114.116.15.1.2] NOTICE WebOS : real server 211.11.11.111 operational nochange!!!!!!!!!!!!!!!!!!!!!! --------------------------- however when I use syslog-ng for collecting logs, the raw log(above) has changed like this~ Nov 22 18:46:48 NOTICE WebOS : real server 211.11.11.111 operational [11.114.116.15.1.2] gone!!!!!!!!!! I think syslog-ng ommit "[11.114.116.15.1.2]" area. Between network device and syslog-ng server , there are no syslog, syslog-ng servers. Do you have same problem? plz help me i use Linux EL4, syslog-ng 1.6.9 <http://www.korea.com> <http://event.korea.com/event/2005/10/fallbooks/>

1 0

syslog-ng, hostname disappear (L4 switch log)
by 김유균 29 Nov '05

29 Nov '05

Hi~ when I use syslog for collecting logs (from network device Route, L2 switch<Cisco>, L4 switch<Alteon>, ) there are no prolbems. => raw log(which is shown in network device) Nov 22 18:46:48 [11.114.116.15.1.2] NOTICE WebOS <slb>: real server 211.11.11.111 operational =>log that syslog get Nov 22 18:46:48 [11.114.116.15.1.2] NOTICE WebOS <slb>: real server 211.11.11.111 operational nochange!!!!!!!!!!!!!!!!!!!!!! --------------------------- however when I use syslog-ng for collecting logs, the raw log(above) has changed like this~ Nov 22 18:46:48 NOTICE WebOS <slb>: real server 211.11.11.111 operational [11.114.116.15.1.2] gone!!!!!!!!!! I think syslog-ng ommit "[11.114.116.15.1.2]" area. Between network device and syslog-ng server , there are no syslog, syslog-ng servers. Do you have same problem? plz help me i use Linux EL4, syslog-ng 1.6.9 <http://www.korea.com> <http://event.korea.com/event/2005/11/dite/>

1 0

Max TCP Connections
by Philip Webster 27 Nov '05

27 Nov '05

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 G'day, I'm currently running syslog-ng 1.6.5 on Red Hat 2.1AS (syslog-ng will be upgraded to 1.6.9 shortly) as a central log server, and I'm having a problem with a growing number of TCP connections. Most hosts log to the central log server via UDP, but there are 7 remote syslog-ng servers using TCP. I currently allow up to 250 connections: source src_tcp { tcp(ip(x.x.x.x) port(514) max-connections(250)); }; but this is reached within about a fortnight. I restart the daemon each week to ensure that it doesn't max out, but figure that there must be a better way. I haven't seen any specific fixes in the last few releases, so is there a particular option I need? Our network is reasonably reliable so I don't think that this is the problem, but there's always a chance ... Any thoughts? Phil -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.7 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFDhoegT2yvGjk/8+MRAoeGAJ42YH4BtcjnIxoggrynxesEeHpp4gCeJT19 gOe3dRE69P3IlPI7BdV39m8= =Pbix -----END PGP SIGNATURE-----

2 2