- syslog-ng - lists.balabit.hu

Re: [syslog-ng]small syslog-ng 1.5.8 bug (Vakantie)
by Erick Staal 08 Aug '01

08 Aug '01

Hallo, Omdat ik tot 27 augustus vakantie vier wil ik graag uw aandacht vragen voor het volgende: Betreft uw mail of afspraak het project RYX: neemt u dan alstublieft contact op met Paul ter Beek voor organisatorische zaken of Maarten Hartsuijker voor technische zaken. Betreft uw mail of afspraak het project Vibase neemt u dan alstublieft contact op met Otte Jaarsma. Voor overige zaken neem ik na mijn vakantie indien gewenst weer contact met u op. mvg, Erick Staal

1 0

Re: [syslog-ng]small syslog-ng 1.5.8 bug (Vakantie)
by Erick Staal 08 Aug '01

08 Aug '01

Hallo, Omdat ik tot 27 augustus vakantie vier wil ik graag uw aandacht vragen voor het volgende: Betreft uw mail of afspraak het project RYX: neemt u dan alstublieft contact op met Paul ter Beek voor organisatorische zaken of Maarten Hartsuijker voor technische zaken. Betreft uw mail of afspraak het project Vibase neemt u dan alstublieft contact op met Otte Jaarsma. Voor overige zaken neem ik na mijn vakantie indien gewenst weer contact met u op. mvg, Erick Staal

1 0

Re: [syslog-ng]small syslog-ng 1.5.8 bug (Vakantie)
by Erick Staal 08 Aug '01

08 Aug '01

Hallo, Omdat ik tot 27 augustus vakantie vier wil ik graag uw aandacht vragen voor het volgende: Betreft uw mail of afspraak het project RYX: neemt u dan alstublieft contact op met Paul ter Beek voor organisatorische zaken of Maarten Hartsuijker voor technische zaken. Betreft uw mail of afspraak het project Vibase neemt u dan alstublieft contact op met Otte Jaarsma. Voor overige zaken neem ik na mijn vakantie indien gewenst weer contact met u op. mvg, Erick Staal

1 0

Re: [syslog-ng]small syslog-ng 1.5.8 bug (Vakantie)
by Erick Staal 08 Aug '01

08 Aug '01

Hallo, Omdat ik tot 27 augustus vakantie vier wil ik graag uw aandacht vragen voor het volgende: Betreft uw mail of afspraak het project RYX: neemt u dan alstublieft contact op met Paul ter Beek voor organisatorische zaken of Maarten Hartsuijker voor technische zaken. Betreft uw mail of afspraak het project Vibase neemt u dan alstublieft contact op met Otte Jaarsma. Voor overige zaken neem ik na mijn vakantie indien gewenst weer contact met u op. mvg, Erick Staal

1 0

Re: [syslog-ng]small syslog-ng 1.5.8 bug (Vakantie)
by Erick Staal 08 Aug '01

08 Aug '01

Hallo, Omdat ik tot 27 augustus vakantie vier wil ik graag uw aandacht vragen voor het volgende: Betreft uw mail of afspraak het project RYX: neemt u dan alstublieft contact op met Paul ter Beek voor organisatorische zaken of Maarten Hartsuijker voor technische zaken. Betreft uw mail of afspraak het project Vibase neemt u dan alstublieft contact op met Otte Jaarsma. Voor overige zaken neem ik na mijn vakantie indien gewenst weer contact met u op. mvg, Erick Staal

1 0

Re: [syslog-ng]small syslog-ng 1.5.8 bug (Vakantie)
by Erick Staal 08 Aug '01

08 Aug '01

Hallo, Omdat ik tot 27 augustus vakantie vier wil ik graag uw aandacht vragen voor het volgende: Betreft uw mail of afspraak het project RYX: neemt u dan alstublieft contact op met Paul ter Beek voor organisatorische zaken of Maarten Hartsuijker voor technische zaken. Betreft uw mail of afspraak het project Vibase neemt u dan alstublieft contact op met Otte Jaarsma. Voor overige zaken neem ik na mijn vakantie indien gewenst weer contact met u op. mvg, Erick Staal

1 0

Re: [syslog-ng]small syslog-ng 1.5.8 bug (Vakantie)
by Erick Staal 08 Aug '01

08 Aug '01

Hallo, Omdat ik tot 27 augustus vakantie vier wil ik graag uw aandacht vragen voor het volgende: Betreft uw mail of afspraak het project RYX: neemt u dan alstublieft contact op met Paul ter Beek voor organisatorische zaken of Maarten Hartsuijker voor technische zaken. Betreft uw mail of afspraak het project Vibase neemt u dan alstublieft contact op met Otte Jaarsma. Voor overige zaken neem ik na mijn vakantie indien gewenst weer contact met u op. mvg, Erick Staal

1 0

Re: [syslog-ng]small syslog-ng 1.5.8 bug (Vakantie)
by Erick Staal 08 Aug '01

08 Aug '01

Hallo, Omdat ik tot 27 augustus vakantie vier wil ik graag uw aandacht vragen voor het volgende: Betreft uw mail of afspraak het project RYX: neemt u dan alstublieft contact op met Paul ter Beek voor organisatorische zaken of Maarten Hartsuijker voor technische zaken. Betreft uw mail of afspraak het project Vibase neemt u dan alstublieft contact op met Otte Jaarsma. Voor overige zaken neem ik na mijn vakantie indien gewenst weer contact met u op. mvg, Erick Staal

1 0

Re: [syslog-ng]small syslog-ng 1.5.8 bug (Vakantie)
by Erick Staal 08 Aug '01

08 Aug '01

Hallo, Omdat ik tot 27 augustus vakantie vier wil ik graag uw aandacht vragen voor het volgende: Betreft uw mail of afspraak het project RYX: neemt u dan alstublieft contact op met Paul ter Beek voor organisatorische zaken of Maarten Hartsuijker voor technische zaken. Betreft uw mail of afspraak het project Vibase neemt u dan alstublieft contact op met Otte Jaarsma. Voor overige zaken neem ik na mijn vakantie indien gewenst weer contact met u op. mvg, Erick Staal

1 0

Re: [syslog-ng]small syslog-ng 1.5.8 bug (Vakantie)
by Erick Staal 08 Aug '01

08 Aug '01

Hallo, Omdat ik tot 27 augustus vakantie vier wil ik graag uw aandacht vragen voor het volgende: Betreft uw mail of afspraak het project RYX: neemt u dan alstublieft contact op met Paul ter Beek voor organisatorische zaken of Maarten Hartsuijker voor technische zaken. Betreft uw mail of afspraak het project Vibase neemt u dan alstublieft contact op met Otte Jaarsma. Voor overige zaken neem ik na mijn vakantie indien gewenst weer contact met u op. mvg, Erick Staal

1 0

Re: [syslog-ng]small syslog-ng 1.5.8 bug (Vakantie)
by Erick Staal 08 Aug '01

08 Aug '01

Hallo, Omdat ik tot 27 augustus vakantie vier wil ik graag uw aandacht vragen voor het volgende: Betreft uw mail of afspraak het project RYX: neemt u dan alstublieft contact op met Paul ter Beek voor organisatorische zaken of Maarten Hartsuijker voor technische zaken. Betreft uw mail of afspraak het project Vibase neemt u dan alstublieft contact op met Otte Jaarsma. Voor overige zaken neem ik na mijn vakantie indien gewenst weer contact met u op. mvg, Erick Staal

1 0

Re: [syslog-ng]small syslog-ng 1.5.8 bug (Vakantie)
by Erick Staal 08 Aug '01

08 Aug '01

Hallo, Omdat ik tot 27 augustus vakantie vier wil ik graag uw aandacht vragen voor het volgende: Betreft uw mail of afspraak het project RYX: neemt u dan alstublieft contact op met Paul ter Beek voor organisatorische zaken of Maarten Hartsuijker voor technische zaken. Betreft uw mail of afspraak het project Vibase neemt u dan alstublieft contact op met Otte Jaarsma. Voor overige zaken neem ik na mijn vakantie indien gewenst weer contact met u op. mvg, Erick Staal

1 0

Re: [syslog-ng]small syslog-ng 1.5.8 bug (Vakantie)
by Erick Staal 08 Aug '01

08 Aug '01

Hallo, Omdat ik tot 27 augustus vakantie vier wil ik graag uw aandacht vragen voor het volgende: Betreft uw mail of afspraak het project RYX: neemt u dan alstublieft contact op met Paul ter Beek voor organisatorische zaken of Maarten Hartsuijker voor technische zaken. Betreft uw mail of afspraak het project Vibase neemt u dan alstublieft contact op met Otte Jaarsma. Voor overige zaken neem ik na mijn vakantie indien gewenst weer contact met u op. mvg, Erick Staal

1 0

NT Eventlog -> UNIX Syslog
by Michael Hargadon 07 Aug '01

07 Aug '01

Since there was a lot of interest in this I figured I'd be nice and provide a link to the software I use. ;) http://www.intersectalliance.com/projects/index.html#BackLog It's free, too.. Regards, Michael Hargadon mhargadon(a)itdepartment.com

1 0

RE: [syslog-ng]Ok, I think I've got it now...
by Nijs, Daniel 06 Aug '01

06 Aug '01

What I am interested in is which software package you are using to allow NT machines to log to a syslog server. I have considered writing the app myself (since I can not find a good free version), but was hoping there is such thing as "free lunch" ;) Best regards, Dan -----Original Message----- From: Michael Hargadon [mailto:MHargadon@ITDepartment.com] Sent: Friday, August 03, 2001 1:34 PM To: 'syslog-ng(a)lists.balabit.hu' Subject: [syslog-ng]Ok, I think I've got it now... Good afternoon. Some time ago I posted a question about matching RegExes and filtering log messages using Syslog-NG. After some investigation, I've determined the following: 1) My regexp's are defined properly 2) Messages are being matched and diverted to the appropriate location 3) The matched messages are still showing up in /var/log/syslog anyways. Since I'm attempting to monitor about fifty Windows NT computers I need some way by which I can filter out the more useless messages (document x was printed, some sucker logged off, etc.). My configuration is as follows: options { long_hostnames(off); sync(0); }; source src { unix-stream("/dev/log"); internal(); udp(); file("/proc/kmsg"); }; destination authlog { file("/var/log/auth.log" owner("root") group("adm") perm(0640)); }; destination syslog { file("/var/log/syslog" owner("root") group("adm") perm(0640)); }; destination cron { file("/var/log/cron.log" owner("root") group("adm") perm(0640)); }; destination daemon { file("/var/log/daemon.log" owner("root") group("adm") perm(0640)); }; destination kern { file("/var/log/kern.log" owner("root") group("adm") perm(0640)); }; destination lpr { file("/var/log/lpr.log" owner("root") group("adm") perm(0640)); }; destination mail { file("/var/log/mail.log" owner("root") group("adm") perm(0640)); }; destination user { file("/var/log/user.log" owner("root") group("adm") perm(0640)); }; destination uucp { file("/var/log/uucp.log" owner("root") group("adm") perm(0640)); }; destination null { file("/var/log/filtered" owner("root") group("root") perm(0666)); }; destination mailinfo { file("/var/log/mail.info" owner("root") group("adm") perm(0640)); }; destination mailwarn { file("/var/log/mail.warn" owner("root") group("adm") perm(0640)); }; destination mailerr { file("/var/log/mail.err" owner("root") group("adm") perm(0640)); }; destination debug { file("/var/log/debug" owner("root") group("adm") perm(0640)); }; destination messages { file("/var/log/messages" owner("root") group("adm") perm(0640)); }; destination console { usertty("root"); }; destination console_all { file("/dev/tty8"); }; destination xconsole { pipe("/dev/xconsole"); }; destination ppp { file("/var/log/ppp.log" owner("root") group("adm") perm(0640)); }; # Filters here filter f_authpriv { facility(auth, authpriv); }; filter f_syslog { not facility(auth, authpriv); }; filter f_cron { facility(cron); }; filter f_daemon { facility(daemon); }; filter f_kern { facility(kern); }; filter f_lpr { facility(lpr); }; filter f_mail { facility(mail); }; filter f_user { facility(user); }; filter f_uucp { facility(uucp); }; filter f_news { facility(news); }; filter f_debug { not facility(auth, authpriv, news, mail); }; filter f_messages { level(info .. warn) and not facility(auth, authpriv, cron, daemon, mail, news); }; filter f_emergency { level(emerg); }; filter f_info { level(info); }; filter f_notice { level(notice); }; filter f_warn { level(warn); }; filter f_crit { level(crit); }; filter f_err { level(err); }; filter f_cnews { level(notice, err, crit) and facility(news); }; filter f_cother { level(debug, info, notice, warn) or facility(daemon, mail); }; filter ppp { facility(local2); }; # Begin NT Event Log Filters Here # Successful Logon # one sample only please filter f_nt0014 { match("POP3 connection accepted from"); }; # Log destinations now please log { source(src); filter(f_nt0014); destination(null); }; log { source(src); filter(f_authpriv); destination(authlog); }; log { source(src); filter(f_syslog); destination(syslog); }; #log { source(src); filter(f_cron); destination(cron); }; log { source(src); filter(f_daemon); destination(daemon); }; log { source(src); filter(f_kern); destination(kern); }; log { source(src); filter(f_lpr); destination(lpr); }; log { source(src); filter(f_mail); destination(mail); }; log { source(src); filter(f_user); destination(user); }; log { source(src); filter(f_uucp); destination(uucp); }; log { source(src); filter(f_mail); filter(f_info); destination(mailinfo); }; log { source(src); filter(f_mail); filter(f_warn); destination(mailwarn); }; log { source(src); filter(f_mail); filter(f_err); destination(mailerr); }; log { source(src); filter(f_news); filter(f_crit); destination(newscrit); }; log { source(src); filter(f_news); filter(f_err); destination(newserr); }; log { source(src); filter(f_news); filter(f_notice); destination(newsnotice); }; log { source(src); filter(f_debug); destination(debug); }; log { source(src); filter(f_messages); destination(messages); }; log { source(src); filter(f_emergency); destination(console); }; #log { source(src); filter(f_cnews); destination(console_all); }; #log { source(src); filter(f_cother); destination(console_all); }; log { source(src); filter(f_cnews); destination(xconsole); }; log { source(src); filter(f_cother); destination(xconsole); }; log { source(src); filter(ppp); destination(ppp); }; Now, if I receive a message as follows: Aug 3 12:55:40 Mail Application 236515 Fri Aug 03 12:53:55 2001 1073818460 MSExchange Pop3 Interface N/A N/A Information xxxxxxx POP3 connection accepted from xxx.xxx.xxx.xx I'll see that message pop up in /var/log/filtered. I'll also see exactly the same message pop up in /var/log/syslog. I know I've got to be overlooking something quite obvious here -- can anyone offer any assistance with this? Regards, Michael Hargadon _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng

2 1

log archiving
by Nate Campi 06 Aug '01

06 Aug '01

I thought I'd share something with the list that I discovered recently. I had been archiving logs for about 70 host using syslog-ng for the last year, and recently we discovered that our ssh gateway had been rooted. I archived the logs by host like this: destination hosts { file("/var/log/HOSTS/$HOST/$FACILITY/$YEAR/$MONTH/$DAY/$FACILITY$YEAR$MONTH$DAY" owner(root) group(root) perm(0600) dir_perm(0700) create_dirs(yes)); }; log { source(src); destination(hosts); }; It's easy to use find and grep for finding pertinent log messages relating to events, but I found that since I was only logging into directories by host, I couldn't easily search ALL logs on a certain date. I could write up scripts to look for certain dates in filenames but that's not as fast and efficient as just seaching a directory with logs from all hosts on just that day. So I did this too: destination times { file("/var/log/TIMES/$YEAR/$MONTH/$DAY/$HOST/$FACILITY$YEAR$MONTH$DAY" owner(root) group(root) perm(0600) dir_perm(0700) create_dirs(yes)); }; log { source(src); destination(times); }; What do people think? I mounted /var/log/HOSTS and /var/log/TIMES each on their own 18 gig ultra2 SCSI 10k RPM drives, on a machine with a single PIII 550, 768 RAM. Performance and disk space aren't an issue, plus I can add another proc and disks (hot swap SCA SCSI). I just want to hear if people think I'm just wasting disk space and I should script something up, or if they like the idea of fast searches by date or hostname. TIA, -- Nate

2 1

RE: [syslog-ng]log archiving
by Hamilton, Andrew Mr RAYTHEON 5 SIG CMD 06 Aug '01

06 Aug '01

Nate, A lot of people treat their logs differently. Some folks use a database they insert their log lines into, some have complicated scripts that process them and spit out HTML on the statistics, some people grep the logs for particular lines and delete them if they aren't there. People use all kinds of different hardware to run their log servers and really there is no one to really tell them that what they are doing is a waste of time or resources. If your logs are important to you and your organization then it really makes no difference to those of us on the outside what you use. I use an Ultra 60 with 2 18 gig scsi drives myself and I log around 1000 devices. So it really all depends on what you think the logs are worth. If it works for you then that is what matters. The system administrators here use a separate script to rotate and archive the logs. And I leave that to them since they are the ones that use them. Your use of the macros is nice though. Regards, Drew -----Original Message----- From: Nate Campi [mailto:nate@campi.cc] Sent: Saturday, August 04, 2001 12:54 AM To: syslog-ng(a)lists.balabit.hu Subject: [syslog-ng]log archiving I thought I'd share something with the list that I discovered recently. I had been archiving logs for about 70 host using syslog-ng for the last year, and recently we discovered that our ssh gateway had been rooted. I archived the logs by host like this: destination hosts { file("/var/log/HOSTS/$HOST/$FACILITY/$YEAR/$MONTH/$DAY/$FACILITY$YEAR$MONTH$ DAY" owner(root) group(root) perm(0600) dir_perm(0700) create_dirs(yes)); }; log { source(src); destination(hosts); }; It's easy to use find and grep for finding pertinent log messages relating to events, but I found that since I was only logging into directories by host, I couldn't easily search ALL logs on a certain date. I could write up scripts to look for certain dates in filenames but that's not as fast and efficient as just seaching a directory with logs from all hosts on just that day. So I did this too: destination times { file("/var/log/TIMES/$YEAR/$MONTH/$DAY/$HOST/$FACILITY$YEAR$MONTH$DAY" owner(root) group(root) perm(0600) dir_perm(0700) create_dirs(yes)); }; log { source(src); destination(times); }; What do people think? I mounted /var/log/HOSTS and /var/log/TIMES each on their own 18 gig ultra2 SCSI 10k RPM drives, on a machine with a single PIII 550, 768 RAM. Performance and disk space aren't an issue, plus I can add another proc and disks (hot swap SCA SCSI). I just want to hear if people think I'm just wasting disk space and I should script something up, or if they like the idea of fast searches by date or hostname. TIA, -- Nate _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng

1 0

RE: [syslog-ng]Ok, I think I've got it now...
by Hamilton, Andrew Mr RAYTHEON 5 SIG CMD 06 Aug '01

06 Aug '01

Michael, Syslog-ng is supposed to do this. As long as the message matches any filter, which in your case it does, syslog-ng will log the message. The only way to prevent this is to make your filters unique. Your configuration looks good but several of your filters will cause your messages to match more than once and will log multiple times. Regards, Drew -----Original Message----- From: Michael Hargadon [mailto:MHargadon@ITDepartment.com] Sent: Friday, August 03, 2001 7:34 PM To: 'syslog-ng(a)lists.balabit.hu' Subject: [syslog-ng]Ok, I think I've got it now... Good afternoon. Some time ago I posted a question about matching RegExes and filtering log messages using Syslog-NG. After some investigation, I've determined the following: 1) My regexp's are defined properly 2) Messages are being matched and diverted to the appropriate location 3) The matched messages are still showing up in /var/log/syslog anyways. Since I'm attempting to monitor about fifty Windows NT computers I need some way by which I can filter out the more useless messages (document x was printed, some sucker logged off, etc.). My configuration is as follows: options { long_hostnames(off); sync(0); }; source src { unix-stream("/dev/log"); internal(); udp(); file("/proc/kmsg"); }; destination authlog { file("/var/log/auth.log" owner("root") group("adm") perm(0640)); }; destination syslog { file("/var/log/syslog" owner("root") group("adm") perm(0640)); }; destination cron { file("/var/log/cron.log" owner("root") group("adm") perm(0640)); }; destination daemon { file("/var/log/daemon.log" owner("root") group("adm") perm(0640)); }; destination kern { file("/var/log/kern.log" owner("root") group("adm") perm(0640)); }; destination lpr { file("/var/log/lpr.log" owner("root") group("adm") perm(0640)); }; destination mail { file("/var/log/mail.log" owner("root") group("adm") perm(0640)); }; destination user { file("/var/log/user.log" owner("root") group("adm") perm(0640)); }; destination uucp { file("/var/log/uucp.log" owner("root") group("adm") perm(0640)); }; destination null { file("/var/log/filtered" owner("root") group("root") perm(0666)); }; destination mailinfo { file("/var/log/mail.info" owner("root") group("adm") perm(0640)); }; destination mailwarn { file("/var/log/mail.warn" owner("root") group("adm") perm(0640)); }; destination mailerr { file("/var/log/mail.err" owner("root") group("adm") perm(0640)); }; destination debug { file("/var/log/debug" owner("root") group("adm") perm(0640)); }; destination messages { file("/var/log/messages" owner("root") group("adm") perm(0640)); }; destination console { usertty("root"); }; destination console_all { file("/dev/tty8"); }; destination xconsole { pipe("/dev/xconsole"); }; destination ppp { file("/var/log/ppp.log" owner("root") group("adm") perm(0640)); }; # Filters here filter f_authpriv { facility(auth, authpriv); }; filter f_syslog { not facility(auth, authpriv); }; filter f_cron { facility(cron); }; filter f_daemon { facility(daemon); }; filter f_kern { facility(kern); }; filter f_lpr { facility(lpr); }; filter f_mail { facility(mail); }; filter f_user { facility(user); }; filter f_uucp { facility(uucp); }; filter f_news { facility(news); }; filter f_debug { not facility(auth, authpriv, news, mail); }; filter f_messages { level(info .. warn) and not facility(auth, authpriv, cron, daemon, mail, news); }; filter f_emergency { level(emerg); }; filter f_info { level(info); }; filter f_notice { level(notice); }; filter f_warn { level(warn); }; filter f_crit { level(crit); }; filter f_err { level(err); }; filter f_cnews { level(notice, err, crit) and facility(news); }; filter f_cother { level(debug, info, notice, warn) or facility(daemon, mail); }; filter ppp { facility(local2); }; # Begin NT Event Log Filters Here # Successful Logon # one sample only please filter f_nt0014 { match("POP3 connection accepted from"); }; # Log destinations now please log { source(src); filter(f_nt0014); destination(null); }; log { source(src); filter(f_authpriv); destination(authlog); }; log { source(src); filter(f_syslog); destination(syslog); }; #log { source(src); filter(f_cron); destination(cron); }; log { source(src); filter(f_daemon); destination(daemon); }; log { source(src); filter(f_kern); destination(kern); }; log { source(src); filter(f_lpr); destination(lpr); }; log { source(src); filter(f_mail); destination(mail); }; log { source(src); filter(f_user); destination(user); }; log { source(src); filter(f_uucp); destination(uucp); }; log { source(src); filter(f_mail); filter(f_info); destination(mailinfo); }; log { source(src); filter(f_mail); filter(f_warn); destination(mailwarn); }; log { source(src); filter(f_mail); filter(f_err); destination(mailerr); }; log { source(src); filter(f_news); filter(f_crit); destination(newscrit); }; log { source(src); filter(f_news); filter(f_err); destination(newserr); }; log { source(src); filter(f_news); filter(f_notice); destination(newsnotice); }; log { source(src); filter(f_debug); destination(debug); }; log { source(src); filter(f_messages); destination(messages); }; log { source(src); filter(f_emergency); destination(console); }; #log { source(src); filter(f_cnews); destination(console_all); }; #log { source(src); filter(f_cother); destination(console_all); }; log { source(src); filter(f_cnews); destination(xconsole); }; log { source(src); filter(f_cother); destination(xconsole); }; log { source(src); filter(ppp); destination(ppp); }; Now, if I receive a message as follows: Aug 3 12:55:40 Mail Application 236515 Fri Aug 03 12:53:55 2001 1073818460 MSExchange Pop3 Interface N/A N/A Information xxxxxxx POP3 connection accepted from xxx.xxx.xxx.xx I'll see that message pop up in /var/log/filtered. I'll also see exactly the same message pop up in /var/log/syslog. I know I've got to be overlooking something quite obvious here -- can anyone offer any assistance with this? Regards, Michael Hargadon _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng

1 0

Ok, I think I've got it now...
by Michael Hargadon 03 Aug '01

03 Aug '01

Good afternoon. Some time ago I posted a question about matching RegExes and filtering log messages using Syslog-NG. After some investigation, I've determined the following: 1) My regexp's are defined properly 2) Messages are being matched and diverted to the appropriate location 3) The matched messages are still showing up in /var/log/syslog anyways. Since I'm attempting to monitor about fifty Windows NT computers I need some way by which I can filter out the more useless messages (document x was printed, some sucker logged off, etc.). My configuration is as follows: options { long_hostnames(off); sync(0); }; source src { unix-stream("/dev/log"); internal(); udp(); file("/proc/kmsg"); }; destination authlog { file("/var/log/auth.log" owner("root") group("adm") perm(0640)); }; destination syslog { file("/var/log/syslog" owner("root") group("adm") perm(0640)); }; destination cron { file("/var/log/cron.log" owner("root") group("adm") perm(0640)); }; destination daemon { file("/var/log/daemon.log" owner("root") group("adm") perm(0640)); }; destination kern { file("/var/log/kern.log" owner("root") group("adm") perm(0640)); }; destination lpr { file("/var/log/lpr.log" owner("root") group("adm") perm(0640)); }; destination mail { file("/var/log/mail.log" owner("root") group("adm") perm(0640)); }; destination user { file("/var/log/user.log" owner("root") group("adm") perm(0640)); }; destination uucp { file("/var/log/uucp.log" owner("root") group("adm") perm(0640)); }; destination null { file("/var/log/filtered" owner("root") group("root") perm(0666)); }; destination mailinfo { file("/var/log/mail.info" owner("root") group("adm") perm(0640)); }; destination mailwarn { file("/var/log/mail.warn" owner("root") group("adm") perm(0640)); }; destination mailerr { file("/var/log/mail.err" owner("root") group("adm") perm(0640)); }; destination debug { file("/var/log/debug" owner("root") group("adm") perm(0640)); }; destination messages { file("/var/log/messages" owner("root") group("adm") perm(0640)); }; destination console { usertty("root"); }; destination console_all { file("/dev/tty8"); }; destination xconsole { pipe("/dev/xconsole"); }; destination ppp { file("/var/log/ppp.log" owner("root") group("adm") perm(0640)); }; # Filters here filter f_authpriv { facility(auth, authpriv); }; filter f_syslog { not facility(auth, authpriv); }; filter f_cron { facility(cron); }; filter f_daemon { facility(daemon); }; filter f_kern { facility(kern); }; filter f_lpr { facility(lpr); }; filter f_mail { facility(mail); }; filter f_user { facility(user); }; filter f_uucp { facility(uucp); }; filter f_news { facility(news); }; filter f_debug { not facility(auth, authpriv, news, mail); }; filter f_messages { level(info .. warn) and not facility(auth, authpriv, cron, daemon, mail, news); }; filter f_emergency { level(emerg); }; filter f_info { level(info); }; filter f_notice { level(notice); }; filter f_warn { level(warn); }; filter f_crit { level(crit); }; filter f_err { level(err); }; filter f_cnews { level(notice, err, crit) and facility(news); }; filter f_cother { level(debug, info, notice, warn) or facility(daemon, mail); }; filter ppp { facility(local2); }; # Begin NT Event Log Filters Here # Successful Logon # one sample only please filter f_nt0014 { match("POP3 connection accepted from"); }; # Log destinations now please log { source(src); filter(f_nt0014); destination(null); }; log { source(src); filter(f_authpriv); destination(authlog); }; log { source(src); filter(f_syslog); destination(syslog); }; #log { source(src); filter(f_cron); destination(cron); }; log { source(src); filter(f_daemon); destination(daemon); }; log { source(src); filter(f_kern); destination(kern); }; log { source(src); filter(f_lpr); destination(lpr); }; log { source(src); filter(f_mail); destination(mail); }; log { source(src); filter(f_user); destination(user); }; log { source(src); filter(f_uucp); destination(uucp); }; log { source(src); filter(f_mail); filter(f_info); destination(mailinfo); }; log { source(src); filter(f_mail); filter(f_warn); destination(mailwarn); }; log { source(src); filter(f_mail); filter(f_err); destination(mailerr); }; log { source(src); filter(f_news); filter(f_crit); destination(newscrit); }; log { source(src); filter(f_news); filter(f_err); destination(newserr); }; log { source(src); filter(f_news); filter(f_notice); destination(newsnotice); }; log { source(src); filter(f_debug); destination(debug); }; log { source(src); filter(f_messages); destination(messages); }; log { source(src); filter(f_emergency); destination(console); }; #log { source(src); filter(f_cnews); destination(console_all); }; #log { source(src); filter(f_cother); destination(console_all); }; log { source(src); filter(f_cnews); destination(xconsole); }; log { source(src); filter(f_cother); destination(xconsole); }; log { source(src); filter(ppp); destination(ppp); }; Now, if I receive a message as follows: Aug 3 12:55:40 Mail Application 236515 Fri Aug 03 12:53:55 2001 1073818460 MSExchange Pop3 Interface N/A N/A Information xxxxxxx POP3 connection accepted from xxx.xxx.xxx.xx I'll see that message pop up in /var/log/filtered. I'll also see exactly the same message pop up in /var/log/syslog. I know I've got to be overlooking something quite obvious here -- can anyone offer any assistance with this? Regards, Michael Hargadon

1 0

$HOST variable
by Chris Sibbitt 03 Aug '01

03 Aug '01

Greetings. I'm having a problem with getting syslog-ng to write out to filenames based on $HOST. My setup is as follows: -I have two machines running syslog-ng -Machine A has a program that sends syslog messages to itself (127.0.0.1) -Machine A has a tcp destination rule like: destination Machine_B{ tcp(10.1.1.1); } -Machine B has a tcp source rule that receives the messages from Machine A -Machine B has a destination rule like: destination host_logs { file("/logs/$HOST.txt"); } The problem is that what I end up with on Machine B is a file called /logs/127.0.0.1 that contains all the messages generated by the program on Machine A. They look like this: Aug 3 09:12:48 127.0.0.1/127.0.0.1/10.1.1.2 myprog[2182]: Test message I've fiddled with setting keep_hostname() to yes and no on Machine B, but to no avail... apparently this doesn't affect the $HOST variable. I know that chain_hostnames(no) fixes the problem (I get a file named 10.1.1.2, which is what I want), but I'd really like to see the whole path the message took because sometimes Machine A will receive messages from others that get forwarded on. Is there any way to accomplish this. In simple terms, I think I'm looking for a way to get the $HOST variable to pick the LAST hostname in the chain instead of the first. Any ideas, or do I need to edit the code? If the later is the case, I'm pretty sure I need to fix something in expand_macro() (in affile.c), if anyone knows exactly how this should be done some help would be appreciated, otherwise I'll dust off my debugger skills and see what I can accomplish. I hope it doesn't come to that though. JetNet Logo Chris Sibbitt Operations Development 106 Schneider Road, Unit C Kanata, Ontario K2K 1Y2 Phone (613) 271-6220, ext228 Fax (613) 271-6229

1 0

Re: [syslog-ng]Re: syslog-ng's memory footprint growing constantly
by Craig J Constantine 03 Aug '01

03 Aug '01

...on 8/2/01 1:47 PM, the following was received from bazsi(a)balabit.hu >> >ok, we have some kind of string leak. what does "hosts.org_dir" in >> >your environment mean? is it a directory which syslog-ng creates, >> >or a filename? or some part of some message? >> >> I have no idea. :) That string isn't in my syslog-ng config, nor >> *anywhere* in the syslogs recorded by syslog-ng today, nor even >> anywhere in the syslog-ng distribution. Might be just a lot over >> strings overlayed in memory... very weird. > >I found hosts.org_dir, it's in the file /lib/libnss_nisplus.so, so I assume >you are using nisplus. syslog-ng resolves a lot of hostnames, isn't it >possible that nisplus is leaking somewhere? > >I've never ran syslog-ng with a computer using nisplus, that might be an >explanation why others didn't encounter this bug. BINGO! NIS+ is the culprit... I adjusted the resolver library setup to not use NIS at all for host resolution. I Restarted my original, non-debugging syslog-ng yesterday afternoon. 12 hours later, it's still a tiny little program just like it should be. :) Thanks Balazs for your help! -Craig

1 0

RE: [syslog-ng]Re: syslog-ng's memory footprint growing constantl y
by Hamilton, Andrew Mr RAYTHEON 5 SIG CMD 03 Aug '01

03 Aug '01

Actually, I ran it with nisplus for a while but it has been some time ago. I didn't have any problems with syslog-ng in connection to nisplus but we had enough problems with nisplus to eventually wind up going back to nis. I have used nis for over a year and don't have any problems using it with syslog-ng. Problems with nisplus would not be surprising. Regards, Drew -----Original Message----- From: Balazs Scheidler [mailto:bazsi@balabit.hu] Sent: Thursday, August 02, 2001 7:48 PM To: Craig J Constantine Cc: syslog-ng(a)lists.balabit.hu Subject: [syslog-ng]Re: syslog-ng's memory footprint growing constantly On Thu, Aug 02, 2001 at 12:05:29PM -0400, Craig J Constantine wrote: > ...on 8/2/01 11:19 AM, the following was received from bazsi(a)balabit.hu > > >ok, we have some kind of string leak. what does "hosts.org_dir" in your > >environment mean? is it a directory which syslog-ng creates, or a filename? > >or some part of some message? > > I have no idea. :) That string isn't in my syslog-ng config, nor > *anywhere* in the syslogs recorded by syslog-ng today, nor even anywhere > in the syslog-ng distribution. Might be just a lot over strings overlayed > in memory... very weird. I found hosts.org_dir, it's in the file /lib/libnss_nisplus.so, so I assume you are using nisplus. syslog-ng resolves a lot of hostnames, isn't it possible that nisplus is leaking somewhere? I've never ran syslog-ng with a computer using nisplus, that might be an explanation why others didn't encounter this bug. -- Bazsi PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1 _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng

1 0

Re: syslog-ng's memory footprint growing constantly
by Balazs Scheidler 02 Aug '01

02 Aug '01

On Thu, Aug 02, 2001 at 12:05:29PM -0400, Craig J Constantine wrote: > ...on 8/2/01 11:19 AM, the following was received from bazsi(a)balabit.hu > > >ok, we have some kind of string leak. what does "hosts.org_dir" in your > >environment mean? is it a directory which syslog-ng creates, or a filename? > >or some part of some message? > > I have no idea. :) That string isn't in my syslog-ng config, nor > *anywhere* in the syslogs recorded by syslog-ng today, nor even anywhere > in the syslog-ng distribution. Might be just a lot over strings overlayed > in memory... very weird. I found hosts.org_dir, it's in the file /lib/libnss_nisplus.so, so I assume you are using nisplus. syslog-ng resolves a lot of hostnames, isn't it possible that nisplus is leaking somewhere? I've never ran syslog-ng with a computer using nisplus, that might be an explanation why others didn't encounter this bug. -- Bazsi PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1

1 0

Re: syslog-ng's memory footprint growing constantly
by Craig J Constantine 02 Aug '01

02 Aug '01

...on 7/27/01 1:02 AM, the following was received from syslog-ng-admin(a)lists.balabit.hu >Date: Thu, 26 Jul 2001 10:25:08 +0200 >From: Balazs Scheidler <bazsi(a)balabit.hu> >To: syslog-ng(a)lists.balabit.hu >Subject: Re: syslog-ng's memory footprint growing constantly > >On Wed, Jul 25, 2001 at 04:17:06PM -0400, Craig J Constantine wrote: >> Hi, >> Basically, the memory size of the process simply grows and grows: >> USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND >> root 13899 0.1 19.3 25412 24764 ? S Jul24 2:05 syslog-ng >> That was snapped after about 4 hours of run time. If I let it run for >> days... well, it once got to 90000 VSZ before the box choked for lack of >> RAM. For gauging how heavily worked syslog-ng is, the line counts for all >> the files it writes (this is one day of logs) are: [rest of my original, snipped] > >There have been memory leaks, but they were fixed in 1.4.8. To help me to >debug your problem, please compile syslog-ng _AND_ libol with --with-debug >enabled (configure time option), and run syslog-ng with --debug --verbose >switches, and save the resulting output. I'm mainly interested in lines >starting with "Objects alive ... garbage collected...". > >If the number after "objects alive" increases then there's some spurious >link within the data structure (the gc frees only unlinked objects), if it >doesn't it is some kind of string data (for example messages). I recompiled libol/syslog-ng from fresh tarballs using the configure flag. Then I fired it up with runtime debug arguments and captured everything to a file. I started the run around 9am, and left it go for a good half an hour. The resulting file was, well, truly huge. 8^) Let me know if you'd like to see it, I can drop it on a server where you can grab it via anon FTP. Likewise if you want to see my config file again. ...around 9:07 syslog-ng was: USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND root 30271 0.3 0.7 1652 1000 pts/1 S 08:54 0:02 ./syslog-ng.debug (ie, tiny :) The "Objects alive" was at 293. ...around 9:37 syslog-ng was: USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND root 30271 0.3 1.1 2104 1452 pts/1 S 08:54 0:09 ./syslog-ng.debug The "Objects alive" was at 299. ...around 10:07: USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND root 30271 0.3 1.4 2552 1900 pts/1 S 08:54 0:16 ./syslog-ng.debug The "Objects alive" was at 299. ...around 10:37: USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND root 30271 0.4 1.9 3124 2472 pts/1 S 08:54 0:24 ./syslog-ng.debug The "Objects alive" was still at 299. ...around 11:07: USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND root 30271 0.4 2.3 3720 3068 pts/1 S 08:54 0:35 ./syslog-ng.debug The "Objects alive" was still at 299. ...looks like the objects count is stable, but the memory size is still growing. I sent it a SIGQUIT and scooped up the core file. It looks like a large portion (more than half) of the image is memory that held plain text messages at some point. Notably the string piece "hosts.org_dir" appears often enough that I visually noticed it. Thoughts? If you want to see the core file, I can make it available via anon FTP as well. -Craig

1 0

RE: [syslog-ng]Adding new facilities
by Hamilton, Andrew Mr RAYTHEON 5 SIG CMD 02 Aug '01

02 Aug '01

Brad, you are correct. It is basically an integer define. regards, Drew -----Original Message----- From: Brad Arlt [mailto:arlt@cpsc.ucalgary.ca] Sent: Wednesday, August 01, 2001 6:57 PM To: syslog-ng(a)lists.balabit.hu Subject: [syslog-ng]Adding new facilities Hello... I am compiling syslog-ng 1.4.12 for Solaris 8, and noticed the IRIX factilities of LOG_AUDIT and LOG_LFMT are not mentioned anywhere in syslog-names.c. Also the facilities of LOG_NTP, LOG_SECURITY, and LOG_CONSOLE from BSD are not mentioned anywhere. I have in the past (and will in this build) manually defined LOG_AUTHPRIV and LOG_FTP for receiving messages from our Linux/BSD machines (AUTHPRIV and FTP are a Linux and BSD thing). If I understand the source code correctly the names are only used to translate configuration file strings to number (e.g. "auth" -> (4<<3)). If I am right, this should mean I can add the above facilities without any problems. Can anyone tell me if this sounds right? I am going to give it a try anyway, but I wanted to see if anyone thought I was crazy, or if there was a better way to do this. ---------------------------------------------------------------------------- __o Bradley Arlt Email: arlt(a)cpsc.ucalgary.ca o__ _ \<_ WWW: www.acs.ucalgary.ca/~bdarlt _>/ _ (_)/(_) -Eat well, sleep peacefully, drink lots, and ride like hell. (_)\(_) _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng

1 0