February 2003 - syslog-ng - lists.balabit.hu

syslog-ng web-based monitoring tool
by Matthew C Valites 04 Feb '03

04 Feb '03

I'm currently logging a bunch of machines remotely over stunnel via syslog-ng. All loggin on the loghost goes to a mysql database. Does anybody know of a web-app to parse the entries from the database in a nice format? Re-coding is not a problem. Anything that comes close to this would be beneficial, as long as it's open sourced. Thanks in advance, ~Matt

2 1

Re: [syslog-ng]Some Boxes Refuse to Write to syslog-ng host
by Paul Thomas 04 Feb '03

04 Feb '03

The loghost is resolving correctly. I get the following in tcpdump which tells me that the packets are being set to the syslog-ng loghost. root@advil:/tmp# tcpdump dst host plague.anc.net tcpdump: listening on eth0 10:44:39.856806 advil.anc.net.syslog > plague.anc.net.syslog: udp 47 (DF) 10:44:39.856851 advil.anc.net.syslog > plague.anc.net.syslog: udp 37 (DF) 10:45:03.885048 advil.anc.net.syslog > plague.anc.net.syslog: udp 47 (DF) 10:45:03.885090 advil.anc.net.syslog > plague.anc.net.syslog: udp 37 (DF) 10:45:05.334610 advil.anc.net.syslog > plague.anc.net.syslog: udp 47 (DF) 10:45:05.334650 advil.anc.net.syslog > plague.anc.net.syslog: udp 37 (DF) 10:45:06.516617 advil.anc.net.syslog > plague.anc.net.syslog: udp 47 (DF) 10:45:06.516815 advil.anc.net.syslog > plague.anc.net.syslog: udp 37 (DF) 8 packets received by filter 0 packets dropped by kernel There is a firewall between the 2 machines but it isn't blocking this port. I know that because there are other machines are the same subnet that are able to get to the loghost and nothing is showing up in my firewall logs. Any more suggestions? Paul At 11:25 PM 2/3/2003, you wrote: >Message: 7 >To: syslog-ng(a)lists.balabit.hu >Cc: Leonard_Mills(a)corpnet.sel.sony.com >Subject: Re: [syslog-ng]Some Boxes Refuse to Write to syslog-ng host ><5.2.0.9.0.20030203161839.022feaf0(a)127.0.0.1> >Date: Mon, 03 Feb 2003 22:25:54 +0000 >From: Leonard Mills <Leonard_Mills(a)corpnet.sel.sony.com> >Reply-To: syslog-ng(a)lists.balabit.hu > > >You might get a good idea by using > >dig @localhost loghost.domain.com > >If that gives you what you need, then try using >tcpdump from one of the failing hosts after a >kill -HUP on syslogd. > >Hope this helps, > >Len

2 1

syslog-ng 1.5.23 is running but not saving any log entries
by Kai Wang 04 Feb '03

04 Feb '03

In the log file, it showes: Feb 4 09:28:15 lcomm5 syslog-ng[23965]: STATS: dropped 0 Feb 4 09:38:15 lcomm5 syslog-ng[23965]: STATS: dropped 0 Feb 4 09:48:15 lcomm5 syslog-ng[23965]: STATS: dropped 0 Thanks Kai

2 1

RE: [syslog-ng]Some Boxes Refuse to Write to syslog-ng host
by Michael Breton 04 Feb '03

04 Feb '03

> -----Original Message----- > From: Paul Thomas [mailto:pwthoma@anc.net] > Sent: Tuesday, February 04, 2003 11:55 AM > To: syslog-ng(a)lists.balabit.hu > Subject: Re: [syslog-ng]Some Boxes Refuse to Write to syslog-ng host > > > The loghost is resolving correctly. > > I get the following in tcpdump which tells me that the > packets are being > set to the syslog-ng loghost. > > root@advil:/tmp# tcpdump dst host plague.anc.net > tcpdump: listening on eth0 > 10:44:39.856806 advil.anc.net.syslog > plague.anc.net.syslog: > udp 47 (DF) > 10:44:39.856851 advil.anc.net.syslog > plague.anc.net.syslog: > udp 37 (DF) > 10:45:03.885048 advil.anc.net.syslog > plague.anc.net.syslog: > udp 47 (DF) > 10:45:03.885090 advil.anc.net.syslog > plague.anc.net.syslog: > udp 37 (DF) > 10:45:05.334610 advil.anc.net.syslog > plague.anc.net.syslog: > udp 47 (DF) > 10:45:05.334650 advil.anc.net.syslog > plague.anc.net.syslog: > udp 37 (DF) > 10:45:06.516617 advil.anc.net.syslog > plague.anc.net.syslog: > udp 47 (DF) > 10:45:06.516815 advil.anc.net.syslog > plague.anc.net.syslog: > udp 37 (DF) > > 8 packets received by filter > 0 packets dropped by kernel Good. That means they are being sent. > There is a firewall between the 2 machines but it isn't blocking this > port. I know that because there are other machines are the > same subnet > that are able to get to the loghost and nothing is showing up in my > firewall logs. > > Any more suggestions? Now check of the receiving system with tcpdump to see if they are received. Michael Breton Commtel

1 0

Increase accepting AF_UNIX connection value
by Serge Leschinsky 04 Feb '03

04 Feb '03

Dear Sirs. I have been having a lot of messages in log file for last time: > Feb 4 13:07:37 lapa syslog-ng[231]: Error accepting AF_UNIX connection, opened connections: 100, max: 100 > Feb 4 13:07:38 lapa syslog-ng[231]: Error accepting AF_UNIX connection, opened connections: 100, max: 100 > Feb 4 13:08:05 lapa syslog-ng[231]: Error accepting AF_UNIX connection, opened connections: 100, max: 100 > Feb 4 13:08:05 lapa syslog-ng[231]: Error accepting AF_UNIX connection, opened connections: 100, max: 100 > Feb 4 13:08:05 lapa syslog-ng[231]: Error accepting AF_UNIX connection, opened connections: 100, max: 100 If I understand correctly, processes try to open too many sockets. So, I want to ask the community about typical value of this parameter, the reason of lack of socket and where I can increase this value (100) for resolving this problem. Thank you in advance. -- Yours sincerely Serge Leschinsky mailto:fish@artlife.tomsknet.ru Tuesday, February 4, 2003

2 2

Some Boxes Refuse to Write to syslog-ng host
by Paul Thomas 03 Feb '03

03 Feb '03

I've got about 15 solaris 8 boxes and 5 slackware linux boxes. For some reason 4 of the solaris and 2 of the linux boxes will not write to the syslog-ng loghost. All the remotes are exactly the same setup and port 514/udp isn't blocked between any of the remotes and the loghost. Here is the syslog.conf for the host machines *.err;kern.notice;auth.notice @loghost.domain.com *.err;kern.debug;daemon.notice;mail.crit @loghost.domain.com *.alert;kern.err;daemon.err operator *.alert root *.emerg *, @loghost.domain.com If anyone has any idea why these logs are being transferred to the loghost I'm open for ideas. I'm running syslog-ng 1.5.26 Thanks Paul

2 1

Re: [syslog-ng]Please help with simple syslog-ng.conf creation qu estions....
by seberino＠spawar.navy.mil 03 Feb '03

03 Feb '03

Andrew Thanks again for all your help. I combined all your tips to make a simple "just put everything in /var/log/messages for now" version of /etc/syslog-ng/syslog-ng.conf.... ---- source src {unix-stream("/dev/log"); pipe("/proc/kmsg"); internel();}; destination dest {file("/var/log/messages");}; log {source(src); destination(dest);}; ---- Can I ask you a few more questions about this? I called my identifiers "src" and "dest". I assume it does not matter what I call these as long as I use those same identifiers in log line??? Since Red Hat is not set up to use syslog-ng by default, I assume I must remove syslogd and klogd from startup "things to do list" and start /usr/local/sbin/syslog-ng from /etc/rc.d/rc.local myself??? If I want syslog-ng to accept remote messages from remote syslog-ng daemons, must I add something else to syslog-ng.conf above such as item regarding port #514 UDP??? (syslog is tied to this port). I need to ask this because the word "remote" is NOT even mentioned in the syslog-ng docs???!?!? Thanks again, Chris On Mon, Feb 03, 2003 at 08:28:32AM -0500, Hamilton, Andrew wrote: > It depends on what you want to do. I think it is a good idea to run kernel > messages through syslog-ng. I have less overhead and less to worry about. > The source you have is correct. > > Regards, > > Drew > > -----Original Message----- > From: seberino(a)spawar.navy.mil [mailto:seberino@spawar.navy.mil] > Sent: Friday, January 31, 2003 6:59 PM > To: Hamilton, Andrew > Cc: syslog-ng(a)lists.balabit.hu > Subject: Re: [syslog-ng]Please help with simple syslog-ng.conf creation > qu estions.... > > > Andrew > > Thanks! There is lots to learn since config file much > more complicated now. > > Do you run syslog-ng daemon AND klogd too??? > > FAQ says to have syslog-ng handle kernel messages I must > kill klogd and add this to syslog-ng.conf.... > > source src { pipe("/proc/kmsg"); unix-stream("/dev/log"); internel(); }; > > I assume this is a good idea. Perhaps that is what you do??? > > Thanks again, > > Chris > > > On Thu, Jan 30, 2003 at 02:14:27PM -0500, Hamilton, Andrew wrote: > > Nope. But you can do this: > > > > source s_all{ unix-stream("/dev/log"); internal(); }; > > destination d_messages { file("/var/log/messages"); }; > > > > log { source(s_all); destination(d_messages); }; > > > > That should do it. If you are using Solaris you should use > > > > source local { sun-streams("/dev/log" door("/etc/.syslog_door")); > > internal(); }; > > > > Look in the syslog-ng/doc directory for sample configs. > > > > > > Regards, > > > > Drew > > > > -----Original Message----- > > From: seberino(a)spawar.navy.mil [mailto:seberino@spawar.navy.mil] > > Sent: Thursday, January 30, 2003 2:01 PM > > To: syslog-ng(a)lists.balabit.hu > > Subject: [syslog-ng]Please help with simple syslog-ng.conf creation > > questions.... > > > > > > configure;make;make install of syslog-ng source did not create > > /etc/syslog-ng directory with > > /etc/syslog-ng.conf ready to go. > > > > I assume I must create this directory and file > > myself??? > > > > syslog-ng has *tons* of features. Please tell > > me how I can just throw everything into /var/log/messages. > > I don't require fancy stuff just yet. > > > > In old syslog.conf this would be: > > > > *.info /var/log/messages > > > > Will syslog-ng recognize a one line syslog-ng.conf > > containing this old style config??? > > > > thanks, > > > > Chris > > -- > > _______________________________________ > > > > Dr. Christian Seberino > > SPAWAR Systems Center San Diego > > Code 2363 > > 49590 Lassing Road, Room A339 > > San Diego, CA 92152-6147 > > U.S.A. > > > > Phone: (619) 553-7940 > > Fax: (619) 553-1269 > > Email: seberino(a)spawar.navy.mil > > _______________________________________ > > > > _______________________________________________ > > syslog-ng maillist - syslog-ng(a)lists.balabit.hu > > https://lists.balabit.hu/mailman/listinfo/syslog-ng > > Frequently asked questions at http://www.campin.net/syslog-ng/faq.html > > > > _______________________________________________ > > syslog-ng maillist - syslog-ng(a)lists.balabit.hu > > https://lists.balabit.hu/mailman/listinfo/syslog-ng > > Frequently asked questions at http://www.campin.net/syslog-ng/faq.html > > -- > _______________________________________ > > Dr. Christian Seberino > SPAWAR Systems Center San Diego > Code 2872 > San Diego, CA 92152-6147 > U.S.A. > > Phone: (619) 553-9973 > Fax: > Email: seberino(a)spawar.navy.mil > _______________________________________ -- _______________________________________ Dr. Christian Seberino SPAWAR Systems Center San Diego Code 2872 San Diego, CA 92152-6147 U.S.A. Phone: (619) 553-9973 Fax: Email: seberino(a)spawar.navy.mil _______________________________________

1 0

Filtering Large Syslog Messages
by Brian E. Seppanen 03 Feb '03

03 Feb '03

Hi Folks: I have snmptrapd running so that any trap that it receives should be logged to local1. I have a filter taking anything received via local1 to a specific file filter snmptrap {facility(local1);}; destination snmptraps { file("/var/log/snmptraps";}; Unfortunately a number of traps are getting cut off at a specific point, and the remainder of the trap ends up in syslog and not in the proper destination. I'm running syslog-ng 1.4.17, libol 0.2.24, redhat linux 7.2 Jan 29 12:34:43 hostname ts.cerent454Mib.cerent454Objects.cerent454AlarmGroup.cerent454AlarmTable.cer ent454AlarmEntry.cerent454AlarmSlotNumber.1.normalCondition = 0, enterprises.cerent.cerentProducts.c erent454Mib.cerent454Objects.cerent454AlarmGroup.cerent454AlarmTable.cerent454AlarmEntry.cerent454Al armPortNumber.1.normalCondition = 0, enterprises.cerent.cerentProducts.cerent454Mib.cerent454Objects .cerent454AlarmGroup.cerent454AlarmTable.cerent454AlarmEntry.cerent454AlarmLineNumber.1.normalCondit ion = 0 Here's the beginning of the trap that found its way to the proper file Jan 29 12:34:43 hostname snmptrapd[11851]: 192.168.0.1 [192.168.0.1]: Trap system.sysUpTime.0 = Timeticks: (2968512686) 343 days, 13:52:06.86, .iso.org.dod.internet.snmpV2.snmpModules.snmpMIB.snmpMIBObjects.snmpTrap.snmpTrapOID.0 = OID: enterprises.cerent.cerentProducts.cerent454Mib.cerent454Events.cerent454V2Events.normalCondition, enterprises.cerent.cerentProducts.cerent454Mib.cerentCommonObjects.cerentCommonObjectsGroup.cerentNodeTime.0 = 20030129123442D, enterprises.cerent.cerentProducts.cerent454Mib.cerent454Objects.cerent454AlarmGroup.cerent454AlarmTable.cerent454AlarmEntry.cerent454AlarmState.1.normalCondition = administrative(20), enterprises.cerent.cerentProducts.cerent454Mib.cerent454Objects.cerent454AlarmGroup.cerent454AlarmTable.cerent454AlarmEntry.cerent454AlarmObjectType.1.normalCondition = ne(10), enterprises.cerent.cerentProducts.cerent454Mib.cerent454Objects.cerent454AlarmGroup.cerent454AlarmTable.cerent454AlarmEntry.cerent454AlarmObjectIndex.1.normalCondition = 0, enterprises.cerent.cerentProduc All of the message would be coming in via local1 so it's not that a pattern match is failing.. Any ideas? Brian Seppanen seppy(a)chartermi.net 906-228-4226 ext 23

4 3

destination 'program'
by Dmitry Frolov 03 Feb '03

03 Feb '03

Hello, It seems that syslog-ng starts external program to pipe logs through only at startup. The documentation says so too and there's nothing about restart. Syslog-ng only writes error message to internal log if program is terminated. Feb 3 15:48:17 loghost syslog-ng[971]: io.c: do_write: write() failed (errno 32), Broken pipe Feb 3 15:48:17 loghost syslog-ng[971]: pkt_buffer::do_flush(): Error flushing data But, in some cases it's desirable to watch for this external filter program and restart it when nesessary. Program may die suddenly or may need full restart for reconfiguration. I know that syslog-ng restarts external filter on SIGHUP, and sure i could write watcher script for automatic SIGHUPing of syslog-ng, but automatic filter program restart, perhaps with some frequency limitations would be quite useful. Is there any considerations against that? -- Dmitry Frolov, Zenon N.S.P. (095) 250-4629, http://www.zenon.net/

2 2

Re: [syslog-ng]logs getting stomped on
by Brian Landers 02 Feb '03

02 Feb '03

Sorry to keep harping on this guys, but this is a ^%$#@ mystery! Once again, my mail log restarted at around 10:30pm tonight. Lest you think I'm just whining and not making an effort to solve this one, here's what I've come up with: 1) I've checked all of my crontabs and at jobs, and there's nothing that runs around that time. 2) Went through all the long-running processes. Nothing that would interact with syslog-ng or the log files directly. No processes started or restarted around that time in ps. No weird logins around that time that would seem to indicate operator error. 3) My /var/adm/messages file did NOT restart, just the mail logs coming in from the remote SSH tunnels. 4) I changed my syslog-ng.conf to not use macros for the filename, but rather to log to a static filename "current.log". Didn't help. 5) I added a second destination, breaking out the logs by hour: destination inboundlog { file("/system/inbound_mail/logs/current.log"); file("/system/inbound_mail/logs/$YEAR/$MONTH/$DAY/$HOUR.log"); }; current.log had the entire day's logs, and was truncated and restarted at 22:33:04, however the hourly log 2003/01/29/22.log was NOT, it kept right on ticking. 6) total size of messages logged up to 22:33:04 was approximately 29mb, so largefile issues should not be a factor. I'm at a loss here, folks. I'm ready to consult an exorcist. I seem to have a workaround in that my logs split out by hour don't appear to be getting truncated, but I'd love to track this one down. Any suggestions for things I've haven't considered or should I start shopping for goats and black candles? Thanks, Brian P.S. sorry for the threading issues, was reading through the web archives but I've now joined the list since I'm busily replacing Solaris syslog everywhere with syslog-ng! -- "We've heard that a million monkeys at a million keyboards coud produce the complete works of Shakespeare; now, thanks to the Internet, we know that is not true."

2 2