Filtering Large Syslog Messages
Hi Folks: I have snmptrapd running so that any trap that it receives should be logged to local1. I have a filter taking anything received via local1 to a specific file filter snmptrap {facility(local1);}; destination snmptraps { file("/var/log/snmptraps";}; Unfortunately a number of traps are getting cut off at a specific point, and the remainder of the trap ends up in syslog and not in the proper destination. I'm running syslog-ng 1.4.17, libol 0.2.24, redhat linux 7.2 Jan 29 12:34:43 hostname ts.cerent454Mib.cerent454Objects.cerent454AlarmGroup.cerent454AlarmTable.cer ent454AlarmEntry.cerent454AlarmSlotNumber.1.normalCondition = 0, enterprises.cerent.cerentProducts.c erent454Mib.cerent454Objects.cerent454AlarmGroup.cerent454AlarmTable.cerent454AlarmEntry.cerent454Al armPortNumber.1.normalCondition = 0, enterprises.cerent.cerentProducts.cerent454Mib.cerent454Objects .cerent454AlarmGroup.cerent454AlarmTable.cerent454AlarmEntry.cerent454AlarmLineNumber.1.normalCondit ion = 0 Here's the beginning of the trap that found its way to the proper file Jan 29 12:34:43 hostname snmptrapd[11851]: 192.168.0.1 [192.168.0.1]: Trap system.sysUpTime.0 = Timeticks: (2968512686) 343 days, 13:52:06.86, .iso.org.dod.internet.snmpV2.snmpModules.snmpMIB.snmpMIBObjects.snmpTrap.snmpTrapOID.0 = OID: enterprises.cerent.cerentProducts.cerent454Mib.cerent454Events.cerent454V2Events.normalCondition, enterprises.cerent.cerentProducts.cerent454Mib.cerentCommonObjects.cerentCommonObjectsGroup.cerentNodeTime.0 = 20030129123442D, enterprises.cerent.cerentProducts.cerent454Mib.cerent454Objects.cerent454AlarmGroup.cerent454AlarmTable.cerent454AlarmEntry.cerent454AlarmState.1.normalCondition = administrative(20), enterprises.cerent.cerentProducts.cerent454Mib.cerent454Objects.cerent454AlarmGroup.cerent454AlarmTable.cerent454AlarmEntry.cerent454AlarmObjectType.1.normalCondition = ne(10), enterprises.cerent.cerentProducts.cerent454Mib.cerent454Objects.cerent454AlarmGroup.cerent454AlarmTable.cerent454AlarmEntry.cerent454AlarmObjectIndex.1.normalCondition = 0, enterprises.cerent.cerentProduc All of the message would be coming in via local1 so it's not that a pattern match is failing.. Any ideas? Brian Seppanen seppy@chartermi.net 906-228-4226 ext 23
On Wed, Jan 29, 2003 at 02:19:26PM -0500, Brian E. Seppanen wrote:
Unfortunately a number of traps are getting cut off at a specific point, and the remainder of the trap ends up in syslog and not in the proper destination.
<snip>
All of the message would be coming in via local1 so it's not that a pattern match is failing..
It's that the message is broken into two, and since syslog messages have the priority (facility/severity) info at the start of the message, the second half has no priority info at all. To conform to rfc3164 a syslog daemon has to prepend the "unknown" priority to a message that doesn't have one (13 or user.info). I'm sure this is what syslog-ng does, though I'm too lazy to look and see. Anyways, the point is that you need syslog-ng to *not* break up your large messages. 1024 bytes is the default. A google search turns up proof of my theory: <URL:http://lists.balabit.hu/pipermail/syslog-ng/2002-April/003169.html> ...and another search finds that syslog-ng has an option to address your need: <URL:http://citadelle.intrinsec.com/mailing/current/HTML/ml_syslogng/0697.html> Up your max message size with log_msg_size(). -- Nate Campi http://www.campin.net "To promise not to do a thing is the surest way in the world to make a body want to go and do that very thing." - Samuel Clemens
On Wed, Jan 29, 2003 at 02:19:26PM -0500, Brian E. Seppanen wrote:
Hi Folks:
I have snmptrapd running so that any trap that it receives should be logged to local1. I have a filter taking anything received via local1 to a specific file
filter snmptrap {facility(local1);}; destination snmptraps { file("/var/log/snmptraps";};
Unfortunately a number of traps are getting cut off at a specific point, and the remainder of the trap ends up in syslog and not in the proper destination.
I'm running syslog-ng 1.4.17, libol 0.2.24, redhat linux 7.2
syslog defaults to 1024 byte long messages, but this value is tunable in syslog-ng 1.5 where you can set it to a higher value. options { log_msg_size(8192); }; -- Bazsi PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1
Balazs Scheidler wrote:
On Wed, Jan 29, 2003 at 02:19:26PM -0500, Brian E. Seppanen wrote:
I have snmptrapd running so that any trap that it receives should be logged to local1. I have a filter taking anything received via local1 to a specific file
Unfortunately a number of traps are getting cut off at a specific point, and the remainder of the trap ends up in syslog and not in the proper destination.
We are running snmptrapd and syslog-ng 1.5.x under Solaris 8 and observed exactly the same problem.
syslog defaults to 1024 byte long messages, but this value is tunable in syslog-ng 1.5 where you can set it to a higher value.
options { log_msg_size(8192); };
This doesn't fix the problem for us. It seems that there is a problem in the syslog(3) implementation at least on Solaris. Maybe on Linux, too. This is important, because snmptrapd feeds its messages via syslog(3) to syslog-ng. So syslog-ng never gets the correct message, because its truncated in libc before syslog-ng receive it. Our solution was, to patch snmptrapd to log its messages via a local Unix DGRAM socket and use this socket as message source for syslog-ng. This fix the problem and works pretty fine and very stable for more than one year in our environment. -- Best regards --Andreas Schulze [phone: +49.5246.80.1275, fax: +49.5246.80.2275] | I believe, it was Dennis Ritchie who said something like: | "C is rarely the best language for a given task, | but it's often the second-best". | The implication being that: "[...]" | http://www.ioccc.org/1990/dds.c
participants (4)
-
Andreas Schulze
-
Balazs Scheidler
-
Brian E. Seppanen
-
Nate Campi