[syslog-ng] Not able to match on a field in syslog messages from a DataPower Appliance

[Fegan, Joe]

USER.DEBUG: Jul 15 18:23:44 DPRASSyslogAudit3 [system][debug] trans(383): cpu usage: 57%(10 sec) 57%(1 min) 57%(10
min) 56%(1 hour) 58%(1 day)

In syslog wire protocol the field immediately after the timestamp is the hostname, so this message would be interpreted as being from a host called DPRASSyslogAudit3. Your filters are not matching these messages because match() only matches elements of the message body and the text you're looking for does not appear in the message body.

Joe.

-----Original Message-----
From: syslog-ng-bounces at lists.balabit.hu [mailto:syslog-ng-bounces at lists.balabit.hu] On Behalf Of Salowitz, Adam (AS.)
Sent: 16 July 2008 21:14
To: syslog-ng at lists.balabit.hu
Subject: [syslog-ng] Not able to match on a field in syslog messages from a DataPower Appliance

I am using syslog-ng 1.6.12 to collect logs from an IBM DataPower
appliance.  The DataPower appliance allows a certain custom string to be
added to the outgoing syslogs called a Local Identifier.  I am trying to
use a regex to match on this Local Identifier.  I have used tcpdump to
capture actual packets and can see that the Local Identifier is in the
Application layer Syslog message field.

My question is why is match not working when I try to use a filter to
grab just a particular Local ID?  No logs are caught by my filter.  I
have tried:

 match ("DP\d+Syslog")
 match ("DPRASSyslogAudit3")
 match ("DPRASSyslogAudit")
 match ("DP")

I have been able to catch other strings in the messages, such as system,
debug, dpHandler, etc..."

These example packets below are copy and pasted from wireshark by
clicking on the Syslog message field in the packet details from and
right clicking -> Copy -> Description.

* Three Syslog messages from the DataPower appliance that show the Local
identifier that I want to match against in the fourth field
(DPRASSyslogAudit3, DP472Syslog).

        Syslog message: USER.DEBUG: Jul 15 18:23:44 DPRASSyslogAudit3
[system][debug] trans(383): cpu usage: 57%(10 sec) 57%(1 min) 57%(10
min) 56%(1 hour) 58%(1 day)\n

        Syslog message: USER.ERR: Jul 15 18:23:43 DP472Syslog
[mpgw][error] trans(8152497)[19.x.x.x]: Request processing failed:
Connection terminated before request headers read\n

        Syslog message: USER.ERR: Jul 15 18:25:11 DP474Syslog
[dpHandler_prod_host474][ssl][error] valcred(pubcert):
trans(8043025)[19.x.x.x]: SSL Proxy Profile 'xgtws': connection error:
peer did not send a certificate\n

* One syslog messsage from a Cisco device which shows the hostname in
the fourth field

        Syslog message: LOCAL7.NOTICE: Jul 15 22:23:48 hostname 2008 Jul
15 22:23:46 %SNMP-5-SNMPAUTHFAIL:Authentication failed for message from
19.x.x.x\n

Does anyone have any idea why this field might be getting missed or can
anyone give me some guidance as to where in the source the incoming
messages are parsed?  I have tried all the macros listed in the code and
never get the local ID.

#DataPower Appliances Extreme Test
destination syncDPextest {
    file("/logs/syncDPextest/$YEAR$MONTH$DAY.txt" template("$FACILITY
$FACILITY_NUM $PRIORITY $LEVEL $LEVEL_NUM $TAG $PRI $HOST $DATE
$FULLHOST_FROM $MESSAGE\n") );
};

user 1 err err 3 0b 11 hostname Jul 16 16:10:59 hostname
[dpHandler_prod_host472][ssl][error] valcred(pubcert):
trans(8320177)[19.x.x.x]: SSL Proxy Profile \'xgtws\': connection error:
peer did not send a certificate

Any help or suggestions whould be appreciated.

Thanks,

Adam
______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.campin.net/syslog-ng/faq.html