[syslog-ng] Not able to match on a field in syslog messages from a DataPower Appliance

[Enigma]

Salowitz, Adam (AS.) wrote:
> I am using syslog-ng 1.6.12 to collect logs from an IBM DataPower
> appliance.  The DataPower appliance allows a certain custom string to be
> added to the outgoing syslogs called a Local Identifier.  I am trying to
> use a regex to match on this Local Identifier.  I have used tcpdump to
> capture actual packets and can see that the Local Identifier is in the
> Application layer Syslog message field.
>
> My question is why is match not working when I try to use a filter to
> grab just a particular Local ID?  No logs are caught by my filter.  I
> have tried:
>
>  match ("DP\d+Syslog")
>  match ("DPRASSyslogAudit3")
>  match ("DPRASSyslogAudit")
>  match ("DP")
>
> I have been able to catch other strings in the messages, such as system,
> debug, dpHandler, etc..."
>
> These example packets below are copy and pasted from wireshark by
> clicking on the Syslog message field in the packet details from and
> right clicking -> Copy -> Description.  
>
> * Three Syslog messages from the DataPower appliance that show the Local
> identifier that I want to match against in the fourth field
> (DPRASSyslogAudit3, DP472Syslog).
>
> 	Syslog message: USER.DEBUG: Jul 15 18:23:44 DPRASSyslogAudit3
> [system][debug] trans(383): cpu usage: 57%(10 sec) 57%(1 min) 57%(10
> min) 56%(1 hour) 58%(1 day)\n
>
> 	Syslog message: USER.ERR: Jul 15 18:23:43 DP472Syslog
> [mpgw][error] trans(8152497)[19.x.x.x]: Request processing failed:
> Connection terminated before request headers read\n
>
> 	Syslog message: USER.ERR: Jul 15 18:25:11 DP474Syslog
> [dpHandler_prod_host474][ssl][error] valcred(pubcert):
> trans(8043025)[19.x.x.x]: SSL Proxy Profile 'xgtws': connection error:
> peer did not send a certificate\n
>
> * One syslog messsage from a Cisco device which shows the hostname in
> the fourth field
>
> 	Syslog message: LOCAL7.NOTICE: Jul 15 22:23:48 hostname 2008 Jul
> 15 22:23:46 %SNMP-5-SNMPAUTHFAIL:Authentication failed for message from
> 19.x.x.x\n
>
> Does anyone have any idea why this field might be getting missed or can
> anyone give me some guidance as to where in the source the incoming
> messages are parsed?  I have tried all the macros listed in the code and
> never get the local ID.
>
> #DataPower Appliances Extreme Test
> destination syncDPextest { 
>     file("/logs/syncDPextest/$YEAR$MONTH$DAY.txt" template("$FACILITY
> $FACILITY_NUM $PRIORITY $LEVEL $LEVEL_NUM $TAG $PRI $HOST $DATE
> $FULLHOST_FROM $MESSAGE\n") ); 
> };
>
> user 1 err err 3 0b 11 hostname Jul 16 16:10:59 hostname
> [dpHandler_prod_host472][ssl][error] valcred(pubcert):
> trans(8320177)[19.x.x.x]: SSL Proxy Profile \'xgtws\': connection error:
> peer did not send a certificate
>
> Any help or suggestions whould be appreciated.
>
> Thanks,
>
> Adam
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.campin.net/syslog-ng/faq.html
>
>
>   

This filter:

match ("DP\d+Syslog")


Will not work because \d is a PCRE metacharacter and IIRC syslog-nx uses 
POSIX-extended regular expressions.  The POSIX metastring for \d is 
[:digit:], so your filter would look like this:

match ("DP[:digit:]+Syslog")