[syslog-ng] Debugging Pattern Match Failures

[Balazs Scheidler]

On Mon, 2010-08-02 at 22:29 -0700, Matthew Hall wrote:
> Hello Martin,
> 
> On Mon, Aug 02, 2010 at 10:07:36PM -0500, Martin Holste wrote:
> > Did you try the patternize utility?  It can automate a lot of the
> > pattern creating.
> 
> First of all thank you very much for pointing out patternize; I did see 
> many of the patterndb related blogs but missed this one. I will 
> certainly investigate this in detail and make as much use of it as 
> possible.
> 

Integrating patternize functionality is on my todo list, so hopefully
you won't have to look too far, since it's going to be part of syslog-ng
itself :)

> > Also, are you using the pdbtool to test the messages?  See this
> > blog post for more info:
> 
> I thought about pdbtool but the problem there was that I needed to know 
> exactly which string the daemon would receive, how it would look when 
> the daemon stripped the headers, and what it would send into the 
> patterndb for matching.
> 
> This is because the messages on the socket have different headers from 
> the headers which are used in the disk files of messages I am using as 
> the source of raw material for creating the patterns. Thus I end up with 
> the same problem I started with, unless I'm missing something here.


Well, if you want to look at the result of the message parsing exactly
as done by syslog-ng, you could use a noop rewrite rule and enable
debugging (though it is not recommended to be done in a production
server):

rewrite r_noop { set("$MESSAGE"); };

This would set $MESSAGE to $MESSAGE, but at the end of the rewrite rule,
syslog-ng would emit a debug message about the contents of the MESSAGE
name-value pair.

Alternatively, you may still be able to use "pdbtool match" which can
read a log file, parse it with syslog-ng's message parser and report the
results per name-value pair.

$ pdbtool match -f /var/log/auth.log -p access/sshd.pdb  | head -10
HOST=bzorp
MESSAGE=pam_unix(cron:session): session opened for user root by (uid=0)
PROGRAM=CRON
PID=7362
LEGACY_MSGHDR=CRON[7362]: 
.classifier.class=unknown

...

This uses the normal BSD syslog parser to read the file (thus if you are
using no-parse flag, or RFC5424 format log files, that may differ)


-- 
Bazsi