On Mon, 2010-08-02 at 22:29 -0700, Matthew Hall wrote:
Hello Martin,
On Mon, Aug 02, 2010 at 10:07:36PM -0500, Martin Holste wrote:
Did you try the patternize utility? It can automate a lot of the pattern creating.
First of all thank you very much for pointing out patternize; I did see many of the patterndb related blogs but missed this one. I will certainly investigate this in detail and make as much use of it as possible.
Integrating patternize functionality is on my todo list, so hopefully you won't have to look too far, since it's going to be part of syslog-ng itself :)
Also, are you using the pdbtool to test the messages? See this blog post for more info:
I thought about pdbtool but the problem there was that I needed to know exactly which string the daemon would receive, how it would look when the daemon stripped the headers, and what it would send into the patterndb for matching.
This is because the messages on the socket have different headers from the headers which are used in the disk files of messages I am using as the source of raw material for creating the patterns. Thus I end up with the same problem I started with, unless I'm missing something here.
Well, if you want to look at the result of the message parsing exactly as done by syslog-ng, you could use a noop rewrite rule and enable debugging (though it is not recommended to be done in a production server): rewrite r_noop { set("$MESSAGE"); }; This would set $MESSAGE to $MESSAGE, but at the end of the rewrite rule, syslog-ng would emit a debug message about the contents of the MESSAGE name-value pair. Alternatively, you may still be able to use "pdbtool match" which can read a log file, parse it with syslog-ng's message parser and report the results per name-value pair. $ pdbtool match -f /var/log/auth.log -p access/sshd.pdb | head -10 HOST=bzorp MESSAGE=pam_unix(cron:session): session opened for user root by (uid=0) PROGRAM=CRON PID=7362 LEGACY_MSGHDR=CRON[7362]: .classifier.class=unknown ... This uses the normal BSD syslog parser to read the file (thus if you are using no-parse flag, or RFC5424 format log files, that may differ) -- Bazsi