On Thu, 2011-01-06 at 16:52 -0800, Matthew Hall wrote:
On Thu, Jan 06, 2011 at 06:01:13PM -0500, Christopher Barry wrote:
Greetings,
I'm new to syslog-ng, and I'm trying to key off of a custom field in a log entry, and put the message in a particular directory named for the field.
a sample log entry would look like this: Dec 16 14:08:51 u910-05 testapp: \ 00000000000000a7:00007f62d170a910:DEBUG :part.cpp : 183: \ | PartitionInfo [0x275f720]
'\' denotes line continuation.
The field I want to key off in this line is: 00000000000000a7
I created a filter to only get stuff from 'testapp', but now I want to make the destination be tied to the field. The field is positional, but can have any value, and cannot be known a priori.
Not sure how to go about it. Any links to examples or where to start would be very much appreciated.
-- Thanks, -Christopher
This is certainly doable.
What you need is the patterndb feature described here:
http://www.balabit.com/sites/default/files/documents/syslog-ng-ose-v3.2-guid...
You will create a series of rules which match your messages to the level of granularity desired, then you can capture variables from the message with names you select. Then you can use the variables anywhere 'downstream' from where you applied the patterndb matching, such as the output file template or rewrite rules, etc.
You could also do this with PCRE capturing, but it would be less efficient. How many messages/sec. are you seeing and planning to support?
If this stream can easily separated from the rest of your logs, then csv-parser with a custom separator might be somewhat easier to use, and still you can apply db-parser() filtering after the csv-parser has exploded the fields into name-value pairs. -- Bazsi