[syslog-ng] destination based on custom field question

[Matthew Hall]

On Thu, Jan 06, 2011 at 06:01:13PM -0500, Christopher Barry wrote:
> Greetings,
> 
> I'm new to syslog-ng, and I'm trying to key off of a custom field in a
> log entry, and put the message in a particular directory named for the
> field.
> 
> a sample log entry would look like this:
> Dec 16 14:08:51 u910-05 testapp: \
> 00000000000000a7:00007f62d170a910:DEBUG :part.cpp     :  183: \ |
> PartitionInfo [0x275f720]
> 
> '\' denotes line continuation.
> 
> The field I want to key off in this line is:
> 00000000000000a7
> 
> I created a filter to only get stuff from 'testapp', but now I want to
> make the destination be tied to the field. The field is positional, but
> can have any value, and cannot be known a priori.
> 
> Not sure how to go about it. Any links to examples or where to start
> would be very much appreciated.
> 
> --
> Thanks,
> -Christopher

This is certainly doable.

What you need is the patterndb feature described here:

http://www.balabit.com/sites/default/files/documents/syslog-ng-ose-v3.2-guide-admin-en.html/index.html-single.html#chapter-patterndb

You will create a series of rules which match your messages to the level 
of granularity desired, then you can capture variables from the message 
with names you select. Then you can use the variables anywhere 
'downstream' from where you applied the patterndb matching, such as the 
output file template or rewrite rules, etc.

You could also do this with PCRE capturing, but it would be less 
efficient. How many messages/sec. are you seeing and planning to 
support?

Matthew.