[syslog-ng] destination based on custom field question

[Martin Holste]

Give this a shot for a patterndb.xml:

<patterndb version="3">
<ruleset name="testapp">
  <pattern>testapp</pattern>
  <rules>
    <rule id="x" class="x">
      <patterns>
        <pattern>@ESTRING:field_name::@@ESTRING:field_name_2::@@ESTRING:level:
@:@ESTRING:file_name: @</pattern>
      </patterns>
      <examples>
        <example>
          <test_message
program="testapp">00000000000000a7:00007f62d170a910:DEBUG :part.cpp
 :  183: PartitionInfo [0x275f720]</test_message>
          <test_values>
             <test_value name="field_name">00000000000000a7</test_value>
             <test_value name="field_name_2">00007f62d170a910</test_value>
             <test_value name="level">DEBUG</test_value>
             <test_value name="file_name">part.cpp</test_value>
          </test_values>
        </example>
      </examples>
    </rule>
  </rules>
</ruleset>
</patterndb>

On Thu, Jan 6, 2011 at 6:52 PM, Matthew Hall <mhall at mhcomputing.net> wrote:
> On Thu, Jan 06, 2011 at 06:01:13PM -0500, Christopher Barry wrote:
>> Greetings,
>>
>> I'm new to syslog-ng, and I'm trying to key off of a custom field in a
>> log entry, and put the message in a particular directory named for the
>> field.
>>
>> a sample log entry would look like this:
>> Dec 16 14:08:51 u910-05 testapp: \
>> 00000000000000a7:00007f62d170a910:DEBUG :part.cpp     :  183: \ |
>> PartitionInfo [0x275f720]
>>
>> '\' denotes line continuation.
>>
>> The field I want to key off in this line is:
>> 00000000000000a7
>>
>> I created a filter to only get stuff from 'testapp', but now I want to
>> make the destination be tied to the field. The field is positional, but
>> can have any value, and cannot be known a priori.
>>
>> Not sure how to go about it. Any links to examples or where to start
>> would be very much appreciated.
>>
>> --
>> Thanks,
>> -Christopher
>
> This is certainly doable.
>
> What you need is the patterndb feature described here:
>
> http://www.balabit.com/sites/default/files/documents/syslog-ng-ose-v3.2-guide-admin-en.html/index.html-single.html#chapter-patterndb
>
> You will create a series of rules which match your messages to the level
> of granularity desired, then you can capture variables from the message
> with names you select. Then you can use the variables anywhere
> 'downstream' from where you applied the patterndb matching, such as the
> output file template or rewrite rules, etc.
>
> You could also do this with PCRE capturing, but it would be less
> efficient. How many messages/sec. are you seeing and planning to
> support?
>
> Matthew.
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.campin.net/syslog-ng/faq.html
>
>