[syslog-ng] how to use macros from patterndb in syslog-ng.conf

10 Jan 2014

      this seems like a simple thing to do, but a few days of searching and some hours of testing has not shown me the answer.

Essentially I want to parse log events and output select fields, so I have created a basic patterndb xml file.

<patterndb version='3' pub_date='2014-01-09'>
  <ruleset name='patternize' id='6cb77f11-6c9b-ee4e-9f62-b97224d4384c'>
    <rules>
      <rule id='2a270520-2ff7-7048-a088-b03d5b3b5f7d' class='system' provider='patternize'>
        <!-- support: 1 -->
        <patterns>
          <pattern>RT_FLOW_SESSION_CLOSE: session closed @ESTRING:J.MSG.CLOSE.REASON::@ @IPv4:J.MSG.SRC@/@NUMBER:J.MSG.SPORT@->@IPv4:J.MSG.DST@/@NUMBER:J.MSG.DPORT@ </pattern>
          <pattern>RT_FLOW_SESSION_CREATE: session created @IPv4:J.MSG.SRC@/@NUMBER:J.MSG.SPORT@->@IPv4:J.MSG.DST@/@NUMBER:J.MSG.DPORT@ </pattern>
        </patterns>
      </rule>
    </rules>
  </ruleset>
</patterndb>

This tests fine using pdbtool on sample data in a file. (cut from a real syslog file of logs)
Inside syslog-ng.conf I want to use the parsed values as a template:
source s_testfile {
    file("/home/n0142566/testfile"
     flags(no-parse) );
};

filter f_juniper_session_create {
  match("RT_FLOW_SESSION_CREATE" value( "MESSAGE" ) );
};
filter f_juniper_session_close {
  match("RT_FLOW_SESSION_CLOSE" value( "MESSAGE" ) );
};

parser juniper_db {
  db-parser (
    file("/home/n0142566/log/juniper_db.xml")
  );
};

destination d_local_create {
        file("/home/n0142566/log/messages-create-$HOST"
          template("${J.MSG.SRC}, ${J.MSG.SPORT}, ${J.MSG.DST}, ${J.MSG.DPORT}\n") );
};

destination d_local_close {
        file("/home/n0142566/log/messages-close-$HOST"
#!#          template("${J.MSG.SRC}, ${J.MSG.SPORT}, ${J.MSG.DST}, ${J.MSG.DPORT}, ${J.MSG.CLOSE.REASON}\n") );
          template("${J.MSG}\n") );
};

log {
  source(s_testfile);
  filter(f_juniper_session_create);
  parser(juniper_db);
  destination(d_local_create);
};
log {
  source(s_testfile);
  filter(f_juniper_session_close);
  parser(juniper_db);
  destination(d_local_close);
};

But when I run syslog-ng and append events to "testfile" the output is simply comma separated blanks :-(

I am quite sure I am missing something *extremely* basic - but at this point I thought I would ask clearer minds for help!!

Thanks,
Jim

[syslog-ng] how to use macros from patterndb in syslog-ng.conf

jrhendri＠roadrunner.com