[syslog-ng] Using pattern parsers as macros

[Marco Mignone]

Hi all,
I am getting a bit crazy about how to use the values from a custom parser_db which I wrote myself.
I think I am missing something quite simple and forgive me if this could very stupid... but if any of you could help I would really appreciate and be thankful.

All I am trying to do is to convert a firewall message into value-pairs in JSON format extracting interesting information to pass to ElasticSearch.

-The original message (received as default syslog)-

Jul 25 12:25:44 172.17.0.1 id=ROHFirewall sn= XXXXXXXX time="2017-07-25 13:25:39" fw=5.148.xxx.xxx pri=4 c=16 m=404 msg="Failed payload verification after decryption; possible preshared key mismatch" n=58631 src=13.81.xx.xx:500 dst=5.148.xxx.xxx:500 proto=udp/500 note="VPN Policy: WAN GroupVPN" fw_action="NA"

-My simple configuration-

source s_file {
  file("/var/log/patterntest3");
};

parser sonicwall {
  db-parser(
    file("/etc/syslog-ng/patterndn.db/sonicwall-pattern.xml")
  );
};

destination d_json {
  file("/var/log/json-test.json" template("$(format-json --scope nv_pairs --key protocol)"));
};

log {
  source(s_file);
  parser(sonicwall);
  destination(d_json);
};


-PDBTool Match Test-
The pattern seem to work fine as the pdbtool gives positive results:

pdbtool match -p /etc/syslog-ng/patterndb.d/sonicwall-pattern.xml -f /var/log/patterntest3

HOST=172.17.0.1
MESSAGE=sn=XXXXXXXX time="2017-07-25 13:25:39" fw=5.148.xxx.xxx pri=4 c=16 m=404 msg="Failed payload verification after decryption; possible preshared key mismatch" n=58631 src=13.81.xx.xx:500 dst=5.148.xxx.xxx:500 proto=udp/500 note="VPN Policy: WAN GroupVPN" fw_action="NA"
PROGRAM=id=ROHFirewall
LEGACY_MSGHDR=id=ROHFirewall
.classifier.class=vpn
.classifier.rule_id=182437592347598
sn= XXXXXXXX
timestamp=2017-07-25 13:25:39
fw.ip=5.148.xxx.xxx
priority=4
cfield=16
mfield=404
msg=Failed payload verification after decryption; possible preshared key mismatch
nfield=58631
src.ip=13.81.xx.xx
src.port=500
dst.ip=5.148.xxx.xxx
dst.port=500
protocol=udp/500
note=VPN Policy: WAN GroupVPN
fw.action=NA
TAGS=.classifier.vpn


-The Results-
When I just use the scope option —nv_pairs I get the following:

{"SOURCE":"s_file","PROGRAM":"id=ROHFirewall","MESSAGE":"sn= XXXXXXXX time=\"2017-07-25 13:25:39\" fw=5.148.xxx.xxx pri=4 c=16 m=404 msg=\"Failed payload verification after decryption; possible preshared key mismatch\" n=58631 src=13.81.xx.xx:500 dst=5.148.xxx.xxx:500 proto=udp/500 note=\"VPN Policy: WAN GroupVPN\" fw_action=\"NA\"","LEGACY_MSGHDR":"id=ROHFirewall ","HOST_FROM":"cf1b071a9e7e","HOST":"cf1b071a9e7e","FILE_NAME":"/var/log/patterntest2"}

What is the template syntax I should use to get any of these value-pairs keys?

Thanks for anyone who will answer this.

Regards,
Marco

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20170727/915818b4/attachment.html>