<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">Hi all,<div class="">I found the error, it was a typo in referencing the pattern db file.</div><div class="">Sorry for bothering you, I think I will now be able to make it work the way I want.</div><div class=""><br class=""></div><div class="">Thanks,</div><div class="">Marco<br class=""><div class=""><br class=""><div><blockquote type="cite" class=""><div class="">On 27 Jul 2017, at 15:21, Marco Mignone <<a href="mailto:info@marcomignone.com" class="">info@marcomignone.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><meta http-equiv="Content-Type" content="text/html charset=utf-8" class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">Hi,<div class="">I forgot to give some info on the system.</div><div class=""><br class=""></div><div class="">I am running syslog-ng v3.10.1 through docker on a MacBook.</div><div class=""><br class=""></div><div class="">Thanks,</div><div class="">Marco</div><div class=""><br class=""></div><div class=""><div class=""><blockquote type="cite" class=""><div class="">On 27 Jul 2017, at 15:19, Marco Mignone <<a href="mailto:info@marcomignone.com" class="">info@marcomignone.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">Hi all,<div class="">I am getting a bit crazy about how to use the values from a custom parser_db which I wrote myself.</div><div class="">I think I am missing something quite simple and forgive me if this could very stupid... but if any of you could help I would really appreciate and be thankful.</div><div class=""><br class=""></div><div class="">All I am trying to do is to convert a firewall message into value-pairs in JSON format extracting interesting information to pass to ElasticSearch.</div><div class=""><br class=""></div><div class=""><b class="">-The original message (received as default syslog)-</b></div><div class=""><b class=""><br class=""></b></div><div class="">Jul 25 12:25:44 172.17.0.1 id=ROHFirewall sn= XXXXXXXX time="2017-07-25 13:25:39" fw=5.148.xxx.xxx pri=4 c=16 m=404 msg="Failed payload verification after decryption; possible preshared key mismatch" n=58631 src=13.81.xx.xx:500 dst=5.148.xxx.xxx:500 proto=udp/500 note="VPN Policy: WAN GroupVPN" fw_action="NA"</div><div class=""><br class=""></div><div class=""><b class="">-My simple configuration-</b></div><div class=""><br class=""></div><div class=""><div class="">source s_file {</div><div class=""> file("/var/log/patterntest3");</div><div class="">};</div></div><div class=""><br class=""></div><div class=""><div class="">parser sonicwall {</div><div class=""> db-parser(</div><div class=""> file("/etc/syslog-ng/patterndn.db/sonicwall-pattern.xml")</div><div class=""> );</div><div class="">};</div></div><div class=""><br class=""></div><div class=""><div class="">destination d_json {</div><div class=""> file("/var/log/json-test.json" template("$(format-json --scope nv_pairs --key protocol)"));</div><div class="">};</div></div><div class=""><br class=""></div><div class=""><div class="">log {</div><div class=""> source(s_file);</div><div class=""> parser(sonicwall);</div><div class=""> destination(d_json);</div><div class="">};</div></div><div class=""><br class=""></div><div class=""><br class=""></div><div class=""><b class="">-PDBTool Match Test-</b></div><div class="">The pattern seem to work fine as the <b class="">pdbtool</b> gives positive results:</div><div class=""><br class=""></div><div class=""><div class=""><b class="">pdbtool match -p /etc/syslog-ng/patterndb.d/sonicwall-pattern.xml -f /var/log/patterntest3</b></div><div class=""><br class=""></div><div class="">HOST=172.17.0.1</div><div class="">MESSAGE=sn=XXXXXXXX time="2017-07-25 13:25:39" fw=5.148.xxx.xxx pri=4 c=16 m=404 msg="Failed payload verification after decryption; possible preshared key mismatch" n=58631 src=13.81.xx.xx:500 dst=5.148.xxx.xxx:500 proto=udp/500 note="VPN Policy: WAN GroupVPN" fw_action="NA"</div><div class="">PROGRAM=id=ROHFirewall</div><div class="">LEGACY_MSGHDR=id=ROHFirewall</div><div class="">.classifier.class=vpn</div><div class="">.classifier.rule_id=182437592347598</div><div class="">sn= XXXXXXXX</div><div class="">timestamp=2017-07-25 13:25:39</div><div class="">fw.ip=5.148.xxx.xxx</div><div class="">priority=4</div><div class="">cfield=16</div><div class="">mfield=404</div><div class="">msg=Failed payload verification after decryption; possible preshared key mismatch</div><div class="">nfield=58631</div><div class="">src.ip=13.81.xx.xx</div><div class="">src.port=500</div><div class="">dst.ip=5.148.xxx.xxx</div><div class="">dst.port=500</div><div class="">protocol=udp/500</div><div class="">note=VPN Policy: WAN GroupVPN</div><div class="">fw.action=NA</div><div class="">TAGS=.classifier.vpn</div></div><div class=""><br class=""></div><div class=""><br class=""></div><div class=""><b class="">-The Results-</b></div><div class="">When I just use the scope option —nv_pairs I get the following:</div><div class=""><br class=""></div><div class="">{"SOURCE":"s_file","PROGRAM":"id=ROHFirewall","MESSAGE":"sn= XXXXXXXX time=\"2017-07-25 13:25:39\" fw=5.148.xxx.xxx pri=4 c=16 m=404 msg=\"Failed payload verification after decryption; possible preshared key mismatch\" n=58631 src=13.81.xx.xx:500 dst=5.148.xxx.xxx:500 proto=udp/500 note=\"VPN Policy: WAN GroupVPN\" fw_action=\"NA\"","LEGACY_MSGHDR":"id=ROHFirewall ","HOST_FROM":"cf1b071a9e7e","HOST":"cf1b071a9e7e","FILE_NAME":"/var/log/patterntest2"}</div><div class=""><br class=""></div><div class="">What is the template syntax I should use to get any of these value-pairs keys?</div><div class=""><br class=""></div><div class="">Thanks for anyone who will answer this.</div><div class=""><br class=""></div><div class="">Regards,</div><div class="">Marco</div><div class=""><br class=""></div></div>______________________________________________________________________________<br class="">Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" class="">https://lists.balabit.hu/mailman/listinfo/syslog-ng</a><br class="">Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" class="">http://www.balabit.com/support/documentation/?product=syslog-ng</a><br class="">FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" class="">http://www.balabit.com/wiki/syslog-ng-faq</a><br class=""><br class=""></div></blockquote></div><br class=""></div></div>______________________________________________________________________________<br class="">Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" class="">https://lists.balabit.hu/mailman/listinfo/syslog-ng</a><br class="">Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" class="">http://www.balabit.com/support/documentation/?product=syslog-ng</a><br class="">FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" class="">http://www.balabit.com/wiki/syslog-ng-faq</a><br class=""><br class=""></div></blockquote></div><br class=""></div></div></body></html>