[syslog-ng] Using pattern parsers as macros

[Marco Mignone]

Hi,
I forgot to give some info on the system.

I am running syslog-ng v3.10.1 through docker on a MacBook.

Thanks,
Marco

> On 27 Jul 2017, at 15:19, Marco Mignone <info at marcomignone.com> wrote:
> 
> Hi all,
> I am getting a bit crazy about how to use the values from a custom parser_db which I wrote myself.
> I think I am missing something quite simple and forgive me if this could very stupid... but if any of you could help I would really appreciate and be thankful.
> 
> All I am trying to do is to convert a firewall message into value-pairs in JSON format extracting interesting information to pass to ElasticSearch.
> 
> -The original message (received as default syslog)-
> 
> Jul 25 12:25:44 172.17.0.1 id=ROHFirewall sn= XXXXXXXX time="2017-07-25 13:25:39" fw=5.148.xxx.xxx pri=4 c=16 m=404 msg="Failed payload verification after decryption; possible preshared key mismatch" n=58631 src=13.81.xx.xx:500 dst=5.148.xxx.xxx:500 proto=udp/500 note="VPN Policy: WAN GroupVPN" fw_action="NA"
> 
> -My simple configuration-
> 
> source s_file {
>   file("/var/log/patterntest3");
> };
> 
> parser sonicwall {
>   db-parser(
>     file("/etc/syslog-ng/patterndn.db/sonicwall-pattern.xml")
>   );
> };
> 
> destination d_json {
>   file("/var/log/json-test.json" template("$(format-json --scope nv_pairs --key protocol)"));
> };
> 
> log {
>   source(s_file);
>   parser(sonicwall);
>   destination(d_json);
> };
> 
> 
> -PDBTool Match Test-
> The pattern seem to work fine as the pdbtool gives positive results:
> 
> pdbtool match -p /etc/syslog-ng/patterndb.d/sonicwall-pattern.xml -f /var/log/patterntest3
> 
> HOST=172.17.0.1
> MESSAGE=sn=XXXXXXXX time="2017-07-25 13:25:39" fw=5.148.xxx.xxx pri=4 c=16 m=404 msg="Failed payload verification after decryption; possible preshared key mismatch" n=58631 src=13.81.xx.xx:500 dst=5.148.xxx.xxx:500 proto=udp/500 note="VPN Policy: WAN GroupVPN" fw_action="NA"
> PROGRAM=id=ROHFirewall
> LEGACY_MSGHDR=id=ROHFirewall
> .classifier.class=vpn
> .classifier.rule_id=182437592347598
> sn= XXXXXXXX
> timestamp=2017-07-25 13:25:39
> fw.ip=5.148.xxx.xxx
> priority=4
> cfield=16
> mfield=404
> msg=Failed payload verification after decryption; possible preshared key mismatch
> nfield=58631
> src.ip=13.81.xx.xx
> src.port=500
> dst.ip=5.148.xxx.xxx
> dst.port=500
> protocol=udp/500
> note=VPN Policy: WAN GroupVPN
> fw.action=NA
> TAGS=.classifier.vpn
> 
> 
> -The Results-
> When I just use the scope option —nv_pairs I get the following:
> 
> {"SOURCE":"s_file","PROGRAM":"id=ROHFirewall","MESSAGE":"sn= XXXXXXXX time=\"2017-07-25 13:25:39\" fw=5.148.xxx.xxx pri=4 c=16 m=404 msg=\"Failed payload verification after decryption; possible preshared key mismatch\" n=58631 src=13.81.xx.xx:500 dst=5.148.xxx.xxx:500 proto=udp/500 note=\"VPN Policy: WAN GroupVPN\" fw_action=\"NA\"","LEGACY_MSGHDR":"id=ROHFirewall ","HOST_FROM":"cf1b071a9e7e","HOST":"cf1b071a9e7e","FILE_NAME":"/var/log/patterntest2"}
> 
> What is the template syntax I should use to get any of these value-pairs keys?
> 
> Thanks for anyone who will answer this.
> 
> Regards,
> Marco
> 
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20170727/186132cb/attachment.html>