[syslog-ng] Using pattern parsers as macros

[Marco Mignone]

Hi all,
I found the error, it was a typo in referencing the pattern db file.
Sorry for bothering you, I think I will now be able to make it work the way I want.

Thanks,
Marco

> On 27 Jul 2017, at 15:21, Marco Mignone <info at marcomignone.com> wrote:
> 
> Hi,
> I forgot to give some info on the system.
> 
> I am running syslog-ng v3.10.1 through docker on a MacBook.
> 
> Thanks,
> Marco
> 
>> On 27 Jul 2017, at 15:19, Marco Mignone <info at marcomignone.com <mailto:info at marcomignone.com>> wrote:
>> 
>> Hi all,
>> I am getting a bit crazy about how to use the values from a custom parser_db which I wrote myself.
>> I think I am missing something quite simple and forgive me if this could very stupid... but if any of you could help I would really appreciate and be thankful.
>> 
>> All I am trying to do is to convert a firewall message into value-pairs in JSON format extracting interesting information to pass to ElasticSearch.
>> 
>> -The original message (received as default syslog)-
>> 
>> Jul 25 12:25:44 172.17.0.1 id=ROHFirewall sn= XXXXXXXX time="2017-07-25 13:25:39" fw=5.148.xxx.xxx pri=4 c=16 m=404 msg="Failed payload verification after decryption; possible preshared key mismatch" n=58631 src=13.81.xx.xx:500 dst=5.148.xxx.xxx:500 proto=udp/500 note="VPN Policy: WAN GroupVPN" fw_action="NA"
>> 
>> -My simple configuration-
>> 
>> source s_file {
>>   file("/var/log/patterntest3");
>> };
>> 
>> parser sonicwall {
>>   db-parser(
>>     file("/etc/syslog-ng/patterndn.db/sonicwall-pattern.xml")
>>   );
>> };
>> 
>> destination d_json {
>>   file("/var/log/json-test.json" template("$(format-json --scope nv_pairs --key protocol)"));
>> };
>> 
>> log {
>>   source(s_file);
>>   parser(sonicwall);
>>   destination(d_json);
>> };
>> 
>> 
>> -PDBTool Match Test-
>> The pattern seem to work fine as the pdbtool gives positive results:
>> 
>> pdbtool match -p /etc/syslog-ng/patterndb.d/sonicwall-pattern.xml -f /var/log/patterntest3
>> 
>> HOST=172.17.0.1
>> MESSAGE=sn=XXXXXXXX time="2017-07-25 13:25:39" fw=5.148.xxx.xxx pri=4 c=16 m=404 msg="Failed payload verification after decryption; possible preshared key mismatch" n=58631 src=13.81.xx.xx:500 dst=5.148.xxx.xxx:500 proto=udp/500 note="VPN Policy: WAN GroupVPN" fw_action="NA"
>> PROGRAM=id=ROHFirewall
>> LEGACY_MSGHDR=id=ROHFirewall
>> .classifier.class=vpn
>> .classifier.rule_id=182437592347598
>> sn= XXXXXXXX
>> timestamp=2017-07-25 13:25:39
>> fw.ip=5.148.xxx.xxx
>> priority=4
>> cfield=16
>> mfield=404
>> msg=Failed payload verification after decryption; possible preshared key mismatch
>> nfield=58631
>> src.ip=13.81.xx.xx
>> src.port=500
>> dst.ip=5.148.xxx.xxx
>> dst.port=500
>> protocol=udp/500
>> note=VPN Policy: WAN GroupVPN
>> fw.action=NA
>> TAGS=.classifier.vpn
>> 
>> 
>> -The Results-
>> When I just use the scope option —nv_pairs I get the following:
>> 
>> {"SOURCE":"s_file","PROGRAM":"id=ROHFirewall","MESSAGE":"sn= XXXXXXXX time=\"2017-07-25 13:25:39\" fw=5.148.xxx.xxx pri=4 c=16 m=404 msg=\"Failed payload verification after decryption; possible preshared key mismatch\" n=58631 src=13.81.xx.xx:500 dst=5.148.xxx.xxx:500 proto=udp/500 note=\"VPN Policy: WAN GroupVPN\" fw_action=\"NA\"","LEGACY_MSGHDR":"id=ROHFirewall ","HOST_FROM":"cf1b071a9e7e","HOST":"cf1b071a9e7e","FILE_NAME":"/var/log/patterntest2"}
>> 
>> What is the template syntax I should use to get any of these value-pairs keys?
>> 
>> Thanks for anyone who will answer this.
>> 
>> Regards,
>> Marco
>> 
>> ______________________________________________________________________________
>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng <https://lists.balabit.hu/mailman/listinfo/syslog-ng>
>> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng <http://www.balabit.com/support/documentation/?product=syslog-ng>
>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq <http://www.balabit.com/wiki/syslog-ng-faq>
>> 
> 
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20170727/6707437f/attachment-0001.html>