Hi, although the entire sample is not visible in your email, we have a checkpoint parser that fixes rfc5424 logs from checkpoint as it is using an incorrect format. More specifically the SDATA field is using colons instead of equal signs as in this example: <134>1 2018-03-21 17:25:25 MDS-72 CheckPoint 13752 - [action:"Update"; flags:"150784"; ifdir:"inbound"; logid:"160571424"; loguid:"{0x5ab27965,0x0,0x5b20a8c0,0x7d5707b6}"; origin:"192.168.32.91"; originsicname:"CN=GW91,O=Domain2_Server..cuggd3"; sequencenum:"1"; time:"1521645925"; version:"5"; auth_method:"Machine Authentication (Active Directory)"; auth_status:"Successful Login"; authentication_trial:"this is a reauthentication for session 9a026bba"; client_name:"Active Directory Query"; client_version:"R80.10"; domain_name:"spec.mgmt"; endpoint_ip:"192.168.32.69"; identity_src:"AD Query"; identity_type:"machine"; product:"Identity Awareness"; snid:"9a026bba"; src:"192.168.32.69"; src_machine_group:"All Machines"; src_machine_name:"yonatanad";] we have a checkpoint-parser() component that parses and fixes this into name-value pairs since 3.21, but with fixes in 3.26: https://github.com/syslog-ng/syslog-ng/blob/master/scl/checkpoint/plugin.con... To use that parser you will need to receive logs with flags(no-parse) and then apply the checkpoint-parser() as defined above. Also, there's a component in syslog-ng that receives messages on all sane channels and parses/fixes them automatically, this is the "default-network-drivers()" source, which apart from receiving on tcp/udp/ssl + rfc5424 will properly parse cisco/checkpoint/etc logs by applying application specific filtering/parsing rules, called application adapters. For instance the adapter for checkpoint looks like this: application checkpoint[syslog-raw] { filter { # "syslog" format message("^(<[0-9]{1,3}>)1 .* CheckPoint ") or # "splunk" format message('^time=[0-9]+\|hostname=[a-zA-Z0-9-]+\|product=Firewall'); }; parser { checkpoint-parser(); }; }; This tells syslog-ng that it should look for checkpoint messages in the "syslog-raw" topic (e.g. stream of messages), apply checkpoint-parser() to log messages where the filter above matches. As you can see it will identify both "syslog" and "splunk" style messages emitted from checkpoint. On Mon, May 18, 2020 at 4:15 AM Russell Fulton <r.fulton@auckland.ac.nz> wrote:
I am trying to ingest logs from a checkpoint firewall over a tcp connection:
source s_checkpoint { network( transport("tcp") flags(no-multi-line) port(2514) keep-alive(yes) flags(syslog-protocol) ); };
# added flags(syslog-protocol) on suggest of checkpoint docs for syslog-ng
destination d_debug { file( "/data/russell//test.log" ); };
log { source( s_checkpoint ); destination( d_debug ); };
This is what turns up in the output:
May 18 12:10:42 secmgrprd02 syslog-ng[1555]: Error processing log message: <134>1 2020-05-17T23:33:27Z smartlog01 CheckPoint 25651 -…..
Any ideas on what is going on or how to find out?
Russell
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
-- Bazsi