[syslog-ng] Help parsing logs

[Balazs Scheidler]

Hi,

There's a Palo alto parser in the SCL that I wrote a few years back:

https://github.com/axoflow/axosyslog/blob/main/scl/paloalto/panos.conf

That parser can be used to extract the Palo fields as name value pairs.

That parser can be used to extract the .panos.device_name field, which you
can then embed in your filename template as ${.panos.device_name}

To use the panos-parser() you need to include scl.conf in your main config
but you probably have that already.

Let me know if it doesn't work out.

-- 
Bazsi
https://discord.gg/NNUJUmC36j

On Fri, Jun 28, 2024, 19:08 John Norton <nortonjco at gmail.com> wrote:

> Hi All,
>
>
>
> I have a panorama system that sends logs from several devices (dvc_name).
> What I want to do is have it write to files based on the dvc_name. The
> dvc_name is in the following message as "ip-10-37-12-142".  These are not
> static as they are spun up as needed and the dvc_name may change.
>
>
>
> Can you help me figure out how to create a parser or filter to take the
> following message, where it will read in the dvc_name "ip-10-37-12-142" and
> write logs that include that dvc_name to a separate file (and other
> dvc_names it sees in messages to their own file?:
>
>
>
> *Jun 27 19:08:15 TERL-PANORAMA-01.times.com
> <http://terl-panorama-01.times.com/> 1,2024/06/27
> 15:08:14,005655000463255,SYSTEM,dhcp,2817,2024/06/27
> 15:08:09,,if-renew-trigger,,0,0,general,informational,"DHCP RENEW:
> interface eth0, ip 10.37.12.142 netmask 255.255.255.224 dhcp server:
> 10.37.12.129",7383625404611793457,0x8000000000000000,0,0,0,0,,ip-10-37-12-142,0,0,2024-06-27T15:08:09.224-04:00*
>
>
>
> I've tried a few different things but can't figure out how to do it
> dynamically. This rewrite worked, but it requires a static entry which
> won't scale and all potential dvc_names cannot be known:
>
>
>
> rewrite r_panorama      { set("ip-10-37-12-142", value("HOST")
> condition(message(".*ip-10-37-12-142"))); };
>
>
>
>
>
> I've also tried creating a filter, but it doesn't actually store what the
> match finds in a macro that I can replace something like $HOST with.
>
>
>
> filter f_panorama                            { match("ip-\d+-\d+-\d+-\d+"
> value("MESSAGE")); };
>
>
>
> I was also thinking that I could do something with the csv parser, but it
> didn't work like I was expecting, but am thinking a parser might work if I
> could get it right (I think partially it doesn't work because it is a
> syslog message and the first part of the message interferes with the csv
> parsing, not sure):
>
>
>
> parser p_panorama {
>
>                 csv-parser(
>
>
> columns("future_use1","receive_time","serial_number","log_type","log_subtype","version","generated_time","vsys","event_id","object","future_use3","future_use4","module","severity","description","sequence_number","action_flags","devicegroup_level1","devicegroup_level2","devicegroup_level3","devicegroup_level4","vsys_name","dvc_name")
>
>                                 delimiters(",")
>
>                                 quote-pairs('""[]')
>
>                 );
>
> };
>
>
>
> destination d_panorama
> {file("`mypath`/panorama/$dvc_name/$dvc_name.log"); };
>
>
>
> log { source(s_panother); parser(p_panorama); destination(d_panorama); };
>
>
>
> Is this something someone could help me with?
>
>
>
> Thank you,
>
> John
>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation:
> http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20240628/46e3296c/attachment.htm>