[syslog-ng] Using custom parser with only a subset of udp traffic

[Faine, Mark R. (MSFC-IS40)[NICS]]

I think I have figured out that I can use one source and always specify the parser manually, that has gotten me most of the way there.  Thank you. Still any suggestions for how to proceed are of course welcome.

Thanks,
-Mark

-----Original Message-----
From: syslog-ng <syslog-ng-bounces at lists.balabit.hu> On Behalf Of Faine, Mark R. (MSFC-IS40)[NICS]
Sent: Thursday, May 20, 2021 09:55
To: Syslog-ng users' and developers' mailing list <syslog-ng at lists.balabit.hu>
Subject: [syslog-ng] Using custom parser with only a subset of udp traffic

Hopefully one of you can help me figure this out.   All of my udp traffic is coming in through one source (network driver), some of it is firewall data but I have to filter it out based on where it is coming from and other fields in the log data.  I want to use the csv_parser here since it would make it easier for me to get the data I need from the firewall data.  Does this require splitting my one source into two sources, since I would need to use flags('no-parse') on a firewall data source?   I also think this means that I can't use a single log path to process both sources?  Also, how to prevent potential duplication since the non-firewall source will still receive the firewall traffic and try to process it with syslog-parser.

I think the flow is something like this:

                        firewall source -> custom parser -> filters ->rewrites 
                      /                                                                                          \
all_data ->                                                                                                 \  -> destination
                    \                                                                                            /
                     all udp source -> filters -> rewrites                             /

I understand this is a bit complex to answer simply, I'm mostly looking for answers to the above questions as well as high level guidance for how to proceed.

Thanks,
-Mark
______________________________________________________________________________
Member info: https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.balabit.hu%2Fmailman%2Flistinfo%2Fsyslog-ng&data=04%7C01%7Cmark.faine%40nasa.gov%7C628f7af2a5e84a4c777408d91b9f7783%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637571193988328633%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=uxsK4JRgx9eWtnferY2JVssWQc81%2BMhuRigh6jyZfmg%3D&reserved=0
Documentation: https://gcc02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fsupport%2Fdocumentation%2F%3Fproduct%3Dsyslog-ng&data=04%7C01%7Cmark.faine%40nasa.gov%7C628f7af2a5e84a4c777408d91b9f7783%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637571193988333617%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=ZnwJ5ArWV1mb6HgLsnKiYxw7ooiryF9TrUKHL6%2BZouE%3D&reserved=0
FAQ: https://gcc02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fwiki%2Fsyslog-ng-faq&data=04%7C01%7Cmark.faine%40nasa.gov%7C628f7af2a5e84a4c777408d91b9f7783%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637571193988333617%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=UNNOY1JjuOewJFAWNW%2Fz6CGCs3MO80LPedLV6OtjIqc%3D&reserved=0