Hi Syslog-ng, My blog about analyzing Suricata log messages has many examples about processing logs differently based on message content: https://www.syslog-ng.com/community/b/blog/posts/analyze-your-suricata-logs-... Have a nice weekend! Peter Peter Czanik (CzP) <peter.czanik@oneidentity.com> Balabit (a OneIdentity company) / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik ________________________________ From: syslog-ng <syslog-ng-bounces@lists.balabit.hu> on behalf of Faine, Mark R. (MSFC-IS40)[NICS] <mark.faine@nasa.gov> Sent: Friday, May 21, 2021 16:05 To: Syslog-ng users' and developers' mailing list <syslog-ng@lists.balabit.hu> Subject: Re: [syslog-ng] Using custom parser with only a subset of udp traffic CAUTION: This email originated from outside of the organization. Do not follow guidance, click links, or open attachments unless you recognize the sender and know the content is safe. I think I have figured out that I can use one source and always specify the parser manually, that has gotten me most of the way there. Thank you. Still any suggestions for how to proceed are of course welcome. Thanks, -Mark -----Original Message----- From: syslog-ng <syslog-ng-bounces@lists.balabit.hu> On Behalf Of Faine, Mark R. (MSFC-IS40)[NICS] Sent: Thursday, May 20, 2021 09:55 To: Syslog-ng users' and developers' mailing list <syslog-ng@lists.balabit.hu> Subject: [syslog-ng] Using custom parser with only a subset of udp traffic Hopefully one of you can help me figure this out. All of my udp traffic is coming in through one source (network driver), some of it is firewall data but I have to filter it out based on where it is coming from and other fields in the log data. I want to use the csv_parser here since it would make it easier for me to get the data I need from the firewall data. Does this require splitting my one source into two sources, since I would need to use flags('no-parse') on a firewall data source? I also think this means that I can't use a single log path to process both sources? Also, how to prevent potential duplication since the non-firewall source will still receive the firewall traffic and try to process it with syslog-parser. I think the flow is something like this: firewall source -> custom parser -> filters ->rewrites / \ all_data -> \ -> destination \ / all udp source -> filters -> rewrites / I understand this is a bit complex to answer simply, I'm mostly looking for answers to the above questions as well as high level guidance for how to proceed. Thanks, -Mark ______________________________________________________________________________ Member info: https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.balabit.hu%2Fmailman%2Flistinfo%2Fsyslog-ng&data=04%7C01%7CPeter.Czanik%40oneidentity.com%7Cecfcc63d70fe4cfe5d7e08d91c6186cb%7C91c369b51c9e439c989c1867ec606603%7C0%7C1%7C637572027476926994%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=pfdd17XkUlj53ZWtwjloGxs%2BFCVyoSyCWg%2FsMzni%2FJE%3D&reserved=0 Documentation: https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fsupport%2Fdocumentation%2F%3Fproduct%3Dsyslog-ng&data=04%7C01%7CPeter.Czanik%40oneidentity.com%7Cecfcc63d70fe4cfe5d7e08d91c6186cb%7C91c369b51c9e439c989c1867ec606603%7C0%7C1%7C637572027476926994%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=ZvjFq4y29jjLnicSXcVFbDkstsh7fp2k46juF2fSXNo%3D&reserved=0 FAQ: https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fwiki%2Fsyslog-ng-faq&data=04%7C01%7CPeter.Czanik%40oneidentity.com%7Cecfcc63d70fe4cfe5d7e08d91c6186cb%7C91c369b51c9e439c989c1867ec606603%7C0%7C1%7C637572027476936958%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=msZEifJqOREtlckmKcBMN4BAlqBeNq3VzRw%2FdsUH%2BM8%3D&reserved=0 ______________________________________________________________________________ Member info: https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.balabit.hu%2Fmailman%2Flistinfo%2Fsyslog-ng&data=04%7C01%7CPeter.Czanik%40oneidentity.com%7Cecfcc63d70fe4cfe5d7e08d91c6186cb%7C91c369b51c9e439c989c1867ec606603%7C0%7C1%7C637572027476936958%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=qavJ9f%2B988KYI3EMI3oPi7z%2BWJ6e01Kiphqo8aiLTzg%3D&reserved=0 Documentation: https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fsupport%2Fdocumentation%2F%3Fproduct%3Dsyslog-ng&data=04%7C01%7CPeter.Czanik%40oneidentity.com%7Cecfcc63d70fe4cfe5d7e08d91c6186cb%7C91c369b51c9e439c989c1867ec606603%7C0%7C1%7C637572027476936958%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=vNxB%2B2ayi4V8aOF0SG6VCv4QK2TPrZIiwznujhvhOJw%3D&reserved=0 FAQ: https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fwiki%2Fsyslog-ng-faq&data=04%7C01%7CPeter.Czanik%40oneidentity.com%7Cecfcc63d70fe4cfe5d7e08d91c6186cb%7C91c369b51c9e439c989c1867ec606603%7C0%7C1%7C637572027476936958%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=msZEifJqOREtlckmKcBMN4BAlqBeNq3VzRw%2FdsUH%2BM8%3D&reserved=0