Re: [syslog-ng] grouping-by question

13 Jun 2019

      Thanks for help! This solution helped me.

I have one more question: I send several messages that must be aggregated into one, but the context-length parameter always turns out to be increased by one. Why it happens?

From: syslog-ng <syslog-ng-bounces@lists.balabit.hu> On Behalf Of Peter Kokai (pkokai)
Sent: Tuesday, June 11, 2019 6:14 PM
To: Syslog-ng users' and developers' mailing list <syslog-ng@lists.balabit.hu>
Subject: Re: [syslog-ng] grouping-by question

Hello,
Maybe there is a better option, but what I ended up doing in similuar case to set a new key-value and filter messages that had that value.

aggregate(
...
   value("should-only-pass" "1")
...
)
and before destination add a filter to only pass messages with should-only-pass
--
kokan
Get Outlook for Android<https://aka.ms/ghei36>

________________________________
From: syslog-ng <syslog-ng-bounces@lists.balabit.hu<mailto:syslog-ng-bounces@lists.balabit.hu>> on behalf of Газин Максим Алексеевич <Maksim.Gazin@rt.ru<mailto:Maksim.Gazin@rt.ru>>
Sent: Tuesday, June 11, 2019 5:03:02 PM
To: syslog-ng@lists.balabit.hu<mailto:syslog-ng@lists.balabit.hu>
Subject: [syslog-ng] grouping-by question

CAUTION: This email originated from outside of the organization. Do not follow guidance, click links, or open attachments unless you recognize the sender and know the content is safe.

Hello,

I'm new to syslog-ng, and I'm having some trouble.

I want to use syslog-ng to aggregate similar events from our firewalls. I use the grouping-by parser to create an aggregated event, but I have a question, how do I get only aggregated events on the output without source ones?

My configuration:

parser p_patterndb {

        db-parser(file("/etc/syslog-ng/conf.d/forti.xml"));

        };

log {

        source {

          file("/tmp/in.log" flags(no-parse));

        };

       parser {

          kv_parser (prefix("forti."));

          grouping-by(

                key("${forti.devname}${forti.srcip}${forti.dstip}${forti.dstport}")

                scope("process")

                timeout(15)

                aggregate(

                  value("MESSAGE" "aggegated devname=${forti.devname} srcip=${forti.srcip} dstip=${forti.dstip} dstport=${forti.dstport} count=$(context-length)")

                  inherit-mode("context")

                )

                 );

                };

        destination {

          file("tmp/out.log");

          };

  };

My current output:

Jun  8 08:42:27 centos7-1 <15>Jun 10 16:10:02 fw01 fw: logver=56 devname="fw01" devid=" " vd="" date=2019-06-04 time=15:53:34 logid="0000000013" type="traffic" subtype="forward" level="notice" eventtime=1559652814 srcip=172.26.104.3 srcport=62444 srcintf="ae1.2051" srcintfrole="lan" dstip=172.16.104.219 dstport=9092 dstintf="ae0.2099" dstintfrole="lan" sessionid=3465151973 proto=6 action="deny" policyid=0 policytype="policy" dstcountry="Reserved" srccountry="Reserved" trandisp="noop" duration=0 sentbyte=0 rcvdbyte=0 sentpkt=0 appcat="unscanned" crscore=30 craction=131072 crlevel="high"
Jun  8 08:42:27 centos7-1 <15>Jun 10 16:10:02 fw01 fw: logver=56 devname="-fw01" devid="" vd=" " date=2019-06-04 time=15:53:34 logid="0000000013" type="traffic" subtype="forward" level="notice" eventtime=1559652814 srcip=172.26.104.3 srcport=62444 srcintf="ae1.2051" srcintfrole="lan" dstip=172.16.104.219 dstport=9092 dstintf="ae0.2099" dstintfrole="lan" sessionid=3465151973 proto=6 action="deny" policyid=0 policytype="policy" dstcountry="Reserved" srccountry="Reserved" trandisp="noop" duration=0 sentbyte=0 rcvdbyte=0 sentpkt=0 appcat="unscanned" crscore=30 craction=131072 crlevel="high"
crscore=30 craction=131072 crlevel="high"
Jun  8 08:42:28 centos7-1 aggegated devname=fw01 srcip=172.26.104.3 dstip=172.16.104.219 dstport=9092 count=3

What I want:
Jun  8 08:42:28 centos7-1 aggegated devname=fw01 srcip=172.26.104.3 dstip=172.16.104.219 dstport=9092 count=3

Any guidance would be very much appreciated.

Maxim Gazin     |              Senior security engineer              |              Rostelecom