[syslog-ng] Requesting help with Grouping-by function

Gabor Nagy (gnagy) Gabor.Nagy at oneidentity.com
Tue Nov 3 16:34:16 UTC 2020

Hi Maciek!

I've checked the documentation and I've found  documentation bug about an example for value() option:

Thanks for your notice, I've checked the whole chapter of grouping-by!
About improving the documentation:
I admit that the chapter where grouping-by() options listed is a bit dense in case of the value() in the aggregate() option.
We will discuss this with the doc writer team, when they process the grouping-by() parser example config bug I've reported.

Otherwise, I think Fabien helped you find out where is the problem: in your destination side template, you only include the .auditd. macros, which have been parsed by linux-auditd-parser().
$MESSAGE macro was missing from the template which is set by the grouping-by parser.

I don't know your use case, but I think your current solution lacks any usage of correlation: even though you set a new name-value pair in the aggregated message (.auditd.test), it's basically the same message as the last message that arrived into the same context.
As Fabien said, you will see that same message twice (the last message before the timeout expired).
The above link shows a good example (I'm copying a fixed version of it) what you can do with message contexts:

  value('MESSAGE' 'An SSH session for ${SSH_USERNAME}@1 from ${SSH_CLIENT_ADDRESS}@2 closed. Session lasted from ${DATE}@2 to ${DATE}')


From: syslog-ng <syslog-ng-bounces at lists.balabit.hu> on behalf of Fabien Wernli <wernli at in2p3.fr>
Sent: Tuesday, November 3, 2020 15:32
To: Maciek Solnicki <msolnicki at gmail.com>
Cc: Syslog-ng users' and developers' mailing list <syslog-ng at lists.balabit.hu>
Subject: Re: [syslog-ng] Requesting help with Grouping-by function

CAUTION: This email originated from outside of the organization. Do not follow guidance, click links, or open attachments unless you recognize the sender and know the content is safe.


If you want to see more Macros in json you can use scopes, for instance:

   format-json -s nv-pairs     # all generic non-dot macros
   format-json -s all-nv-pairs # all generic macros
   format-json -s everything   # as advertised


Member info: https://nam05.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.balabit.hu%2Fmailman%2Flistinfo%2Fsyslog-ng&data=04%7C01%7Cgabor.nagy%40oneidentity.com%7C5fb2a20040a54610eba508d8800559da%7C91c369b51c9e439c989c1867ec606603%7C0%7C1%7C637400107786812381%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=qHmBE%2BaE5kDGmXbGg2E1KdJCKmHp1%2Bw62Uy7BupqlLM%3D&reserved=0
Documentation: https://nam05.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fsupport%2Fdocumentation%2F%3Fproduct%3Dsyslog-ng&data=04%7C01%7Cgabor.nagy%40oneidentity.com%7C5fb2a20040a54610eba508d8800559da%7C91c369b51c9e439c989c1867ec606603%7C0%7C1%7C637400107786812381%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=l0Y1NnxYxEyQstr7N%2Bp%2BKkLTIMvUt6ATDec8B18ufZo%3D&reserved=0
FAQ: https://nam05.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fwiki%2Fsyslog-ng-faq&data=04%7C01%7Cgabor.nagy%40oneidentity.com%7C5fb2a20040a54610eba508d8800559da%7C91c369b51c9e439c989c1867ec606603%7C0%7C1%7C637400107786812381%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=dp%2FGkudcz87cnFnuj12bSKjc4TZP1YsoWZnHi1uHmXE%3D&reserved=0

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20201103/72751d9f/attachment-0001.html>

More information about the syslog-ng mailing list