[syslog-ng] Requesting help with Grouping-by function

[Gabor Nagy (gnagy)]

Hi Maciek!

I've checked the documentation and I've found  documentation bug about an example for value() option:
https://www.syslog-ng.com/technical-documents/doc/syslog-ng-open-source-edition/3.26/administration-guide/86#TOPIC-1431237

Thanks for your notice, I've checked the whole chapter of grouping-by!
About improving the documentation:
I admit that the chapter where grouping-by() options listed is a bit dense in case of the value() in the aggregate() option.
We will discuss this with the doc writer team, when they process the grouping-by() parser example config bug I've reported.

Otherwise, I think Fabien helped you find out where is the problem: in your destination side template, you only include the .auditd. macros, which have been parsed by linux-auditd-parser().
$MESSAGE macro was missing from the template which is set by the grouping-by parser.

I don't know your use case, but I think your current solution lacks any usage of correlation: even though you set a new name-value pair in the aggregated message (.auditd.test), it's basically the same message as the last message that arrived into the same context.
As Fabien said, you will see that same message twice (the last message before the timeout expired).
The above link shows a good example (I'm copying a fixed version of it) what you can do with message contexts:

aggregate(
  ...
  value('MESSAGE' 'An SSH session for ${SSH_USERNAME}@1 from ${SSH_CLIENT_ADDRESS}@2 closed. Session lasted from ${DATE}@2 to ${DATE}')
)

Regards,
Gabor

________________________________
From: syslog-ng <syslog-ng-bounces at lists.balabit.hu> on behalf of Fabien Wernli <wernli at in2p3.fr>
Sent: Tuesday, November 3, 2020 15:32
To: Maciek Solnicki <msolnicki at gmail.com>
Cc: Syslog-ng users' and developers' mailing list <syslog-ng at lists.balabit.hu>
Subject: Re: [syslog-ng] Requesting help with Grouping-by function

CAUTION: This email originated from outside of the organization. Do not follow guidance, click links, or open attachments unless you recognize the sender and know the content is safe.


Hi,

If you want to see more Macros in json you can use scopes, for instance:

   format-json -s nv-pairs     # all generic non-dot macros
   format-json -s all-nv-pairs # all generic macros
   format-json -s everything   # as advertised

cheers

______________________________________________________________________________
Member info: https://nam05.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.balabit.hu%2Fmailman%2Flistinfo%2Fsyslog-ng&data=04%7C01%7Cgabor.nagy%40oneidentity.com%7C5fb2a20040a54610eba508d8800559da%7C91c369b51c9e439c989c1867ec606603%7C0%7C1%7C637400107786812381%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=qHmBE%2BaE5kDGmXbGg2E1KdJCKmHp1%2Bw62Uy7BupqlLM%3D&reserved=0
Documentation: https://nam05.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fsupport%2Fdocumentation%2F%3Fproduct%3Dsyslog-ng&data=04%7C01%7Cgabor.nagy%40oneidentity.com%7C5fb2a20040a54610eba508d8800559da%7C91c369b51c9e439c989c1867ec606603%7C0%7C1%7C637400107786812381%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=l0Y1NnxYxEyQstr7N%2Bp%2BKkLTIMvUt6ATDec8B18ufZo%3D&reserved=0
FAQ: https://nam05.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fwiki%2Fsyslog-ng-faq&data=04%7C01%7Cgabor.nagy%40oneidentity.com%7C5fb2a20040a54610eba508d8800559da%7C91c369b51c9e439c989c1867ec606603%7C0%7C1%7C637400107786812381%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=dp%2FGkudcz87cnFnuj12bSKjc4TZP1YsoWZnHi1uHmXE%3D&reserved=0

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20201103/72751d9f/attachment-0001.html>