[syslog-ng] Requesting help with Grouping-by function

[Maciek Solnicki]

Nevermind! I think I got it :-)

*log {*
*  source {*
*    file('/var/log/audit/audit.log' flags(no-parse));*
*  };*
*  parser {*
*    linux-audit-parser(prefix(".auditd."));*
*  };*
*  parser {*
*    grouping-by(*
*        key("${.auditd.msg}")*
*        aggregate(*
*                value(".auditd.test" "#########TEST")*
*        )*
*        timeout(10)*
*    );*
*  };*
*  destination {*
*    file('/tmp/test.json' template("$(format-json .auditd.exe .auditd.cwd
.auditd.a* .auditd.test .auditd.success .auditd.pid .auditd.msg)\n"));*
*  };*
*};*

This seems to work as now result looks like this:

*{*
*  "_auditd": {*
*    "test": "#########TEST",*
*    "success": "yes",*
*    "pid": "57483",*
*    "msg": "audit(1604412718.657:211575):",*
*    "exe": "/usr/bin/jq",*
*    "cwd": "/etc/syslog-ng/conf.d",*
*    "auid": "1000",*
*    "argc": "1",*
*    "arch": "c000003e",*
*    "a3": "fffffffffffff878",*
*    "a2": "564235a17dd0",*
*    "a1": "564235a20680",*
*    "a0": "jq"*
*  }*
*}*

If you have any improvements ideas please let me know.

Thanks!
Maciej

wt., 3 lis 2020 o 14:30 Maciek Solnicki <msolnicki at gmail.com> napisał(a):

> Hi Fabien,
>
> Thanks for your response.
>
> I was using this snippet of code (taken from documentation) before, but
> doesn't seem to work either:
>
> *value("MESSAGE" "$(format-json .auditd.*)")*
>
> So how do I output an aggregated message to the test.json file ? Or any
> file.
>
> I apologize if my questions are basic, but I had a hard time finding
> answers in documentation. Grouping-by() function could definitely use more
> explanation.
>
> Kind regards
> Maciej
>
> wt., 3 lis 2020 o 13:05 Fabien Wernli <wernli at in2p3.fr> napisał(a):
>
>> Hi Maciek,
>>
>> On Tue, Nov 03, 2020 at 12:24:40PM +0100, Maciek Solnicki wrote:
>> > *  destination {*
>> > *    file('/tmp/test.json' template("$(format-json .auditd.*)\n"));*
>> > *  };*
>> > *};*
>>
>> This means you're outputting the contents of all `.auditd.` macros to file
>> test.json.
>>
>> But your grouping-by parser generates a message with the macro MESSAGE set
>> to the value "TEST" :
>>
>> > *    grouping-by(*
>> > *      key("${.auditd.msg}")*
>> > *      timeout(10)*
>> > *      aggregate(value("MESSAGE" "TEST"))*
>> > *    );*
>>
>> So you won't see it in test.json.
>> Well, more exactly you should see it as the default value of
>> aggregate(inherit-mode()) is "context", but you won't see the MESSAGE:TEST
>> macro as you're omitting it from your output.
>> I'm guessing you should see a message twice in that case.
>>
>>
>> ______________________________________________________________________________
>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>> Documentation:
>> http://www.balabit.com/support/documentation/?product=syslog-ng
>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20201103/d16188ba/attachment.html>