[syslog-ng] How to rewrite structured data - XML, Windows Events and format-json

[Várady, László]

Hi,

In your test file destination, the content of $MESSAGE is used (which is in
raw XML format).
Additionally to the raw XML format, syslog-ng parses all XML tags and
attributes and stores them as key-value pairs.
When you switch to format-json(), these key-value pairs are used instead of
$MESSAGE, that's why your rewrite rule stops working.

My suggestion would be to change the rewrite rule to remove the unwanted
text from the appropriate key-value pair.
For example,

subst(
    "This event.*",
    "DEADBEEF",
    value(".windowsevent.Event.a.b.c.Message")
    ...
);

The exact name of the key can be extracted from the JSON output.

--
László Várady


On Tue, Aug 28, 2018 at 8:31 PM Jim Hendrick <james.r.hendrick at gmail.com>
wrote:

> I am struggling with a configuration to rewrite specific sections of a
> Windows Event log message and then send them along to splunk (I think json
> is probably the easiest for splunk to ingest).
>
> Many windows events contain extraneous text (e.g. "This event.....") that
> I would like to remove.
>
> I have been able to rewrite it using a regex
> "This event.*</Message>" and replacing that with "DEADBEEF</Message>"
>
> The problem is that I can only see this in a test file destination and not
> in one that uses format-json (I set this up from the syslog-ng WEF
> configuration guide)
>
> Has anyone done anything like this?
>
> I think my problem is that I don't understand how the message is changed
> from it's arrival through processing. My test can rewrite it within MSG or
> MESSAGE - but when I try to use the rewrite() in the log statement using
> the format-json destination, nothing seems to be changed.
>
> Tips?
>
> Thanks!
> Jim
>
>
>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation:
> http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20180828/a0f7cdfc/attachment.html>