Hi Bazsi, thanks you for informations. Would you be so kind and provide me a sample, where to put $(format-json --exclude *payload*) ? In fact, I need to modify ${MSG} content only - the syslog header should be unchanged.. I have tried following, but this is not working for me.. template json_template { template("${DATE} ${HOST} $PROGRAM: ${MSG}$(format-json --exclude *payload*)\n") }; Or what --scope options should I use, experimented with various scope options, but without success. Also tried --exclude * to see, if the option is removing anything, but none JSON fields are removed, it seems conversion is not working. Thank you Dne 24. 09. 20 v 7:44 Balazs Scheidler napsal(a):
Well, you could either remove the unnecessary field using a regexp but that's pretty fragile (due to escaping and stuff).
Or, you can reformat the message using the $(format-json) template function which can recreate the original json, but now with the "payload" field removed.
$(format-json) even has an --exclude option, meaning it can remove a field without having to explicitly unset it.
Bazsi
On Thu, Sep 24, 2020, 00:18 Nirgil <nirgil@honeynet.cz <mailto:nirgil@honeynet.cz>> wrote:
Hello all,
sorry to bother you, but I'm bit lost ;]
Trying to accomplish the following:
Have a JSON inside syslog message and I need to rewrite, or better to remove, one selected particular field, it contains lot of data and I just want to drop this one field before forwarding to different destination.
I have created a parser for JSON:
parser p_json {
channel { parser { json-parser (prefix(".json.")); }; }; };
Now I need remove or change a JSON field, lets say payload.
So I created the following rewrite rule and tried multiple options to remove or change the content
rewrite { unset(value("${.json.payload}")); unset(value(".json.payload")); groupunset(values("${.json.payload}")); set("TEST", value(".json.payload")); };
This is working on default fields ${HOST}, etc, but not for JSON fields.
Message can be reformatted via template ie:
template json_template {
template("${DATE} ${HOST} $PROGRAM: {\"payload\"\:\""${.json.payload}"\"}")
};
In that case, json.payload can be affected by appropriate rewrite rule.
But I'd like to affect the json.payload field inside the original ${MSG} without re-formatting whole message via custom template, because the original message has lot of optional fields, other inner JSON's and it is not so simple to made an universal template for this.
template json_template {
template("${DATE} ${HOST} $PROGRAM: ${MSG}")
};
Is there a way how to remove JSON field from original message (${MSG}) without reformatting the whole message via custom template ?
Thanks for any thoughts !
Sample of Message:
Sep 23 15:22:48 hostname myapplication: {"payload":"verylongpayload","field1":"value",.."lot of other fields, included inner JSON"...} ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq