[syslog-ng] Rewrite JSON field without re-formating via custom template

[Nirgil]

Hello all,

sorry to bother you, but I'm bit lost ;]



Trying to accomplish the following:

Have a JSON inside syslog message and I need to rewrite, or better to
remove, one selected particular field, it contains lot of data and I
just want to drop this one field before forwarding to different destination.

I have created a parser for JSON:

parser p_json {

	channel {
		parser {
			json-parser (prefix(".json."));
		};
	};
};


Now I need remove or change a JSON field, lets say payload.


So I created the following rewrite rule and tried multiple options to
remove or change the content

rewrite {
	unset(value("${.json.payload}"));
	unset(value(".json.payload"));
	groupunset(values("${.json.payload}"));
	set("TEST", value(".json.payload"));
};

This is working on default fields ${HOST}, etc, but not for JSON fields.


Message can be reformatted via template ie:

template json_template {
	
	template("${DATE} ${HOST} $PROGRAM: {\"payload\"\:\""${.json.payload}"\"}")


};

In that case, json.payload can be affected by appropriate rewrite rule.

But I'd like to affect the json.payload field inside the original ${MSG}
without re-formatting whole message via custom template, because the
original message has lot of optional fields, other inner JSON's and it
is not so simple to made an universal template for this.



template json_template {
	
	template("${DATE} ${HOST} $PROGRAM: ${MSG}")

};


Is there a way how to remove JSON field from original message (${MSG})
without reformatting the whole message via custom template ?


Thanks for any thoughts !


Sample of Message:


Sep 23 15:22:48 hostname myapplication:
{"payload":"verylongpayload","field1":"value",.."lot of other fields,
included inner JSON"...}