[syslog-ng] RFC: value-pairs key rewrite framework, part N+1

[Gergely Nagy]

Gergely Nagy <algernon at balabit.hu> writes:

>>> Something along the lines of:
>>>
>>> $(format-json --key .cee.* --rewrite replace .cee=Event)
>>
>> This is not done yet, either. I'll make key() take a glob first, then
>> proceed with the commandline support.
>
> Similarly, this will turn into:
>
> $(format-json --key .cee.* --rekey .cee.* replace .cee=Event)
>
> (--key includes it, --rekey rewrites the keys)

Ladies and gentlemen, I present you the final piece of the value-pairs
key rewrite patchset:

 https://github.com/algernon/syslog-ng/compare/algernon:upstream/mirror/3.4...feature/3.4/value-pairs/rekey

Now, it is possible to create a config like the following:

source s_cee { tcp(port(12345) flags(no-parse)); };
parser p_cee { json-parser(prefix(".cee.")); };
template t_cee { template("$(format-json --key .cee.* --rekey .cee.* --shift 4)\n"); };
destination d_json { file("/var/log/cee.json" template(t_cee)); };
log { source(s_cee); parser(p_cee); destination(d_cee); };

And we should get back the same JSON that entered. More or less,
anyway. The JSON parser can't handle nested objects yet (nor can
format-json). But that will be the next step.

The current state of the key rewriting branch is, in my opinion, pretty
good. It could, perhaps, use a little bit of cleanup here and there, and
then it's merge ready as far as I can see.

However, the scratch-buffer patch needs to go in first, and I'll rebase
the rekey work on top of that then. At the moment, the rekey branch is
still using non-thread-safe GStrings.

-- 
|8]