[syslog-ng] RFC: value-pairs key rewrite framework, part N+1

[Gergely Nagy]

Gergely Nagy <algernon at balabit.hu> writes:

>> We've discussed this IRL and came to the conclusion that it is very
>> handy to allow key-rewrite to be applied on a per-glob basis (e.g. to
>> associate the rewrite function to the set specified by --key).
>
> This is now (partially) done on my feature/3.4/value-pairs/rekey
> branch. Since it's still a work in progress, I'm not including the
> merged patches yet, but give a pointer to a diff between 3.4 master and
> my branch instead:
>
> https://github.com/algernon/syslog-ng/compare/algernon:upstream/mirror/3.4...algernon:feature/3.4/value-pairs/rekey

The branch is now updated a bit, and I ran into a silly issue, that
currently prevents me from wiring rekey() into key(): key() is used to
add extra elements into the set, so turning it into a match-only thing
is not really an option.

Ie, if I want to rewrite every key, and prefix them with "foo.", I
currently do this:

value-pairs(scope(dot-nv-pairs)
            rekey("*" add-prefix("foo."))
);

However, if I keep key()'s current behaviour of adding stuff to the set,
and wire rekey into it:

value-pairs(scope(dot-nv-pairs)
            key("*" rekey(add-prefix("foo.")))
);

This will do something completely different: it will also include EVERY
key, despite our scope. And we can't rewrite everything that's *in* the
scope already, but nothing else.

So I either change the behaviour of key(), which I wouldn't want to, or
I keep rekey() separate.

I believe keeping rekey() separate is the better, and more flexible
option.

>> Something along the lines of:
>>
>> $(format-json --key .cee.* --rewrite replace .cee=Event)
>
> This is not done yet, either. I'll make key() take a glob first, then
> proceed with the commandline support.

Similarly, this will turn into:

$(format-json --key .cee.* --rekey .cee.* replace .cee=Event)

(--key includes it, --rekey rewrites the keys)

Perhaps a bit more verbose, but mostly backwards compatible.

Anyway, the current tip of my feature/3.4/value-pairs/rekey branch also
modifies the behaviour of key(): it now accepts a glob, and will include
every key in the set that matches the glob. Except if it is excluded by
a later exclude(). The key() and exclude() options are now evaluated in
order, and the last one wins.

So value-pairs(key(".cee.*") exclude(".*")) will end up with an empty
set.

-- 
|8]