[RFC]: value-pairs key rewriting outside of key()
Hi! One thing I realized just now, is that there was a slight design mistake made when doing the value-pairs() key rewriting functionality: it was baked into the key() statement, so that as of this writing, one can only rewrite keys that were explicitly selected. While this sounds good on paper, it has one major drawback: If I want to rewrite keys that were selected by the scope, I have to explicitly select those keys myself. This makes a lot of things far more complicated than they need be, and this is why there's a hack in the mongodb destination that replaces leading dots with an underscore. Instead of doing that, I'd like to move the mongodb destination to use the key rewriting functionality, and while there, also teach $(format-json) to do something similar. Fortunately, it's only the config parser that needs an update, as the code below it was written in a way that allows me to do what I want. So, I'd like to add another way to configure key rewriting: value-pairs(scope(nv-pairs) rekey(".*", shift(1), add-prefix("_"))) This is very similar to the already existing way, except it's not inside a key(), but in the main value-pairs() body, thus, it will not add extra keys to the set, but will apply the transformations to keys already in it. For format-json, it would be fairly similar: $(format-json --scope nv-pairs --rekey .* --shift 1 --add-prefix "_") This functionality will then make it possible to rewrite keys that were implicitly added to the set by the scope, which in turn would make me much happier. As an FYI, I'll push this change to my merge-queue/3.4 branch tomorrow, unless anyone objects to it, or has a better suggestion by then. -- |8]
participants (1)
-
Gergely Nagy