[syslog-ng] character handling in kv-parser

[Balazs Scheidler]

Keys have to consist of [A-Za-z0-9-_] and doing a restriction is pretty
important to identify keys in odd formats.

We can change the set of characters pretty easily but for now it is
hardcoded.

I have a pending patchset that adds '.' as a char, as I needed that, and
that's what syslog-ng uses to separate key portions.

Would using a dot work for you? Then it would ve pretty easy to do the json
conversion right within syslog-ng.


Something like:

parser { kv-parser(); };
destination { file("foobar" template("$(format-json *)\n")); };

Format json would generate embedded objects at dot boundaries.

On Dec 11, 2016 00:03, "David Boles (dboles)" <dboles at micron.com> wrote:

> I am trying to process structured data coming from the Linux kernel's
> printk_emit() function which shows up on /dev/kmesg. Since printk_emit()
> forces you to flatten hierarchical data into key/value pairs I use a
> character delimiter between pieces of the key. So, with the '!' character
> as a separator something that looks like { "cat": { "family" : "mammal" } }
> would become "cat!family=mammal".
>
> When I use a kv-parser, and I use characters like '.', '$', '!', '+' or
> '^' as a separator the parser seems to just discard everything in the key
> except the last part. Alphabetic characters and '_' appear to work - but
> they are the most likely to conflict with other strings.
>
> Is there some way to tell the kv-parser to only consider '=' as special
> (or whatever the separator was defined to be) and keep my keys intact?
>
> Thanks,
>
>  - db
>
> ____________________________________________________________
> __________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?
> product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20161211/efdfd8c4/attachment.html>