[syslog-ng] character handling in kv-parser

[Balazs Scheidler]

btw: this is the branch that adds '.' as a possible key delimiter:

https://github.com/balabit/syslog-ng/pull/1271


On Sun, Dec 11, 2016 at 7:13 AM, Balazs Scheidler <bazsi77 at gmail.com> wrote:

> Keys have to consist of [A-Za-z0-9-_] and doing a restriction is pretty
> important to identify keys in odd formats.
>
> We can change the set of characters pretty easily but for now it is
> hardcoded.
>
> I have a pending patchset that adds '.' as a char, as I needed that, and
> that's what syslog-ng uses to separate key portions.
>
> Would using a dot work for you? Then it would ve pretty easy to do the
> json conversion right within syslog-ng.
>
>
> Something like:
>
> parser { kv-parser(); };
> destination { file("foobar" template("$(format-json *)\n")); };
>
> Format json would generate embedded objects at dot boundaries.
>
> On Dec 11, 2016 00:03, "David Boles (dboles)" <dboles at micron.com> wrote:
>
>> I am trying to process structured data coming from the Linux kernel's
>> printk_emit() function which shows up on /dev/kmesg. Since printk_emit()
>> forces you to flatten hierarchical data into key/value pairs I use a
>> character delimiter between pieces of the key. So, with the '!' character
>> as a separator something that looks like { "cat": { "family" : "mammal" } }
>> would become "cat!family=mammal".
>>
>> When I use a kv-parser, and I use characters like '.', '$', '!', '+' or
>> '^' as a separator the parser seems to just discard everything in the key
>> except the last part. Alphabetic characters and '_' appear to work - but
>> they are the most likely to conflict with other strings.
>>
>> Is there some way to tell the kv-parser to only consider '=' as special
>> (or whatever the separator was defined to be) and keep my keys intact?
>>
>> Thanks,
>>
>>  - db
>>
>> ____________________________________________________________
>> __________________
>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>> Documentation: http://www.balabit.com/support/documentation/?product=
>> syslog-ng
>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>
>>


-- 
Bazsi
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20161211/a6adcc44/attachment.html>