Hi,

Thanks for the compliments :) Right now syslog-ng doesn't really support arrays, although we had plans about those in the past, but nothing concrete yet.

syslog-ng has builtin support for tags (e.g. the tags() option for various sources and the db-parser()/patterndb configuration files), but those can also be limited a bit. Can you elaborate about your usecase? What part of your setup would associate the tags with the message?

To add arrays to syslog-ng, one would need to add the appropriate logic to $(format-json), we've figured that the flat name-value pairs structure of syslog-ng would simply be formatted to be an array. Given the following set of name-value pairs:

tags[0] = 'foo'
tags[1] = 'bar'
tags[2] = 'baz'

Would become an array automatically, when formatted via format-json, e.g.

tags = [ "foo", "bar", "baz" ]

The only part missing is basically the recognition that a specific name has brackets at the end and sorting the elements properly. (right now we iterate in alphabetical order, which wouldn't work with numerical indices).

Once this is in place, we would only need to add some rewrite operations to "append"/"pop" on an existing array.

Such a contribution would be absolutely appreciated.

Cheers,

Bazsi



On Fri, Jul 18, 2014 at 6:57 PM, Radu Gheorghe <radu.gheorghe@sematext.com> wrote:
Hi,

This is my first post here, so I have to start by thanking all the contributors for an awesome product :)

My question is about adding an array to a JSON document. What I'm trying to do is to send a message like this:

@cee: {"message": "test message", "tags":["test", "message"]}

My template looks a like this:

template("@cee: $(format-json --pair message=\"$MSG\" --pair tags="test")\n")

This works fine for a single tag, but how can I add multiple ones?

The broader use-case is that I want to add tags to logs matching a specific filter. For example:
----------------------
filter user_tests { facility(user) and message(test) };

destination logsene_tests {
      transport("tcp")
      port(514)
      template("@cee: $(format-json --pair message=\"$MSG\" --pair tags=\"test\")\n")
    );
};

log { source(all_syslog); filter(user_tests); destination(logsene_tests); flags(final); };
----------------------

If there's a better way to add multiple tags to a log, please tell me - I'm good with making big changes if it leads to a cleaner/better config.

Best regards,
Radu
--
Performance Monitoring * Log Analytics * Search Analytics
Solr & Elasticsearch Support * http://sematext.com/

______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq





--
Bazsi