<div dir="ltr">Hello Bazsi, and thanks for your reply!<div><br></div><div>My use-case is with clients sending logs to <a href="http://sematext.com/logsene/index.html">Logsene</a>, which accepts Elasticsearch-style JSON over HTTP or N flavors of syslog.</div>
<div><br></div><div>Some clients may want to add tags to their logs on their way to Logsene. For example, if a message is an error and contains some text, you would give it a certain tag. The tag itself would be a part of the CEE-formatted JSON over syslog. This works well and we've <a href="https://sematext.atlassian.net/wiki/display/PUBLOGSENE/syslog-ng#syslog-ng-Tagyourlogs">documented</a> it, but right now we can't figure out how to add multiple tags.</div>
<div><br></div><div>I thought that maybe I'm missing something that's already possible. Thanks a lot for clarifying!</div><div><div class="gmail_extra"><br></div><div class="gmail_extra">Best regards,</div><div class="gmail_extra">
Radu<br clear="all"><div><div dir="ltr"><div>--</div><div>Performance Monitoring * Log Analytics * Search Analytics</div><div><span style="font-family:arial,sans-serif;font-size:13px">Solr & Elasticsearch Support * </span><a href="http://sematext.com/" style="font-size:13px;font-family:arial,sans-serif" target="_blank">http://sematext.com/</a></div>
</div></div>
<br><br><div class="gmail_quote">On Sun, Jul 20, 2014 at 10:06 AM, Balazs Scheidler <span dir="ltr"><<a href="mailto:bazsi77@gmail.com" target="_blank">bazsi77@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr"><div><div><div><div>Hi,<br><br></div>Thanks for the compliments :) Right now syslog-ng doesn't really support arrays, although we had plans about those in the past, but nothing concrete yet.<br><br></div>
syslog-ng has builtin support for tags (e.g. the tags() option for various sources and the db-parser()/patterndb configuration files), but those can also be limited a bit. Can you elaborate about your usecase? What part of your setup would associate the tags with the message?<br>
<br></div>To add arrays to syslog-ng, one would need to add the appropriate logic to $(format-json), we've figured that the flat name-value pairs structure of syslog-ng would simply be formatted to be an array. Given the following set of name-value pairs:<br>
<br></div><div>tags[0] = 'foo'<br></div><div>tags[1] = 'bar'<br></div><div>tags[2] = 'baz'<br><br>Would become an array automatically, when formatted via format-json, e.g.<br><br></div><div>tags = [ "foo", "bar", "baz" ]<br>
</div><div><br>The only part missing is basically the recognition that a specific name has brackets at the end and sorting the elements properly. (right now we iterate in alphabetical order, which wouldn't work with numerical indices).<br>
<br></div><div>Once this is in place, we would only need to add some rewrite operations to "append"/"pop" on an existing array.<br><br></div><div>Such a contribution would be absolutely appreciated.<br>
<br></div><div>Cheers,<br><br>Bazsi<br></div><div><br></div></div><div class="gmail_extra"><br><br><div class="gmail_quote"><div class="">On Fri, Jul 18, 2014 at 6:57 PM, Radu Gheorghe <span dir="ltr"><<a href="mailto:radu.gheorghe@sematext.com" target="_blank">radu.gheorghe@sematext.com</a>></span> wrote:<br>
</div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div class="h5"><div dir="ltr">Hi,<div><br></div><div>This is my first post here, so I have to start by thanking all the contributors for an awesome product :)</div>
<div><br></div><div>My question is about adding an array to a JSON document. What I'm trying to do is to send a message like this:</div>
<div><br></div><div>@cee: {"message": "test message", "tags":["test", "message"]}</div><div><br></div><div>My template looks a like this:</div><div><br></div><div>template("@cee: $(format-json --pair message=\"$MSG\" --pair tags="test")\n")<br>
</div><div><br></div><div>This works fine for a single tag, but how can I add multiple ones?</div><div><br></div><div>The broader use-case is that I want to add tags to logs matching a specific filter. For example:</div>
<div>
----------------------</div><div><div>filter user_tests { facility(user) and message(test) };</div><div><br></div><div>destination logsene_tests {</div><div> syslog("<a href="http://logsene-receiver-syslog.sematext.com" target="_blank">logsene-receiver-syslog.sematext.com</a>"</div>
<div> transport("tcp")</div><div> port(514)</div><div> template("@cee: $(format-json --pair message=\"$MSG\" --pair tags=\"test\")\n")<br></div><div> );</div><div>
};</div></div><div><br></div><div>log { source(all_syslog); filter(user_tests); destination(logsene_tests); flags(final); };<br></div><div><div>----------------------</div><div></div></div><div><br></div><div>If there's a better way to add multiple tags to a log, please tell me - I'm good with making big changes if it leads to a cleaner/better config.</div>
<div><br></div><div>Best regards,</div><div>Radu</div><div><div><div dir="ltr"><div>--</div><div>Performance Monitoring * Log Analytics * Search Analytics</div><div><span style="font-family:arial,sans-serif;font-size:13px">Solr & Elasticsearch Support * </span><a href="http://sematext.com/" style="font-size:13px;font-family:arial,sans-serif" target="_blank">http://sematext.com/</a></div>
</div></div>
</div></div>
<br></div></div>______________________________________________________________________________<br>
Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" target="_blank">https://lists.balabit.hu/mailman/listinfo/syslog-ng</a><br>
Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" target="_blank">http://www.balabit.com/support/documentation/?product=syslog-ng</a><br>
FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" target="_blank">http://www.balabit.com/wiki/syslog-ng-faq</a><br>
<br>
<br></blockquote></div><span class="HOEnZb"><font color="#888888"><br><br clear="all"><br>-- <br>Bazsi
</font></span></div>
<br>______________________________________________________________________________<br>
Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" target="_blank">https://lists.balabit.hu/mailman/listinfo/syslog-ng</a><br>
Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" target="_blank">http://www.balabit.com/support/documentation/?product=syslog-ng</a><br>
FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" target="_blank">http://www.balabit.com/wiki/syslog-ng-faq</a><br>
<br>
<br></blockquote></div><br></div></div></div>