My recommendation would be to write a very simple program that does it and use the program() driver to send your data and the JSON template to pre-format it as JSON. You could probably do exactly what you're describing in 10 lines or less of Perl/Python/Ruby/etc. Here's one in Perl: syslog-ng json template: $(format_json --select .classifier.* --select usracct.* --exclude *.*id) destination d_rabbitmq { program("/usr/local/bin/to_rabbitmq.pl"); }; #!/usr/bin/perl use strict; use Net::RabbitMQ; use JSON; my $mq = Net::RabbitMQ->new(); $mq->connect("localhost", { user => "guest", password => "guest" }); $mq->channel_open(1); while (<>){ $mq->publish(1, "queuename", $_ ); # message is already JSON from Syslog-NG } $mq->disconnect(); <shameless project promotion> Logstash is nice and well polished. If you find that you are getting good results from parsing and classifying your logs with pattern-db but ElasticSearch is too slow when you start getting higher volumes or you want additional features like per-log permissions, transforms, and other plugins, please check out ELSA (enterprise-log-search-and-archive.googlecode.com). </shameless project promotion> On Tue, Feb 7, 2012 at 2:29 PM, Thomas Wollner <tw@wollner-net.de> wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Hi List,
some days ago I found logstash and I`m very impressed about the capabilities. It took me half an hour to have a running system and with patternized messages it is real fun to search and find logs. My plans are to classify messages with patterndb and deliver json-format to logstash via amqp for further indexing and searching. I think the options to the driver should be the same like monogdb destination driver, only the delivery is different. do not insert into database, but write it to an amqp enabled message queue like rabbitmq.
are there any plans to create an AMQP destination driver? any thoughts on that?
best regards,
Tom -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iD8DBQFPMYmRTCCRT+dccOYRAjKmAJ9nfAtCo4aIumsK97ebmXaRM9TO/gCeO+D7 JiHk0SOdi/E3AkzyBo462/0= =Xt93 -----END PGP SIGNATURE----- ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq