On Thu, 2011-04-28 at 10:19 -0500, Martin Holste wrote:
Logging with any expensive mechanism like HTTP posts will be problematic at logging rates over a hundred or so per second unless the messages can be batched. Even then, HTTP may not be viable. In any case, definitely start with an external program until you're sure that the backend you're logging to is doing what you want it do before worrying about natively logging from syslog-ng.
Yes, this was troubling me as well. My draft proposal envisioned using an AMQP provider to ensure that the queues are retained, though this is clearly outside of the realm of syslog-ng specifically. Something like : syslog-ng -> amqp.rb -> RabbitMQ -> inserter.rb -> ES Just thinking out loud here : an AMQP driver for syslog-ng could be highly useful for a variety of potential environments, including (but not limited to) this sort of end game... -- Daniel Maher « can't talk, too busy calculating computrons. »