You cant, otherwise the fallback solution would be easy to implement. The problem arises from the way syslog-ng processes multiple destinations. If you have multiple destinations, syslog-ng hands the message to the first destination driver, and then hands it to the second. The first driver may not have even written out the message when the second driver gets it, it just has it in its queue. This is so that if you are indeed logging to multiple destinations, and the first destination is dead, it wont hold up the second destination. In theory, I guess it might be possible for the destination driver to hand the message back to the syslog-ng core, and let it send it to an alternate destination, but this would have to be driver specific, as there is no common way of doing this that all the destination drivers would be able to easily implement :-( Sent: Sunday, April 25, 2010 10:51:29 PM From: noel anderson <nascentcatalyst@yahoo.com> To: syslog-ng@lists.balabit.hu Subject: Re: [syslog-ng] Process stored logs
Thanks patrick.
This raises another question, how can I Quantify processed logs. Like what is been processed/ un-processed/ lost.
Thanks, Noel (hsxtrt)
Date: Thu, 22 Apr 2010 11:26:59 -0600 From: "Patrick H." <syslogng@feystorm.net> Subject: Re: [syslog-ng] Process stored logs To: Syslog-ng users' and developers' mailing list <syslog-ng@lists.balabit.hu> Message-ID: <4BD086E3.3030200@feystorm.net> Content-Type: text/plain; charset="iso-8859-1"
The log_fifo_size variable controls how many messages the output buffer will hold. So if server Z is relaying to A, and A goes down, Z will start storing messages in this buffer. Unfortunately there is no way to say 'if destination A fails, log to destination A2 (which may be a file output or something) instead'. The premium version does support disk-based buffering though, so that if log_fifo_size fills up, it'll start writing out to a disk based file instead.
Sent: Thursday, April 22, 2010 12:13:40 AM From: noel anderson <nascentcatalyst@yahoo.com> To: syslog-ng@lists.balabit.hu Subject: [syslog-ng] Process stored logs
I'm building an infra across the geo's to collect logs at a central repository. I have set up syslog-ng in 3 geo's (say for e.g X, Y, Z) to collect logs form servers in respective Geo. A forurth server (say for eg. A) where the logs are forwarded from the 3 log servers to aggregate all the logs from all GEO's.
The problem where I fail to understand is, if my aggregator server (A) goes down, how do i process my stored logs on (X,) (Y), (Z), so that i do not loose any logs during my downtime.
Is it possible to process backlog of logs on the server or do i have to change my infra so that i do not loose these logs?
Thanks Noel (hsxtrt)
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html