Thanks patrick. This raises another question, how can I Quantify processed logs. Like what is been processed/ un-processed/ lost. Thanks, Noel (hsxtrt) Date: Thu, 22 Apr 2010 11:26:59 -0600 From: "Patrick H." <syslogng@feystorm.net> Subject: Re: [syslog-ng] Process stored logs To: Syslog-ng users' and developers' mailing list <syslog-ng@lists.balabit.hu> Message-ID: <4BD086E3.3030200@feystorm.net> Content-Type: text/plain; charset="iso-8859-1" The log_fifo_size variable controls how many messages the output buffer will hold. So if server Z is relaying to A, and A goes down, Z will start storing messages in this buffer. Unfortunately there is no way to say 'if destination A fails, log to destination A2 (which may be a file output or something) instead'. The premium version does support disk-based buffering though, so that if log_fifo_size fills up, it'll start writing out to a disk based file instead. Sent: Thursday, April 22, 2010 12:13:40 AM From: noel anderson <nascentcatalyst@yahoo.com> To: syslog-ng@lists.balabit.hu Subject: [syslog-ng] Process stored logs
I'm building an infra across the geo's to collect logs at a central repository. I have set up syslog-ng in 3 geo's (say for e.g X, Y, Z) to collect logs form servers in respective Geo. A forurth server (say for eg. A) where the logs are forwarded from the 3 log servers to aggregate all the logs from all GEO's.
The problem where I fail to understand is, if my aggregator server (A) goes down, how do i process my stored logs on (X,) (Y), (Z), so that i do not loose any logs during my downtime.
Is it possible to process backlog of logs on the server or do i have to change my infra so that i do not loose these logs?
Thanks Noel (hsxtrt)