[syslog-ng] Process stored logs

[Patrick H.]

You cant, otherwise the fallback solution would be easy to implement. 
The problem arises from the way syslog-ng processes multiple 
destinations. If you have multiple destinations, syslog-ng hands the 
message to the first destination driver, and then hands it to the 
second. The first driver may not have even written out the message when 
the second driver gets it, it just has it in its queue. This is so that 
if you are indeed logging to multiple destinations, and the first 
destination is dead, it wont hold up the second destination.
In theory, I guess it might be possible for the destination driver to 
hand the message back to the syslog-ng core, and let it send it to an 
alternate destination, but this would have to be driver specific, as 
there is no common way of doing this that all the destination drivers 
would be able to easily implement :-(

Sent: Sunday, April 25, 2010 10:51:29 PM
From: noel anderson <nascentcatalyst at yahoo.com>
To: syslog-ng at lists.balabit.hu
Subject: Re: [syslog-ng] Process stored logs
> Thanks patrick.
>
> This raises another  question, how can I Quantify processed logs. Like what is been processed/ un-processed/ lost.
>
>
> Thanks,
> Noel (hsxtrt)
>
> Date: Thu, 22 Apr 2010 11:26:59 -0600
> From: "Patrick H." <syslogng at feystorm.net>
> Subject: Re: [syslog-ng] Process stored logs
> To: Syslog-ng users' and developers' mailing list
>     <syslog-ng at lists.balabit.hu>
> Message-ID: <4BD086E3.3030200 at feystorm.net>
> Content-Type: text/plain; charset="iso-8859-1"
>
> The log_fifo_size variable controls how many messages the output buffer 
> will hold. So if server Z is relaying to A, and A goes down, Z will 
> start storing messages in this buffer. Unfortunately there is no way to 
> say 'if destination A fails, log to destination A2 (which may be a file 
> output or something) instead'. The premium version does support 
> disk-based buffering though, so that if log_fifo_size fills up, it'll 
> start writing out to a disk based file instead.
>
> Sent: Thursday, April 22, 2010 12:13:40 AM
> From: noel anderson <nascentcatalyst at yahoo.com>
> To: syslog-ng at lists.balabit.hu
> Subject: [syslog-ng] Process stored logs
>   
>> I'm building an infra across the geo's to collect logs at a central repository. I have set up syslog-ng  in 3 geo's (say for e.g X, Y, Z) to collect logs form servers in respective Geo. A forurth server (say for eg. A)  where the logs are forwarded from the 3 log servers to aggregate all the logs from all GEO's.
>>  
>> The problem where I fail to understand is, if my aggregator server (A) goes down, how do i process my stored logs on (X,) (Y), (Z), so that i do not loose any logs during my downtime.
>>  
>> Is it possible to process backlog of logs on the server or do i have to change my infra so that i do not loose these logs?
>>
>> Thanks
>> Noel (hsxtrt)
>>     
>
>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.campin.net/syslog-ng/faq.html
>
>   
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.balabit.hu/pipermail/syslog-ng/attachments/20100426/ef9ed5fc/attachment.htm