[syslog-ng] Process stored logs

[noel anderson]

Thanks patrick.

This raises another  question, how can I Quantify processed logs. Like what is been processed/ un-processed/ lost.


Thanks,
Noel (hsxtrt)

Date: Thu, 22 Apr 2010 11:26:59 -0600
From: "Patrick H." <syslogng at feystorm.net>
Subject: Re: [syslog-ng] Process stored logs
To: Syslog-ng users' and developers' mailing list
    <syslog-ng at lists.balabit.hu>
Message-ID: <4BD086E3.3030200 at feystorm.net>
Content-Type: text/plain; charset="iso-8859-1"

The log_fifo_size variable controls how many messages the output buffer 
will hold. So if server Z is relaying to A, and A goes down, Z will 
start storing messages in this buffer. Unfortunately there is no way to 
say 'if destination A fails, log to destination A2 (which may be a file 
output or something) instead'. The premium version does support 
disk-based buffering though, so that if log_fifo_size fills up, it'll 
start writing out to a disk based file instead.

Sent: Thursday, April 22, 2010 12:13:40 AM
From: noel anderson <nascentcatalyst at yahoo.com>
To: syslog-ng at lists.balabit.hu
Subject: [syslog-ng] Process stored logs
> I'm building an infra across the geo's to collect logs at a central repository. I have set up syslog-ng  in 3 geo's (say for e.g X, Y, Z) to collect logs form servers in respective Geo. A forurth server (say for eg. A)  where the logs are forwarded from the 3 log servers to aggregate all the logs from all GEO's.
>  
> The problem where I fail to understand is, if my aggregator server (A) goes down, how do i process my stored logs on (X,) (Y), (Z), so that i do not loose any logs during my downtime.
>  
> Is it possible to process backlog of logs on the server or do i have to change my infra so that i do not loose these logs?
>
> Thanks
> Noel (hsxtrt)