[syslog-ng] Syslog-ng and network failover

[Evan Rempel]

Hari Sekhon wrote:
> I don't think syslog-ng supports this nor do I know of any other logger 
> that does.
> 
> Would it be acceptable to have it log to 2 different log servers, that 
> way, you would get a measure of high availability.
> 
> 
> In any case, if you're using tcp destinations then it will hold the logs 
> until the server becomes available again (up to a point - then any 
> following logs will be lost)
> 
> You could give your clients massive fifos to try to buy more time (well 
> more log store actually), I think this will make them more resilient to 
> server unavailability as they can cache more logs.
> 
> Anyone else got any bright ideas on this?
> 
> 
> -h

There are all kinds of architectures that go a long way to addressing this.

One example;

Set up a pair of syslog servers that are configures as a high availablity active/passive pair.
These servers should share the SAME logging space. This space can be a clustered filessystem which would be best,
or could be an network mount. They could even be a single drive/lun/iscsi device.

The two servers keep a heartbeat and when one goes down, the other takes over the syslog service and the associated
IP address.

- Server A - shutdown syslog-ng
- Server A - unmount log space (clustered filesystem and network mount don't need this)
- Server A - release IP address
- Server B - take over IP address
- Server B - mount log space
- Server B - start syslog-ng

This could even happen automatically every 5 minutes so that the situations where some servers can not reach Server A 
and others can not reach Server B, they would all buffer for 5 minutes and then dump the messages to the server in the 
next 5 minutes.

This does make the log timestamps correct, but out of order on the syslog server :-(

Another example is to use 2 syslog servers, but then you have to merge the log files, which may be non trivial if you 
have lots of log messages (we have 10G each day).

Finally, you can wait for someone to implement disk based buffering in syslog-ng. It is on the wish list, but there are 
only so many hours in a day :-(

Evan.