[syslog-ng] Configuration tuning for reliability

[Ben Tisdall]

Dear list,

Apologies for the long winded post but I'd really appreciate your comments on:

1) A configuration that did not perform adequately
2) My understanding of the relevant tuneables

Number of clients: 40
Log sources per client: 35. 34 of these are pure file source driver,
system logs use the standard redhat config.
Dest: single loghost over tcp/ip, loghost uses mysql driver

Due to other pressures I made no attempt to tune parameters that
influence reliability before the test and performance was
commensurately poor (about 40% of entries discarded at the client,
confirmed by packet capture on the loghost).

Client global opts:

* log_msg_size (24576)
* log_fifo_size (1000)
* log_fetch_limit (10)
* flush_lines(0)

Client logging options:

* each source does a program-override for loghost filtering, no other processing
* flags(flow-control) NOT SET

Loghost global opts

* log_msg_size(32768)
* log_fifo_size(1000)

Loghost logging opts:

* flags(flow-control) - NOT SET
* streams undergoes a rewrite pass a simple filter (on program name),
rewrite, parse, then mysql dest - the loghost load has always been
negligable.