You should not be having problems with your load. We had a thread earlier this year ("UDP packet loss with syslog-ng") in which Lars identified similar performance issues on RHEL. His problems were solved by setting the net.core.rmem_default to 2MB using sysctl. I would try setting that and then checking your performance. On Thu, Nov 11, 2010 at 7:53 AM, Ben Tisdall <ben.tisdall@photobox.com> wrote:
Dear list,
Apologies for the long winded post but I'd really appreciate your comments on:
1) A configuration that did not perform adequately 2) My understanding of the relevant tuneables
Number of clients: 40 Log sources per client: 35. 34 of these are pure file source driver, system logs use the standard redhat config. Dest: single loghost over tcp/ip, loghost uses mysql driver
Due to other pressures I made no attempt to tune parameters that influence reliability before the test and performance was commensurately poor (about 40% of entries discarded at the client, confirmed by packet capture on the loghost).
Client global opts:
* log_msg_size (24576) * log_fifo_size (1000) * log_fetch_limit (10) * flush_lines(0)
Client logging options:
* each source does a program-override for loghost filtering, no other processing * flags(flow-control) NOT SET
Loghost global opts
* log_msg_size(32768) * log_fifo_size(1000)
Loghost logging opts:
* flags(flow-control) - NOT SET * streams undergoes a rewrite pass a simple filter (on program name), rewrite, parse, then mysql dest - the loghost load has always been negligable.
From what I NOW understand I need to do something like this:
Client:
log_iw_size >= SOURCES_PER_CLIENT * log_fetch_limit
eg 35 * 10 = 350
log_fifo_size >= SOURCES_PER_CLIENT * log_fetch_limit
eg 35 * 10 = 350
AND
log_fifo_size >= SOURCES_PER_CLIENT * log_iw_size
eg 35 * 350 = 12250
So it appears to me that setting log_fifo_size to > 12250 would be correct.
Loghost
Less idea about this, do I need:
log_iw_size >= NUMBER_OF_CLIENTS * log_fetch_limit ( * SOURCES_PER_CLIENT ? )
eg 40 * 10 * 35 = 14000
And log_fifo_size >= log_iw_size ?
Is flow control important between the network source and the mysql dests?
Further information:
* Some of the larger logfiles output ~ 4 lines/sec * Approx 4G aggregate logs generated over 14 hours by the 40 hosts * Reliability is more important than speed - these logs are not analysed in real time. That said, the client should have to spend hours completing the log transfer once its workload has been processed..
Again, apologies for the length of the post and many thanks in advance for any help.
Ben Tisdall PhotoBox ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html