On Mon, 2010-01-11 at 11:38 -0500, Doug Warner wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 01/05/2010 11:53 AM, Doug Warner wrote:
On 12/21/2009 09:41 AM, Doug Warner wrote:
On 12/21/2009 02:50 AM, Balazs Scheidler wrote:
Periodically when syslog-ng seems to be working fine we'll see syslog-ng using
a bit of CPU (~20%), but more commonly right now we see it using very little and the recv-q building up.
Non of our src/dest in syslog-ng have any flags set other than the above global options; so whatever is the default for 3.0.4 OSE should be what we're using. Strange. Can you start strace on the syslog-ng process, preferably with timestamps (-ttT -s 256 -p <pid>) options to see what it is doing?
Just as a follow-up, I was able to solve this by setting flow-control on my log path and increasing my log_fetch_limit() and log_iw_size() on my tcp source and increasing log_fifo_size() globally. Doing these 4 things brought my memory consumption back in line and syslog-ng seems to be processing the logs at the proper pace again.
Great to know, and sorry for not answering any sooner. I was a bit overwhelmed with other stuff. Since the settings that you changed are somewhat contradictory, I guess a smaller set of changes would have resolved the issue as well. log_fetch_limit() - with this you increased the number of messages syslog-ng would receive with a single main loop iteration. the default value here is 10, which is quite conservative. If you have a lot of connections, increasing this value to a very high number could make some connections to starve. Enabling flow control will only help throughput if otherwise syslog-ng drops messages actively (e.g. receive it and then decide there's no space in the destination FIFO). If you enable flow control, the kernel would drop/stall messages (instead of syslog-ng), which makes syslog-ng to care about output messages instead of receiving them just to drop them. I think in your specific case the increase of fetch-limit was the solution. But increasing fifo_size() when fetch_limit() is increased is a wise thing to do. I doubt enabling flow-control would have a real effect. -- Bazsi