On Fri, 2011-09-02 at 13:21 +0200, Gergely Nagy wrote:
Paul Krizak <paul.krizak@amd.com> writes:
We had this problem at AMD. The problem turned out to be that /dev/console was attached to a device (an iLO in our case) that went offline occasionally and would block on writes. We fixed it by updating our syslog-ng.conf to write to the console using a pipe() directive instead of file().
You may have something similar, especially if there are occasional messages that get routed to /dev/console (or any other pipe/device that may block).
I managed to reliably reproduce the problem, thanks to the above suggestion:
source s_src { system(); tcp(port(12345)); }; destination d_xconsole { pipe("/dev/xconsole"); }; log { source(s_src); destination(d_xconsole); };
Throwing a few thousand logs on this while there's nothing listening on the other end of /dev/xconsole will eventually hang syslog-ng 3.3, even if I use pipe() instead of file(). Emptying /dev/xconsole will, as expected, restore normal operation.
This used to work with previous releases, and judging by Paul's mail, it works with whatever version they have (which, I assume, is not 3.3).
Since I never ever use xconsole, I just removed that destination from my config, but the underlying bug should be fixed nevertheless (unless this is the expected behaviour, which I doubt).
This patch fixes this issue: commit bb0057e78b81c574f2ce677d13d23ac6df7ac057 Author: Balazs Scheidler <bazsi@balabit.hu> Date: Sat Sep 3 10:37:28 2011 +0200 pipe/file destination: fixed flipped slow-flow-control state The condition to enable soft flow control was flipped between file and pipe destinations. Files should have had it enabled, pipe() disabled. This patch fixes that. Reported-By: Gergely Nagy <algernon@balabit.hu> Signed-off-by: Balazs Scheidler <bazsi@balabit.hu> -- Bazsi