On Thu, 2006-07-20 at 11:29 -0400, Valdis.Kletnieks@vt.edu wrote:
On Thu, 20 Jul 2006 09:57:44 CDT, "SOLIS, ALEX" said:
I guess I forgot that a pipe is not a file. I thought it would just store the logs until the reader woke up again. Is the pipe is limited by system resources or something else?
The basic problem is that all the buffered data ends up sitting in memory, and usually in locked, unswappable pages. As a result, if you let a pipe have a 128K buffer, then each pipe that gets stuck can potentially nail down 128K of RAM, causing extreme memory pressure. (As an aside, making these pages swappable is often harder than you think, which is why they end up being nailed-down pages).
In the Linux 2.16.18-rc1-mm1 kernel I'm running now, a writer to a pipe is allowed up to 16 4K pages (or a total of 64K) of writes before it blocks waiting for a existing reader to actually accept data - and if the reader goes away entirely, it generates an immediate SIGPIPE.
Just one note, when using the pipe() destination in syslog-ng, it will not generate a SIGPIPE as syslog-ng opens its pipe destination in read-write mode, e.g. there's guaranteed to be one writer left: syslog-ng itself. If it is a program() destination, then you are right, if the program goes away, syslog-ng gets a SIGPIPE, and the destination is closed. (in syslog-ng 2.0, it is restarted in this case) syslog-ng should however write messages to all other destinations, and the dropped counter is global, and one last bit: file destinations never fail to write messages, unless a write error occurs (e.g. out of disk space). -- Bazsi