[syslog-ng] Syslog-ng to Pipe question

[Balazs Scheidler]

On Thu, 2006-07-20 at 11:29 -0400, Valdis.Kletnieks at vt.edu wrote:
> On Thu, 20 Jul 2006 09:57:44 CDT, "SOLIS, ALEX" said:
> > I guess I forgot that a pipe is not a file.  I thought it would just
> > store the logs until the reader woke up again.  Is the pipe is limited
> > by system resources or something else?
> 
> The basic problem is that all the buffered data ends up sitting in memory,
> and usually in locked, unswappable pages.  As a result, if you let a pipe
> have a 128K buffer, then each pipe that gets stuck can potentially nail
> down 128K of RAM, causing extreme memory pressure.  (As an aside, making
> these pages swappable is often harder than you think, which is why they
> end up being nailed-down pages).
> 
> In the Linux 2.16.18-rc1-mm1 kernel I'm running now, a writer to a pipe is
> allowed up to 16 4K pages (or a total of 64K) of writes before it blocks
> waiting for a existing reader to actually accept data - and if the reader goes
> away entirely, it generates an immediate SIGPIPE.

Just one note, when using the pipe() destination in syslog-ng, it will
not generate a SIGPIPE as syslog-ng opens its pipe destination in
read-write mode, e.g. there's guaranteed to be one writer left:
syslog-ng itself.

If it is a program() destination, then you are right, if the program
goes away, syslog-ng gets a SIGPIPE, and the destination is closed. (in
syslog-ng 2.0, it is restarted in this case)

syslog-ng should however write messages to all other destinations, and
the dropped counter is global, and one last bit: file destinations never
fail to write messages, unless a write error occurs (e.g. out of disk
space).


-- 
Bazsi