[syslog-ng] program destination application status

[Balazs Scheidler]

On Wed, 2009-08-26 at 15:43 +0530, Jain, Vaibhav (GE Healthcare) wrote:
> Hi Marton,
> 
> Thanks for your quick response.
> 
> If program stops reading the message from stdin for some time, then in
> that time, will program lose the log message from syslog-ng ? Or syslog
> will starts feeding the logs from next to the last log mesg sent to
> program.

there's a buffer between your program and syslog-ng: the pipe buffer.
This buffer is usually 8k in size, but that can depend from system to
system.

The flow of data between syslog-ng and your program works as follows:
  * syslog-ng writes messages to this buffer if there's space and
suspends writing if the buffer is full
  * your program reads messages from the buffer if there's data

This boils down to the following: 
  * if your program is slower than syslog-ng, then this 8k of buffer
becomes full quite fast, which syslog-ng notices and waits for the
program to process more messages
  * if your program crashes, then this 8k of data can be lost, since
syslog-ng has no way to differentiate between data sent to the buffer
and data already processed by your application.

Also, if there's a stall in the output processing in syslog-ng, it will
suspend reading on its input, if you enable flow control on your
logpath.


> 
> If my program is slower than incoming freq of syslog-ng logs then in
> that case will program lose the log mesgs from syslog-ng? 
> 
> If syslog-ng stops reading logs from its source for some time then in
> that case will syslog-ng lose the mesg from source ?  If source is tcp
> port.
> 
> Note: I am using open source version of syslog-ng (3.0.3-1 On rhel5)

-- 
Bazsi