Balazs Scheidler wrote:
On Sat, Dec 28, 2002 at 10:01:07AM +0300, Borzenkov Andrey wrote:
Another possibility is to push buffer back into output queue when write returns EAGAIN/EWOULDBLOCK. It has some potential of blocking output to a single destination, but then if destination cannot accept input anymore, it is probably broken anyway. Messages can be dropped on input side then (i.e. if queue is full, simply no new message is queued for this destination).
This was the indented behaviour and the patch I sent in my previous mail does this.
Sounds good. This is the other solution. I'll try the patch out. Thanks a lot for it. But here are two questions for this: - file() destinations have a feature (log_fifo_size()), that allows configuration of output queue. Seems, this's a good feature for pipe() destinations, too? - But, the more important... As you say above, if the output queue is full, syslog-ng drops on the input side. But we must ensure, that _exactly_ whole messages are dropped or queued, regardless of the available queue size in bytes. This is what I meant with line based queuing. We shouldn't forget over all the I/O buffering, that our basic data structure is a newline terminated syslog message.
BTW it looks like syslog-ng makes writes in fixed 4096 buffer length. For pipes it is wrong - system guarantees atomic delivery of a single write up to PIPE_BUF; system is free to reorder parts of any single write with larger size. Portable way to query for PIPE_BUF on modern systems is to use (f)pathconf utility.
Yepp. PIPE_BUF is 4KB on linux, but 5KB on Solaris (See limits(4)). So, to ensure an atomic write(2) on Solaris, you can safely use a 5k buffer.
BTW: syslog-ng puts the pipe fd in nonblocking mode, so if the chunk is too large, write() should simply indicate that fewer bytes was written than originally asked to.
Yes. Its the usual way, use only atomic write(2) with <=PIPE_BUF on nonblocking pipes, to avoid all the stuff that can happen, if only parts of the buffer are written to the fd. -- Best regards --Andreas Schulze [phone: +49.5246.80.1275, fax: +49.5246.80.2275] | I believe, it was Dennis Ritchie who said something like: | "C is rarely the best language for a given task, | but it's often the second-best". | The implication being that: "[...]" | | sh# cat>$$.c<<EOT | main(l,a,n,d)char**a;{for(d=atoi(a[1])/10*80-atoi(a[2])/5-596;n="@NK\ | ACLCCGZAAQBEAADAFaISADJABBA^SNLGAQABDAXIMBAACTBATAHDBANZcEMMCCCCAAhE\ | IJFAEAAABAfHJETBdFLDAANEfDNBPHdBcBBBEA_AL H E L L O, W O R L D! " | [l++-3];)for(;n-->64;)putchar(!d+++33^l&1);} | EOT | gcc -o$$ $$.c;clear;./$$ 52 8;rm -f $$*