Well, I have found the answer to the reliability problem! The flaw is in this code in pkt_buffer.c: static int do_write_str(struct abstract_write *c, struct ol_string *string) { CAST(pkt_buffer, self, c); debug("do_write_str\n"); if (self->super.closed) { ol_string_free(string); return ST_FAIL | ST_CLOSE; } if (self->queue_size == self->queue_max) { /* fifo full */ ol_string_free(string); return ST_FAIL | ST_OK; } So you can see that when the queue is full, the messages simply get dropped. That's bad! The solution is to flush the queue when it is full, and then continue looping over the available input. The question is, what code should be responsible for doing this flushing? Shouldn't some piece of code be checking for the ST_FAIL return code? Cheers, Jeffrey
So you can see that when the queue is full, the messages simply get dropped. That's bad!
The solution is to flush the queue when it is full, and then continue looping over the available input. The question is, what code should be responsible for doing this flushing? Shouldn't some piece of code be checking for the ST_FAIL return code?
Could you explain why it is better to flush already buffered data? -- Bazsi PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1 url: http://www.balabit.hu/pgpkey.txt
On Sat, 30 Sep 2000, Balazs Scheidler wrote:
So you can see that when the queue is full, the messages simply get dropped. That's bad!
The solution is to flush the queue when it is full, and then continue looping over the available input. The question is, what code should be responsible for doing this flushing? Shouldn't some piece of code be checking for the ST_FAIL return code?
Could you explain why it is better to flush already buffered data?
Yes. Assume that you have a queue which can hold N, and initially holds n == 0 entries. You have a source which can produce entries at the rate p', and a destination that can accept entries at the rate of r'. r' < p'. Therefore, the queue will become full at the rate n' = p' - r'. Eventually n will reach N, and the queue must be flushed immediately, regardless of the fact that more entries will be read from the source. The only other option is to simply throw away the extra entries from the source, which is what is currently happening. Note that using the log_fifo_size() option to increase N is not acceptable, since at steady-state that means that syslog-ng's memory consumption will grow without bounds. -jwb
participants (2)
-
Balazs Scheidler
-
Jeffrey W. Baker