Hello,
I'm testing syslog-ng with TCP transport, and I'm a little surprised- perhaps I'm doing something wrong? I'm finding severe message loss, which I had imagined TCP would eliminate.
I've reported the same thing on this list in the past. Indeed, my machines can even lose messages when the input is a pipe and the output is a local file!
And I replied that you are _not_ correct. Several times. Please stop spreading misinformation.
The culprit is this bit of code at line 71 in libol:
if (self->queue_size == self->queue_max) { /* fifo full */ ol_string_free(string); return ST_FAIL | ST_OK; }
If the queue is full, your message is dismissed and cannot be recovered. If you put a printf() statement in that code block, I bet you will see that you are hitting this misfeature. I tried fixing this problem, but there is not a simple way. It requires significant changes to syslog-ng's message handling loop.
You have to raise the log_fifo_size variable. Try to estimate the number of possible received messages in a single poll run. Each message source receives maximum 1024 bytes of data. A single log message is about 20 to 30 bytes. So on a single run each log source may emit 1024/20 = 50 messages. Your log fifo must be able to hold this number of messages for each source. So for two sources, you'll need at least 100 slots in your fifo: destination d_tcp { tcp(x.x.x.x log_fifo_size(100)); }; -- Bazsi PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1 url: http://www.balabit.hu/pgpkey.txt