[syslog-ng] Flow-control behaviour and Cached lines timestamping, plus a suspend error in 2.1beta1

[Behal, Pavel]

Dear Bazsi,

I have found two "features" and one bug I would like to discuss a little:

1) We use the syslog-ng 2.1alpha1 as a root system syslog. The problem is, we have had two destinations setup. First destination was the traditional write of system messages to the file like "/var/log/messages" etc. The second destination path was to the TCP stream to the central syslog server. And the problem is, we have used the flow-control flag on the central TCP path. But, when the central TCP syslog-ng becomes unavailable, after all the buffers filled up, the local system syslog-ng stopped processing any system messages. It did not write anything through the local file destinations until the TCP communication was re-established.
Is this a correct behaviour? We have found, that to allow smooth processing of local system sources, we can not use flow-control flag on any destination paths it used..


2) When messages are in syslog-ng buffer, they do not contain any timestamp. We have realized, that the timestamp is added at the time the syslog-ng flushes all the buffers to the disk. This is not very good in case you need to debug the messages ages and all of them have same and non-related timestamp.
Is it possible to modify the logic of syslog-ng to timestamp the messages as soon as possible - even before the buffer, at the collection time? We have found it important for log integrity reasons.


3) Directly in the 2.1beta1 there has been introduced some error with the "suspend writing a destination file when an I/O error ..." patch. During our tests we have found it continously thinks the destination is unavailable and writes strange messages into to the internal log:

May 29 15:00:02 europa syslog-ng[16798]: I/O error occurred while writing; fd='7', error='Connection timed out (110)'
May 29 15:00:02 europa syslog-ng[16798]: Connection broken; time_reopen='60'
May 29 15:00:02 europa syslog-ng[16798]: Suspending write operation because of an I/O error; fd='7', time_reopen='60'

the lsof -p 16798:

COMMAND     PID USER   FD   TYPE     DEVICE    SIZE       NODE NAME
syslog-ng 16798 root  cwd    DIR      253,0    4096          2 /
syslog-ng 16798 root  rtd    DIR      253,0    4096          2 /
syslog-ng 16798 root  txt    REG      253,0  114584     671841 /sbin/syslog-ng
syslog-ng 16798 root  mem    REG      253,0  106397     820372 /lib/ld-2.3.4.so
syslog-ng 16798 root  mem    REG      253,0 1454546     820373 /lib/tls/libc-2.3.4.so
syslog-ng 16798 root  mem    REG      253,0   28504    2919694 /usr/lib/libwrap.so.0.7.6
syslog-ng 16798 root  mem    REG      253,0   93985     820376 /lib/tls/libpthread-2.3.4.so
syslog-ng 16798 root  mem    REG      253,0   47671     820386 /lib/tls/librt-2.3.4.so
syslog-ng 16798 root  mem    REG      253,0  505200    2930064 /usr/lib/libglib-2.0.so.0.400.7
syslog-ng 16798 root  mem    REG      253,0   95148     820384 /lib/libnsl-2.3.4.so
syslog-ng 16798 root  mem    REG      253,0   36639    2920956 /usr/local/lib/libevtlog.so.0.0.0
syslog-ng 16798 root    0r   CHR        1,3               1619 /dev/null
syslog-ng 16798 root    1w   CHR        1,3               1619 /dev/null
syslog-ng 16798 root    2w   CHR        1,3               1619 /dev/null
syslog-ng 16798 root    3r   REG        0,3       0 4026531850 /proc/kmsg
syslog-ng 16798 root    4u  unix 0xc1be7ba0             351802 /dev/log
syslog-ng 16798 root    5u  IPv4     351804                TCP europa.training.hp.com:37205->europa.training.hp.com:5140 (ESTABLISHED)
syslog-ng 16798 root    6w   REG      253,0    4802    1655155 /var/log/messages
syslog-ng 16798 root    8w   REG      253,0     525    1655159 /var/log/boot.log

This happens during standard system syslog processing on RH ELinux 4 and Solaris 9. You have the syslog-ng confs attached. Could you pleae check it?

Best Regards,

Pavel
-------------- next part --------------
A non-text attachment was scrubbed...
Name: syslog-ng.conf
Type: application/octet-stream
Size: 2385 bytes
Desc: syslog-ng.conf
Url : http://lists.balabit.hu/pipermail/syslog-ng/attachments/20080529/61e7f65b/attachment.obj 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: syslog-ng-app.conf
Type: application/octet-stream
Size: 794 bytes
Desc: syslog-ng-app.conf
Url : http://lists.balabit.hu/pipermail/syslog-ng/attachments/20080529/61e7f65b/attachment-0001.obj