On Wed, 2011-07-20 at 11:14 +0200, maxime.denier@orange-ftgroup.com wrote:
Hello,
I have recently installed syslog-ng OSE 3.1 as log collector and I face a problem. A great number of logs arrive on the server, but a little part of them arrive in the destination files, but all the destination files have some logs on it. I have enabled the verbose mode and I see this: Jul 20 07:52:04 sparte1 syslog-ng[2557]: Initializing destination file writer; template='/var/logs/${NSM.DEVICE:-Unknown_device}/${NSM.RECEIVED_TIME.YEAR}${NSM.RECEIVED_TIME.MONTH}${NSM.RECEIVED_TIME.DAY}2400.csv', filename='/var/logs/zidane2/201107202400.csv' Jul 20 07:52:31 sparte1 syslog-ng[2557]: Reaping unused destination files; template='/var/logs/${NSM.DEVICE:-Unknown_device}/${NSM.RECEIVED_TIME.YEAR}${NSM.RECEIVED_TIME.MONTH}${NSM.RECEIVED_TIME.DAY}2400.csv' Jul 20 07:53:01 sparte1 syslog-ng[2557]: Reaping unused destination files; template='/var/logs/${NSM.DEVICE:-Unknown_device}/${NSM.RECEIVED_TIME.YEAR}${NSM.RECEIVED_TIME.MONTH}${NSM.RECEIVED_TIME.DAY}2400.csv' Jul 20 07:53:01 sparte1 syslog-ng[2557]: Destination timed out, reaping; template='/var/logs/${NSM.DEVICE:-Unknown_device}/${NSM.RECEIVED_TIME.YEAR}${NSM.RECEIVED_TIME.MONTH}${NSM.RECEIVED_TIME.DAY}2400.csv', filename='/var/logs/peony2/201107202400.csv' Jul 20 07:53:01 sparte1 syslog-ng[2557]: Closing log transport fd; fd='31' Jul 20 07:53:01 sparte1 syslog-ng[2557]: Destination timed out, reaping; template='/var/logs/${NSM.DEVICE:-Unknown_device}/${NSM.RECEIVED_TIME.YEAR}${NSM.RECEIVED_TIME.MONTH}${NSM.RECEIVED_TIME.DAY}2400.csv', filename='/var/logs/decca2/201107202400.csv' Jul 20 07:53:01 sparte1 syslog-ng[2557]: Closing log transport fd; fd='19' Jul 20 07:53:16 sparte1 syslog-ng[2557]: Initializing destination file writer; template='/var/logs/${NSM.DEVICE:-Unknown_device}/${NSM.RECEIVED_TIME.YEAR}${NSM.RECEIVED_TIME.MONTH}${NSM.RECEIVED_TIME.DAY}2400.csv', filename='/var/logs/hyenne2/201107202400.csv' Jul 20 07:53:17 sparte1 syslog-ng[2557]: Initializing destination file writer; template='/var/logs/${NSM.DEVICE:-Unknown_device}/${NSM.RECEIVED_TIME.YEAR}${NSM.RECEIVED_TIME.MONTH}${NSM.RECEIVED_TIME.DAY}2400.csv', filename='/var/logs/olive2/201107202400.csv' Jul 20 07:53:31 sparte1 syslog-ng[2557]: Reaping unused destination files; template='/var/logs/${NSM.DEVICE:-Unknown_device}/${NSM.RECEIVED_TIME.YEAR}${NSM.RECEIVED_TIME.MONTH}${NSM.RECEIVED_TIME.DAY}2400.csv' Jul 20 07:53:31 sparte1 syslog-ng[2557]: Destination timed out, reaping; template='/var/logs/${NSM.DEVICE:-Unknown_device}/${NSM.RECEIVED_TIME.YEAR}${NSM.RECEIVED_TIME.MONTH}${NSM.RECEIVED_TIME.DAY}2400.csv', filename='/var/logs/zidane2/201107202400.csv' Jul 20 07:53:31 sparte1 syslog-ng[2557]: Closing log transport fd; fd='24'
I haven't found information about the root cause of these timed out.
These only indicate that syslog-ng is properly garbage-collecting destination files that receive no data. This is not an error, that's why you only get this if you enable --debug / --verbose (i'm not sure which one).
This seems to be a writing problem. Before using syslog-ng, Logs were processed by a application owned by the firewall publisher on the same type of hardware without this great number of log lost.
Is this UDP? syslog-ng doesn't increase udp receive buffer sizes unless explicitly told so, using the so-rcvbuf() option on the udp source. you probably need to increase that. But I'd recommend not using udp, as that can cause a lot of lost messages easily. (if you create a simple loop that sends udp frames to the syslog receiver, you can easily see as much as 97% messages lost!) easy DoS.
If any body have already faced this problem and have a solution, it would be wonderful.
There was a guy on this list, who has published charts and numbers how he had to tune the various buffering options. I forgot his name, but if you google syslog-ng, udp buffer size, you'll probably find it. -- Bazsi