[syslog-ng] Lot of log lost

[maxime.denier at orange-ftgroup.com]

Hello,
 
I have recently installed syslog-ng OSE 3.1 as log collector and I face
a problem.
A great number of logs arrive on the server, but a little part of them
arrive in the destination files, but all the destination files have some
logs on it.
I have enabled the verbose mode and I see this:
Jul 20 07:52:04 sparte1 syslog-ng[2557]: Initializing destination file
writer;
template='/var/logs/${NSM.DEVICE:-Unknown_device}/${NSM.RECEIVED_TIME.YE
AR}${NSM.RECEIVED_TIME.MONTH}${NSM.RECEIVED_TIME.DAY}2400.csv',
filename='/var/logs/zidane2/201107202400.csv'
Jul 20 07:52:31 sparte1 syslog-ng[2557]: Reaping unused destination
files;
template='/var/logs/${NSM.DEVICE:-Unknown_device}/${NSM.RECEIVED_TIME.YE
AR}${NSM.RECEIVED_TIME.MONTH}${NSM.RECEIVED_TIME.DAY}2400.csv'
Jul 20 07:53:01 sparte1 syslog-ng[2557]: Reaping unused destination
files;
template='/var/logs/${NSM.DEVICE:-Unknown_device}/${NSM.RECEIVED_TIME.YE
AR}${NSM.RECEIVED_TIME.MONTH}${NSM.RECEIVED_TIME.DAY}2400.csv'
Jul 20 07:53:01 sparte1 syslog-ng[2557]: Destination timed out, reaping;
template='/var/logs/${NSM.DEVICE:-Unknown_device}/${NSM.RECEIVED_TIME.YE
AR}${NSM.RECEIVED_TIME.MONTH}${NSM.RECEIVED_TIME.DAY}2400.csv',
filename='/var/logs/peony2/201107202400.csv'
Jul 20 07:53:01 sparte1 syslog-ng[2557]: Closing log transport fd;
fd='31'
Jul 20 07:53:01 sparte1 syslog-ng[2557]: Destination timed out, reaping;
template='/var/logs/${NSM.DEVICE:-Unknown_device}/${NSM.RECEIVED_TIME.YE
AR}${NSM.RECEIVED_TIME.MONTH}${NSM.RECEIVED_TIME.DAY}2400.csv',
filename='/var/logs/decca2/201107202400.csv'
Jul 20 07:53:01 sparte1 syslog-ng[2557]: Closing log transport fd;
fd='19'
Jul 20 07:53:16 sparte1 syslog-ng[2557]: Initializing destination file
writer;
template='/var/logs/${NSM.DEVICE:-Unknown_device}/${NSM.RECEIVED_TIME.YE
AR}${NSM.RECEIVED_TIME.MONTH}${NSM.RECEIVED_TIME.DAY}2400.csv',
filename='/var/logs/hyenne2/201107202400.csv'
Jul 20 07:53:17 sparte1 syslog-ng[2557]: Initializing destination file
writer;
template='/var/logs/${NSM.DEVICE:-Unknown_device}/${NSM.RECEIVED_TIME.YE
AR}${NSM.RECEIVED_TIME.MONTH}${NSM.RECEIVED_TIME.DAY}2400.csv',
filename='/var/logs/olive2/201107202400.csv'
Jul 20 07:53:31 sparte1 syslog-ng[2557]: Reaping unused destination
files;
template='/var/logs/${NSM.DEVICE:-Unknown_device}/${NSM.RECEIVED_TIME.YE
AR}${NSM.RECEIVED_TIME.MONTH}${NSM.RECEIVED_TIME.DAY}2400.csv'
Jul 20 07:53:31 sparte1 syslog-ng[2557]: Destination timed out, reaping;
template='/var/logs/${NSM.DEVICE:-Unknown_device}/${NSM.RECEIVED_TIME.YE
AR}${NSM.RECEIVED_TIME.MONTH}${NSM.RECEIVED_TIME.DAY}2400.csv',
filename='/var/logs/zidane2/201107202400.csv'
Jul 20 07:53:31 sparte1 syslog-ng[2557]: Closing log transport fd;
fd='24'
 
I haven't found information about the root cause of these timed out.
This seems to be a writing problem.
Before using syslog-ng, Logs were processed by a application owned by
the firewall publisher on the same type of hardware without this great
number of log lost.
 
If any body have already faced this problem and have a solution, it
would be wonderful.

Regards,
 
Maxime Denier

********************************************************************************
IMPORTANT.Les informations contenues dans ce message electronique y compris les fichiers attaches sont strictement confidentielles
et peuvent etre protegees par la loi.
Ce message electronique est destine exclusivement au(x) destinataire(s) mentionne(s) ci-dessus.
Si vous avez recu ce message par erreur ou s il ne vous est pas destine, veuillez immediatement le signaler  a l expediteur et effacer ce message 
et tous les fichiers eventuellement attaches.
Toute lecture, exploitation ou transmission des informations contenues dans ce message est interdite.
Tout message electronique est susceptible d alteration.
A ce titre, le Groupe France Telecom decline toute responsabilite notamment s il a ete altere, deforme ou falsifie.
De meme, il appartient au destinataire de s assurer de l absence de tout virus.

IMPORTANT.This e-mail message and any attachments are strictly confidential and may be protected by law. This message is
intended only for the named recipient(s) above.
If you have received this message in error, or are not the named recipient(s), please immediately notify the sender and delete this e-mail message.
Any unauthorized view, usage or disclosure ofthis message is prohibited.
Since e-mail messages may not be reliable, France Telecom Group shall not be liable for any message if modified, changed or falsified.
Additionally the recipient should ensure they are actually virus free.
********************************************************************************

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.balabit.hu/pipermail/syslog-ng/attachments/20110720/e4c61bfd/attachment-0001.htm