[syslog-ng] [Bug 258] syslog-ng 3.4.2 stalls on Ubuntu 12.04 Precise with /var/ log full

[bugzilla at bugzilla.balabit.com]

https://bugzilla.balabit.com/show_bug.cgi?id=258


Carl Chenet <carl.chenet-ext at cloudwatt.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |carl.chenet-
                   |                            |ext at cloudwatt.com




--- Comment #2 from Carl Chenet <carl.chenet-ext at cloudwatt.com>  2013-12-19 15:42:39 ---
Hi,

I tried to implement the flow control but I still some hosts which stall when the file system is full. I have the following setup on my syslog-ng clients (only
relevant parts):

log_msg_size(16384);
log_fifo_size(60000);

source s_local {
        system();
        internal();
        syslog(ip("127.0.0.1") transport(tcp) port(514) host_override("host1") log_iw_size(30000) max-connections(300));
        syslog(ip("127.0.0.1") transport(udp) port(514) host_override("host1") log_iw_size(30000) max-connections(300));
};

log {
    source(s_local);
    destination(d_system);
    flags(flow-control);
};

But I always have some host stalling when /var/log is full. It seems to happen at the exact time when the logrotate of the system reloads syslog-ng

my logrotate conf is :
/var/log/auth.log
/var/log/cron.log
/var/log/mail.log
/var/log/kernel.log
/var/log/system.log
{
  rotate 7
  daily
  missingok
  compress
  delaycompress
  sharedscripts
  postrotate
    service syslog-ng reload > /dev/null
  endscript
}

The log follows:

Dec 14 06:15:56 p-wkrcbx-0004.adm.prd1.val.cloudwatt.net chef-client[4824]: INFO: ruby_block[generate-mcollective-classes-delayed] sending run action to
ruby_block[generate-mcollective-classes] (delayed)
Dec 14 06:15:56 p-wkrcbx-0004.adm.prd1.val.cloudwatt.net chef-client[4824]: INFO: ruby_block[generate-mcollective-classes] called
Dec 14 06:15:56 p-wkrcbx-0004.adm.prd1.val.cloudwatt.net chef-client[4824]: INFO: Chef Run complete in 10.500146744 seconds
Dec 14 06:15:56 p-wkrcbx-0004.adm.prd1.val.cloudwatt.net chef-client[4824]: INFO: Running report handlers
Dec 14 06:15:56 p-wkrcbx-0004.adm.prd1.val.cloudwatt.net chef-client[4824]: INFO: Report handlers complete
Dec 14 06:25:15 p-wkrcbx-0004.adm.prd1.val.cloudwatt.net syslog-ng[26524]: I/O error occurred while writing; fd='47', error='No space left on device (28)'
Dec 14 06:25:15 p-wkrcbx-0004.adm.prd1.val.cloudwatt.net syslog-ng[26524]: Suspending write operation because of an I/O error; fd='47', time_reopen='60'
Dec 14 06:26:15 p-wkrcbx-0004.adm.prd1.val.cloudwatt.net syslog-ng[26524]: Error suspend timeout has elapsed, attempting to write again; fd='47'
Dec 14 06:26:15 p-wkrcbx-0004.adm.prd1.val.cloudwatt.net syslog-ng[26524]: I/O error occurred while writing; fd='47', error='No space left on device (28)'

And 06:25:15 is the time of the daily logrotate:

# m h dom mon dow user  command
17 *    * * *   root    cd / && run-parts --report /etc/cron.hourly
25 6    * * *   root    test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily )

Don't hesitate to contact me for more information about this issue.

Regards,
Carl Chenet


-- 
Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching all bug changes.