[syslog-ng] [Bug 175] New: syslog-ng doesnt perform fsync() with fsync option enabled on file destinations

[bugzilla at bugzilla.balabit.com]

https://bugzilla.balabit.com/show_bug.cgi?id=175

           Summary: syslog-ng doesnt perform fsync() with fsync option
                    enabled on file destinations
           Product: syslog-ng
           Version: 3.3.x
          Platform: PC
        OS/Version: Linux
            Status: NEW
          Severity: normal
          Priority: unspecified
         Component: syslog-ng
        AssignedTo: bazsi at balabit.hu
        ReportedBy: bugzilla.syslogng at feystorm.net
Type of the Report: ---
   Estimated Hours: 0.0


While trying to track down an issue on our production system where our box would experience hanging issues every 30 seconds, I discovered this was because
syslog-ng wasnt flushing its data, and so when the linux pdflush woke up, it would flush out a ton of data to NFS causing syslog and other applications to hang
until the flush completed. Ultimately this was causing messages to be dropped as input buffers were filling up while syslog-ng was hung.
This shouldnt have been happening as I have syslog-ng configured with the flush option enabled on the file destination which writes to NFS. I have a workaround
in place by forcing pdflush to run more frequently, and have also increased several of the input buffers so they dont fill up so fast.

I confirmed this issue by building syslog-ng-3.3.5 and putting a gdb break point on lib/logtransport.c:170, and the break point never triggers.


I do have a question about the fsync option as well. Does this option tell syslog-ng to flush() after every write, or after all lines have been written (like
from the flush_lines or flush_timeout() options)? If it flushes after every write, would it be possible to get an option to flush only once all the messages in
the batch have been written?



This is the stripped down configuration I have duplicated the issue with:
@version: 3.3
options { 
        keep_hostname(yes);
        chain_hostnames(no); 
        normalize_hostnames(yes);
        use_dns(no);
        use_fqdn(yes);
        stats_freq(0); 
        mark_freq(0);
        create_dirs(yes);
        dir_perm(0755);
        perm(0644);
};

source s_sys {
        unix-dgram("/dev/log" max-connections(256));
        internal();
        file("/proc/kmsg");
};

destination d_default { file("/var/log/$FACILITY" flush_lines(1) fsync(yes)); };
log { source(s_sys); destination(d_default); flags(fallback); };


-- 
Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching all bug changes.