[syslog-ng] lost messages with follow_freq()?

[Balazs Scheidler]

On Wed, 2008-08-20 at 15:14 -0400, Joe Shaw wrote:
> Hi again,
> 
> On Wed, Aug 20, 2008 at 12:39 PM, Balazs Scheidler <bazsi at balabit.hu> wrote:
> > The problem is probably related to the fifo size on the destination
> > file. file related statistics are not reported (as they'd clutter the
> > log statistics message).
> >
> > so please increase log_fifo_size() on your destination file.
> 
> You were right, I was dropping messages on the destination side and
> increasing the log_fifo_size() there stopped the "dropped messages"
> warnings.  However, it didn't solve the problem.  The file on the
> destination size is still considerably smaller, and it doesn't explain
> why making the follow_freq() window smaller (from 1s to 50ms) improves
> the results considerably.

Hmm.. are you sure that the given file destination is the only one where
message loss can occur?

what is your exact scenario? something like this:

source s_file { file("/var/log/file-to-be-tailed" follow-freq(1)); };
destination d_file { file("/var/log/destination-file"); };

log { source(s_file); destination(d_file); };

If this is similar to your exact scenario, then only the fifo of the
destination file can get full.

Decreasing the follow frequency will only increase your load and not
solve the problem. The source side cannot really drop messages, not even
in cases when more time elapses.

-- 
Bazsi