On Wed, 2007-06-20 at 08:03 +0100, Geller, Sandor (IT) wrote:
[NEED] I want syslog-ng to write logs to local disk at pace of about 4000 lines per second without any lines losing. However, lines were in fact lost in the local file.
It would be good to see the statistics of syslog-ng. You should set the stats_freq() option as well, and analyse the output. I would like to recommend using stats_freq(60);
As you have omitted your log sources I don't know whether you are logging messages originating from the network. If you did, you should check the receive buffer options.
Also please note that using time_sleep(0) might cause performance drops, so you should try using time_sleep(10) or higher, the optimal setting depends on your environment...
First of all we need to know what your exact scenario is. You might be missing a receive buffer size tweak, or you might have something else. The information you provided is not enough.
[Consideration] "log_fifo_size" in global option is set as 10000. I tried to set sync() parameter, for instance 3000, in global option section. This did not succeed with this messages when reloading the process, " The value of flush_lines must be less than fifo_size; fifo_size='1000', flush_lines='3000' ".
Looks like line #502 of logwriter.c might be the cause of this. Seems that the global log_fifo_size isn't propagated correctly.
However you can override that by using the log_fifo_size() option in your destination definition too.
Right, the log_fifo_size() limit propagation has a problem, it maximizes the fifo size in 1000 entries, unless specified locally. This patch fixes it: diff --git a/src/logwriter.c b/src/logwriter.c index eea6814..955c333 100644 --- a/src/logwriter.c +++ b/src/logwriter.c @@ -499,7 +499,7 @@ log_writer_options_init(LogWriterOptions *options, GlobalConfig *cfg, guint32 fl options->template = template; options->flags = flags; if (options->fifo_size == -1) - options->fifo_size = MIN(1000, cfg->log_fifo_size); + options->fifo_size = MAX(1000, cfg->log_fifo_size); if (options->use_time_recvd == -1) options->use_time_recvd = cfg->use_time_recvd; -- Bazsi