[syslog-ng] syslog-ng 2.0.4 How can syslog-ng achieve thisperformance???

[Balazs Scheidler]

On Wed, 2007-06-20 at 08:03 +0100, Geller, Sandor (IT) wrote:
> > [NEED]
> > I want syslog-ng to write logs to local disk at pace of about 4000
> > lines per second without any lines losing.
> > However, lines were in fact lost in the local file.
> 
> It would be good to see the statistics of syslog-ng. You should
> set the stats_freq() option as well, and analyse the output. I
> would like to recommend using stats_freq(60);
> 
> As you have omitted your log sources I don't know whether you are
> logging messages originating from the network. If you did, you
> should check the receive buffer options.
> 
> Also please note that using time_sleep(0) might cause performance
> drops, so you should try using time_sleep(10) or higher, the optimal
> setting depends on your environment...
> 

First of all we need to know what your exact scenario is. You might be
missing a receive buffer size tweak, or you might have something else.
The information you provided is not enough.

> > [Consideration]
> > "log_fifo_size" in global option is set as 10000.
> > I tried to set sync() parameter, for instance 3000, in global 
> > option section.
> > This did not succeed with this messages when reloading the process, 
> > " The value of flush_lines must be less than fifo_size; 
> > fifo_size='1000',
> > flush_lines='3000' ".
> 
> Looks like line #502 of logwriter.c might be the cause of this.
> Seems that the global log_fifo_size isn't propagated correctly.
> 
> However you can override that by using the log_fifo_size() option
> in your destination definition too.

Right, the log_fifo_size() limit propagation has a problem, it maximizes
the fifo size in 1000 entries, unless specified locally. This patch
fixes it:

diff --git a/src/logwriter.c b/src/logwriter.c
index eea6814..955c333 100644
--- a/src/logwriter.c
+++ b/src/logwriter.c
@@ -499,7 +499,7 @@ log_writer_options_init(LogWriterOptions *options, GlobalConfig *cfg, guint32 fl
   options->template = template;
   options->flags = flags;
   if (options->fifo_size == -1)
-    options->fifo_size = MIN(1000, cfg->log_fifo_size);
+    options->fifo_size = MAX(1000, cfg->log_fifo_size);
   if (options->use_time_recvd == -1)
     options->use_time_recvd = cfg->use_time_recvd;
     


-- 
Bazsi