syslog-ng 2.0.4 How can syslog-ng achieve this performance???
Hi All, NEED HELP!!!!! [Environment] SunOS 5.9 Generic_122300-07 sun4u sparc SUNW syslog-ng 2.0.4 disk : single disk (no RAID) syslog-ng conf (global option part) sync (3000); time_reopen (10); time_sleep(0); log_fifo_size (10000); long_hostnames (off); use_dns (no); use_fqdn (no); create_dirs (no); keep_hostname (yes); [NEED] I want syslog-ng to write logs to local disk at pace of about 4000 lines per second without any lines losing. However, lines were in fact lost in the local file. I am trying to know the syslog-ng max performable point. [Consideration] "log_fifo_size" in global option is set as 10000. I tried to set sync() parameter, for instance 3000, in global option section. This did not succeed with this messages when reloading the process, " The value of flush_lines must be less than fifo_size; fifo_size='1000', flush_lines='3000' ". The administration guide says "sync" is alias for "flush_lines". In addition, resource usage at 4000lines/s load was as below, result of vmstat kthr memory page disk faults cpu r b w swap free re mf pi po fr de sr s3 sd sd -- in sy cs us sy id 0 0 0 3954792 3895288 155 8 353 0 0 0 12 0 2 0 0 669 4368 857 4 3 93 0 0 0 3955248 3926424 3 8 0 0 0 0 0 0 2 0 0 4112 27017 5889 21 15 64 0 0 0 3955248 3925664 0 0 0 0 0 0 0 0 3 0 0 4120 27251 5653 16 19 64 0 0 0 3955248 3924528 0 0 0 0 0 0 0 0 1 0 0 4129 27251 5914 15 18 67 0 0 0 3955248 3923400 0 0 0 0 0 0 0 0 1 0 0 4113 27236 6052 17 15 68 0 0 0 3955176 3922568 0 0 0 0 0 0 0 0 32 0 0 4156 26028 5405 19 13 68 0 0 0 3955176 3921808 0 0 0 0 0 0 0 0 1 0 0 4161 27316 5757 16 18 66 0 0 0 3955176 3921056 0 0 0 0 0 0 0 0 1 0 0 4120 27254 6136 14 18 68 0 0 0 3955176 3920296 0 0 0 0 0 0 0 0 1 0 0 4110 27244 5648 18 17 65 0 0 0 3955176 3919544 0 0 0 0 0 0 0 0 1 0 0 4115 27253 6042 17 19 64 0 0 0 3955176 3918784 0 0 0 0 0 0 0 0 1 0 0 4108 27238 6469 18 16 65 0 0 0 3955176 3918032 0 0 0 0 0 0 0 0 1 0 0 4107 27235 6106 16 18 66 0 0 0 3955176 3917272 0 0 0 0 0 0 0 0 1 0 0 4139 27264 5850 17 18 65 0 0 0 3955176 3916520 0 0 0 0 0 0 0 0 1 0 0 4107 27259 5867 19 14 67 result of iostat extended device statistics r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device 0.0 1.0 0.0 775.9 0.0 0.0 0.0 27.3 0 3 c1t0d0s3 1.0 2.0 8.0 744.1 0.0 0.0 0.0 11.4 0 3 c1t0d0s3 0.0 2.0 0.0 864.0 0.0 0.0 0.0 15.7 0 3 c1t0d0s3 0.0 4.0 0.0 856.0 0.0 0.1 0.0 17.3 0 3 c1t0d0s3 0.0 1.0 0.0 456.0 0.0 0.0 0.0 19.5 0 2 c1t0d0s3 0.0 1.0 0.0 856.1 0.0 0.0 0.0 28.1 0 3 c1t0d0s3 0.0 1.0 0.0 855.9 0.0 0.0 0.0 24.4 0 2 c1t0d0s3 0.0 1.0 0.0 856.1 0.0 0.0 0.0 27.3 0 3 c1t0d0s3 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c1t0d0s3 0.0 1.0 0.0 856.0 0.0 0.0 0.0 19.0 0 2 c1t0d0s3 0.0 1.0 0.0 856.0 0.0 0.0 0.0 26.9 0 3 c1t0d0s3 0.0 1.0 0.0 832.1 0.0 0.0 0.0 24.8 0 2 c1t0d0s3 0.0 1.0 0.0 735.9 0.0 0.0 0.0 26.4 0 3 c1t0d0s3 0.0 1.0 0.0 856.1 0.0 0.0 0.0 28.7 0 3 c1t0d0s3 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c1t0d0s3 [Question] Question 1. Does the "fifo_size" in above message mean "log_fifo_size" in global option ? Is the value "fifo_size='1000'" max value ? Is it possible to set "log_fifo_size" far more than 1000 ? Is it possible to set "sync" far more than 1000 ? If possible, then how do I do it ? Question 2. To achieve the NEED, I am considering the parameters below, sync() log_fifo_size() . Are there any other parameters I MUST consider for syslog-ng configuration? Regards George
Hi All,
NEED HELP!!!!!
Don't panic :)
[Environment] SunOS 5.9 Generic_122300-07 sun4u sparc SUNW syslog-ng 2.0.4 disk : single disk (no RAID) syslog-ng conf (global option part) sync (3000); time_reopen (10); time_sleep(0); log_fifo_size (10000); long_hostnames (off); use_dns (no); use_fqdn (no); create_dirs (no); keep_hostname (yes);
[NEED] I want syslog-ng to write logs to local disk at pace of about 4000 lines per second without any lines losing. However, lines were in fact lost in the local file.
It would be good to see the statistics of syslog-ng. You should set the stats_freq() option as well, and analyse the output. I would like to recommend using stats_freq(60); As you have omitted your log sources I don't know whether you are logging messages originating from the network. If you did, you should check the receive buffer options. Also please note that using time_sleep(0) might cause performance drops, so you should try using time_sleep(10) or higher, the optimal setting depends on your environment...
I am trying to know the syslog-ng max performable point.
Depends on the speed of the CPU, the disks, your syslog-ng filters, the ordering of the filters, ...
[Consideration] "log_fifo_size" in global option is set as 10000. I tried to set sync() parameter, for instance 3000, in global option section. This did not succeed with this messages when reloading the process, " The value of flush_lines must be less than fifo_size; fifo_size='1000', flush_lines='3000' ".
Looks like line #502 of logwriter.c might be the cause of this. Seems that the global log_fifo_size isn't propagated correctly. However you can override that by using the log_fifo_size() option in your destination definition too. Regards, Sandor -------------------------------------------------------- NOTICE: If received in error, please destroy and notify sender. Sender does not intend to waive confidentiality or privilege. Use of this email is prohibited when received in error.
On Wed, 2007-06-20 at 08:03 +0100, Geller, Sandor (IT) wrote:
[NEED] I want syslog-ng to write logs to local disk at pace of about 4000 lines per second without any lines losing. However, lines were in fact lost in the local file.
It would be good to see the statistics of syslog-ng. You should set the stats_freq() option as well, and analyse the output. I would like to recommend using stats_freq(60);
As you have omitted your log sources I don't know whether you are logging messages originating from the network. If you did, you should check the receive buffer options.
Also please note that using time_sleep(0) might cause performance drops, so you should try using time_sleep(10) or higher, the optimal setting depends on your environment...
First of all we need to know what your exact scenario is. You might be missing a receive buffer size tweak, or you might have something else. The information you provided is not enough.
[Consideration] "log_fifo_size" in global option is set as 10000. I tried to set sync() parameter, for instance 3000, in global option section. This did not succeed with this messages when reloading the process, " The value of flush_lines must be less than fifo_size; fifo_size='1000', flush_lines='3000' ".
Looks like line #502 of logwriter.c might be the cause of this. Seems that the global log_fifo_size isn't propagated correctly.
However you can override that by using the log_fifo_size() option in your destination definition too.
Right, the log_fifo_size() limit propagation has a problem, it maximizes the fifo size in 1000 entries, unless specified locally. This patch fixes it: diff --git a/src/logwriter.c b/src/logwriter.c index eea6814..955c333 100644 --- a/src/logwriter.c +++ b/src/logwriter.c @@ -499,7 +499,7 @@ log_writer_options_init(LogWriterOptions *options, GlobalConfig *cfg, guint32 fl options->template = template; options->flags = flags; if (options->fifo_size == -1) - options->fifo_size = MIN(1000, cfg->log_fifo_size); + options->fifo_size = MAX(1000, cfg->log_fifo_size); if (options->use_time_recvd == -1) options->use_time_recvd = cfg->use_time_recvd; -- Bazsi
diff --git a/src/logwriter.c b/src/logwriter.c index eea6814..955c333 100644 --- a/src/logwriter.c +++ b/src/logwriter.c @@ -499,7 +499,7 @@ log_writer_options_init(LogWriterOptions *options, GlobalConfig *cfg, guint32 fl options->template = template; options->flags = flags; if (options->fifo_size == -1) - options->fifo_size = MIN(1000, cfg->log_fifo_size); + options->fifo_size = MAX(1000, cfg->log_fifo_size); if (options->use_time_recvd == -1) options->use_time_recvd = cfg->use_time_recvd;
With this patch there will be a problem when someone tries to use log_fifo_size lower than 1000 :) Why not using the same value as cfg->log_fifo_size? It defaults to 1000, so you'll set up a sane value if there wasn't log_fifo_size specified in the config file. Another solution would be to sanitise the value later, and send a warning message when it needs to be fixed. Regards, Sandor -------------------------------------------------------- NOTICE: If received in error, please destroy and notify sender. Sender does not intend to waive confidentiality or privilege. Use of this email is prohibited when received in error.
On Wed, 2007-06-20 at 10:44 +0100, Geller, Sandor (IT) wrote:
diff --git a/src/logwriter.c b/src/logwriter.c index eea6814..955c333 100644 --- a/src/logwriter.c +++ b/src/logwriter.c @@ -499,7 +499,7 @@ log_writer_options_init(LogWriterOptions *options, GlobalConfig *cfg, guint32 fl options->template = template; options->flags = flags; if (options->fifo_size == -1) - options->fifo_size = MIN(1000, cfg->log_fifo_size); + options->fifo_size = MAX(1000, cfg->log_fifo_size); if (options->use_time_recvd == -1) options->use_time_recvd = cfg->use_time_recvd;
With this patch there will be a problem when someone tries to use log_fifo_size lower than 1000 :) Why not using the same value as cfg->log_fifo_size? It defaults to 1000, so you'll set up a sane value if there wasn't log_fifo_size specified in the config file. Another solution would be to sanitise the value later, and send a warning message when it needs to be fixed.
That was my intention. I don't want to allow anything lower than 1000, with the latest multi-fetch patches (in 1.6.11 and 2.0.x), anything lower than 1000 may be dropping messages even when the destination is a file. -- Bazsi
participants (3)
-
Balazs Scheidler
-
Geller, Sandor (IT)
-
Root Administrator