On Wed, 2012-02-29 at 15:25 -0600, Ivanov, Konstantin wrote:
I have two servers running syslog-ng 2.0.9 on Dell R710's. I did some benchmarking with loggen [1] and was able to generate 41000 msg/sec locally. When I try to forward log messages from one server to the other I only get about 60 msg/sec. tcpdump shows that only that many are sent over the wire as well. I test this by writing to the log file from a script. Any idea what can be the problem? Here are both configs from the sending and receiving servers:
----------- Sending syslog-ng server --------------
options {
log_fifo_size(120000); time_reopen (1); time_reap(1); long_hostnames (off); use_dns (no); use_fqdn (no); create_dirs (yes); group(adm); perm(0640); dir_perm(0755); keep_hostname (yes); log_fetch_limit(60000);
};
source s_zxtm { file("/var/log/zxtm/access_log",follow_freq(1), flags(no-parse)); };
destination d_logrelay { tcp(somehost.com port(514)); };
log { source(s_zxtm); destination(d_logrelay); flags(flow-control); };
--------------- Receiving syslog-ng server ------------------
options {
log_fifo_size(3000000); time_reopen (1); time_reap(1); long_hostnames (off); use_dns (no); use_fqdn (no); create_dirs (yes); group(adm); perm(0640); dir_perm(0755); keep_hostname (yes); stats_freq(600); sync(0);
};
source s_zxtm { tcp(max-connections(500), port(514)); };
destination d_zxtm { file("/var/log/zxtm/incoming/$YEAR$MONTH$DAY $HOUR-access_log.aggregated" owner("glassfish") perm(0644) template("$MESSAGE\n") template_escape(no)); };
log { source(s_zxtm); destination(d_zxtm); flags(flow-control); };
--------------
[1]. loggen -iS -r 10000000 -s 200 -I 60 localhost 514 average rate = 41053.50 msg/sec, count=410535
Hm... I'm not really sure what the problem might be, 2.0.x is ancient by all means. It might be related to flush_timeout() though, try to set that to 0, or if at all possible try to upgrade to a newer version. Hmm.. now for a second idea, I think you are bitten by the fact, that you're checking the access log once every second (follow-freq(1)), and in every iteration, syslog-ng only pulls in 8k of data (log-msg-size() default), which is probably about 60 messages. Also, file position tracking wasn't implemented in 2.0.x, so it wasn't really good for tailing files like you do. This means that whenever you restart syslog-ng, the file will be resent again. -- Bazsi