I just happened to re-read my original post. Is it purely coincidental that no matter what the message rate was that I sent (5kmps in the first test vs 600mps in the second), the result was a log file of only around 8k messages? On Tue, Apr 13, 2010 at 10:30 PM, Clayton Dukes <cdukes@gmail.com> wrote:
Finally getting a chance to revisit this. I'm still seeing the problem.
If I run loggen like so: /www/svn/loggen -r 600 -D -I 30 127.0.0.1 514 average rate = 607.51 msg/sec, count=18226, time=30.012, msg size=256, bandwidth=151.88 kB/sec
I only get around 8k messages: wc -l /var/log/logzilla/syslog.log 8740 /var/log/logzilla/syslog.log
I've tried bumping up flush_lines and the fifo but neither seemed to make much of a difference.
Here's my config: options { long_hostnames(off); log_msg_size(8192); flush_lines(1); # Note: I've tried this up to 1000 log_fifo_size(35535); time_reopen(10); use_dns(yes); dns_cache(yes); use_fqdn(yes); keep_hostname(yes); chain_hostnames(no); };
destination df_logzilla { file("/var/log/logzilla/syslog.log" template("$HOST\t$FACILITY\t$LEVEL\t$TAG\t$YEAR-$MONTH-$DAY\t$HOUR:$MIN:$SEC\t$PROGRAM\t$MSG\n") ); };
log { source(s_all); destination(df_logzilla); }; On Thu, Apr 1, 2010 at 9:33 AM, Martin Holste <mcholste@gmail.com> wrote:
What do you get if you send the loggen data to a simple netcat session with its output redirected to a flat file? Do you see all 55k messages using wc -l?
On Thu, Apr 1, 2010 at 6:51 AM, Clayton Dukes <cdukes@gmail.com> wrote:
I should have mentioned that this is logging directly to a file.
destination df_logzilla { file("/var/log/logzilla/syslog.log"
template("$HOST\t$FACILITY\t$LEVEL\t$TAG\t$YEAR-$MONTH-$DAY\t$HOUR:$MIN:$SEC\t$PROGRAM\t$MSG\n") ); };
On Wed, Mar 31, 2010 at 11:47 PM, Clayton Dukes <cdukes@gmail.com> wrote:
Hi Folks, I'm trying to run a test to check insert rates. If I run this command:
./loggen -r 5000 -D -I 10 127.0.0.1 514
The output shows: average rate = 5441.60 msg/sec, count=54420, time=10.007, msg size=256, bandwidth=1360.40 kB/sec
But, my stats don't show that many messages received:
syslog-ng[6660]: Log statistics; dropped=\'pipe(/dev/xconsole)=0\', processed=\'center(queued)=24232\', processed=\'center(received)=8077, processed=\'destination(df_logzilla)=8077\'
As you can see, it sent 55k messages, but I only received 8k. Am I doing something wrong?
Here are my options in the syslog-ng config: options { long_hostnames(off); log_msg_size(8192); flush_lines(1); log_fifo_size(16384); time_reopen(10); use_dns(yes); dns_cache(yes); use_fqdn(yes); keep_hostname(yes); chain_hostnames(no); perm(0644); stats_freq(60);
};
-- ______________________________________________________________
Clayton Dukes ______________________________________________________________
-- ______________________________________________________________
Clayton Dukes ______________________________________________________________
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html
-- ______________________________________________________________
Clayton Dukes ______________________________________________________________
-- ______________________________________________________________ Clayton Dukes ______________________________________________________________