<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Hi,<br>
<br>
I think it's not a syslog-ng problem, the udp buffer of your kernel
will be full, and the kernel drops the udp packages (to make sure, you
can try to use netcat (netcat -lu -p 514 >> aaa.txt) instead of
syslog-ng, I think the logs will be missed in this case too).<br>
<br>
before running loggen, please check the value of the packet receive
errors:<br>
root@thor:/var/log# netstat -su<br>
Udp:<br>
124383 packets received<br>
3 packets to unknown port received.<br>
<b>82487 packet receive errors</b><br>
166196 packets sent<br>
RcvbufErrors: 82487<br>
<br>
then check it after running. I guess, you will see the missing packets
(just check the difference between before and after).<br>
<br>
so, if I am right, you just have to increase the size of the udp
receive buffer and it will work.<br>
For example:<br>
echo "88888888" > /proc/sys/net/core/rmem_default (or rmem_max)<br>
<br>
for more details about udp buffering:<br>
<a class="moz-txt-link-freetext" href="http://www.29west.com/docs/THPM/udp-buffer-sizing.html">http://www.29west.com/docs/THPM/udp-buffer-sizing.html</a><br>
<br>
<br>
Clayton Dukes wrote:
<blockquote
cite="mid:u2je54865561004131930w6a8f96d1oe54b6cc0f68b0bbe@mail.gmail.com"
type="cite">
<pre wrap="">Finally getting a chance to revisit this.
I'm still seeing the problem.
If I run loggen like so:
/www/svn/loggen -r 600 -D -I 30 127.0.0.1 514
average rate = 607.51 msg/sec, count=18226, time=30.012, msg size=256,
bandwidth=151.88 kB/sec
I only get around 8k messages:
wc -l /var/log/logzilla/syslog.log
8740 /var/log/logzilla/syslog.log
I've tried bumping up flush_lines and the fifo but neither seemed to
make much of a difference.
Here's my config:
options {
long_hostnames(off);
log_msg_size(8192);
flush_lines(1); # Note: I've tried this up to 1000
log_fifo_size(35535);
time_reopen(10);
use_dns(yes);
dns_cache(yes);
use_fqdn(yes);
keep_hostname(yes);
chain_hostnames(no);
};
destination df_logzilla {
file("/var/log/logzilla/syslog.log"
template("$HOST\t$FACILITY\t$LEVEL\t$TAG\t$YEAR-$MONTH-$DAY\t$HOUR:$MIN:$SEC\t$PROGRAM\t$MSG\n")
);
};
log {
source(s_all);
destination(df_logzilla);
};
On Thu, Apr 1, 2010 at 9:33 AM, Martin Holste <a class="moz-txt-link-rfc2396E" href="mailto:mcholste@gmail.com"><mcholste@gmail.com></a> wrote:
</pre>
<blockquote type="cite">
<pre wrap="">What do you get if you send the loggen data to a simple netcat session with
its output redirected to a flat file? Do you see all 55k messages using wc
-l?
On Thu, Apr 1, 2010 at 6:51 AM, Clayton Dukes <a class="moz-txt-link-rfc2396E" href="mailto:cdukes@gmail.com"><cdukes@gmail.com></a> wrote:
</pre>
<blockquote type="cite">
<pre wrap="">I should have mentioned that this is logging directly to a file.
destination df_logzilla {
file("/var/log/logzilla/syslog.log"
template("$HOST\t$FACILITY\t$LEVEL\t$TAG\t$YEAR-$MONTH-$DAY\t$HOUR:$MIN:$SEC\t$PROGRAM\t$MSG\n")
);
};
On Wed, Mar 31, 2010 at 11:47 PM, Clayton Dukes <a class="moz-txt-link-rfc2396E" href="mailto:cdukes@gmail.com"><cdukes@gmail.com></a> wrote:
</pre>
<blockquote type="cite">
<pre wrap="">Hi Folks,
I'm trying to run a test to check insert rates.
If I run this command:
./loggen -r 5000 -D -I 10 127.0.0.1 514
The output shows:
average rate = 5441.60 msg/sec, count=54420, time=10.007, msg size=256,
bandwidth=1360.40 kB/sec
But, my stats don't show that many messages received:
syslog-ng[6660]: Log statistics; dropped=\'pipe(/dev/xconsole)=0\',
processed=\'center(queued)=24232\', processed=\'center(received)=8077,
processed=\'destination(df_logzilla)=8077\'
As you can see, it sent 55k messages, but I only received 8k.
Am I doing something wrong?
Here are my options in the syslog-ng config:
options {
long_hostnames(off);
log_msg_size(8192);
flush_lines(1);
log_fifo_size(16384);
time_reopen(10);
use_dns(yes);
dns_cache(yes);
use_fqdn(yes);
keep_hostname(yes);
chain_hostnames(no);
perm(0644);
stats_freq(60);
};
--
______________________________________________________________
Clayton Dukes
______________________________________________________________
</pre>
</blockquote>
<pre wrap="">
--
______________________________________________________________
Clayton Dukes
______________________________________________________________
______________________________________________________________________________
Member info: <a class="moz-txt-link-freetext" href="https://lists.balabit.hu/mailman/listinfo/syslog-ng">https://lists.balabit.hu/mailman/listinfo/syslog-ng</a>
Documentation:
<a class="moz-txt-link-freetext" href="http://www.balabit.com/support/documentation/?product=syslog-ng">http://www.balabit.com/support/documentation/?product=syslog-ng</a>
FAQ: <a class="moz-txt-link-freetext" href="http://www.campin.net/syslog-ng/faq.html">http://www.campin.net/syslog-ng/faq.html</a>
</pre>
</blockquote>
<pre wrap="">
</pre>
</blockquote>
<pre wrap=""><!---->
</pre>
</blockquote>
<br>
<br>
<pre class="moz-signature" cols="72">--
pzolee
</pre>
</body>
</html>