I have a FreeBSD 5.4 system running syslog-ng 2.0.5 - there's no load on it. If I run the following perl code: ** Begin Code ** use Sys::Syslog qw(:DEFAULT setlogsock); # start some logging setlogsock 'unix'; openlog('task', 'pid', 'daemon'); syslog 'info', 'starting'; $txt = 'blah'; for (1..8) { syslog 'debug', $_. ' len('. length($txt) . ') '. $txt; $txt = "$txt $txt"; } for (1..100) { syslog 'debug', $_. ' len('. length($txt) . ') '. $txt; $txt = "$txt bleargh"; } ** End Code ** I seem to be dropping a random message or two during the second loop. I've tried tuning the syslog-ng conf file several different ways without luck. The relevant pieces of the current iteration of the conf file are: options { long_hostnames(off); flush_lines(1); time_reopen(10); time_reap(600); log_fifo_size(4096); }; source src { unix-dgram("/var/run/log" log_msg_size(65536) log_fetch_limit(5)); unix-dgram("/var/run/logpriv" perm(0600) log_msg_size(65536) log_fetch_limit(5)); udp(log_msg_size(65536) log_fetch_limit(5)); internal(); file("/dev/klog" log_msg_size(65536) log_fetch_limit(5)); }; destination task_log { file( "/var/log/task.log" perm(0664) ); }; filter f_task{ program("task"); }; log {source(src); filter(f_task); destination(task_log); flags(flow-control);}; Anybody have any ideas why I'm dropping random lines? Thanks in advance... -- Cole Tuininga http://www.tuininga.org/
On Nov 21, 2007 12:51 PM, Cole Tuininga <cole.tuininga@gmail.com> wrote:
I seem to be dropping a random message or two during the second loop. I've tried tuning the syslog-ng conf file several different ways without luck.
The unix-dgram driver can (will) be lossy if messages are being written faster than they are being read out by syslog-ng, setting log_fetch_limit(5) along with flags(flow-control) makes random dropping of messages at the input socket much more likely. You might consider increasing flush_lines and log_fetch_limit until the loss stops.
Anybody have any ideas why I'm dropping random lines? Thanks in advance...
Does FreeBSD have a kernel tunable for unix datagram buffers? Another interesting test would be to use Time::HiRes qw( usleep ) sand stick a usleep(1) into the loop, maybe just every fifth time around. Kevin
On Nov 21, 2007 3:23 PM, K K <kkadow@gmail.com> wrote:
The unix-dgram driver can (will) be lossy if messages are being written faster than they are being read out by syslog-ng,
This makes sense.
You might consider increasing flush_lines and log_fetch_limit until the loss stops.
Interestingly, this seemed to *increase* the frequency of the drops.
Does FreeBSD have a kernel tunable for unix datagram buffers?
Indeed it does, and this helped greatly. I changed it from 4K to 64K and that fixed the problem consistently with my little test script. Thanks for the idea. -- Cole Tuininga http://www.tuininga.org/
participants (2)
-
Cole Tuininga
-
K K