Excellent link, thanks!
That does seem to be the problem, however, if I set the buffer all the
way up to 1G using:
sysctl -w net.core.rmem_max=1073741824


Then I'm still dropping messages when using a test rate of 6kmps:

# ./loggen -r 6000 -D -I 10 127.0.0.1 514
average rate = 6526.63 msg/sec, count=65272, time=10.008, msg
size=256, bandwidth=1631.66 kB/sec

# wc -l /tmp/logs
62933 /tmp/logs

Is there a recommendation on what the buffer should be set to for high
insertion rates?
My test server has 8G of memory, but I can give it more (up to 24G).

Also, note that this is a VMWare ESXi server - might that have
something to do with it?


  

On Wed, Apr 14, 2010 at 6:16 AM, Zoltán Pallagi <pzolee@balabit.hu> wrote:
  

Hi,

I think it's not a syslog-ng problem, the udp buffer of your kernel will be
full, and the kernel drops the udp packages (to make sure, you can try to
use netcat (netcat -lu -p 514 >> aaa.txt) instead of syslog-ng, I think the
logs will be missed in this case too).

before running loggen, please check the value of the packet receive errors:
root@thor:/var/log# netstat -su
Udp:
    124383 packets received
    3 packets to unknown port received.
    82487 packet receive errors
    166196 packets sent
    RcvbufErrors: 82487

then check it after running. I guess, you will see the missing packets (just
check the difference between before and after).

so, if I am right, you just have to increase the size of the udp receive
buffer and it will work.
For example:
echo "88888888" > /proc/sys/net/core/rmem_default (or rmem_max)

for more details about udp buffering:
http://www.29west.com/docs/THPM/udp-buffer-sizing.html


Clayton Dukes wrote:

Finally getting a chance to revisit this.
I'm still seeing the problem.

If I run loggen like so:
/www/svn/loggen -r 600 -D -I 30 127.0.0.1 514
average rate = 607.51 msg/sec, count=18226, time=30.012, msg size=256,
bandwidth=151.88 kB/sec

I only get around 8k messages:
wc -l /var/log/logzilla/syslog.log
8740 /var/log/logzilla/syslog.log


I've tried bumping up flush_lines and the fifo but neither seemed to
make much of a difference.

Here's my config:
options {
      long_hostnames(off);
      log_msg_size(8192);
      flush_lines(1); # Note: I've tried this up to 1000
      log_fifo_size(35535);
      time_reopen(10);
      use_dns(yes);
      dns_cache(yes);
      use_fqdn(yes);
      keep_hostname(yes);
      chain_hostnames(no);
};

destination df_logzilla {
   file("/var/log/logzilla/syslog.log"

template("$HOST\t$FACILITY\t$LEVEL\t$TAG\t$YEAR-$MONTH-$DAY\t$HOUR:$MIN:$SEC\t$PROGRAM\t$MSG\n")
   );
};

log {
   source(s_all);
      destination(df_logzilla);
};
On Thu, Apr 1, 2010 at 9:33 AM, Martin Holste <mcholste@gmail.com> wrote:


What do you get if you send the loggen data to a simple netcat session with
its output redirected to a flat file?  Do you see all 55k messages using wc
-l?

On Thu, Apr 1, 2010 at 6:51 AM, Clayton Dukes <cdukes@gmail.com> wrote:


I should have mentioned that this is logging directly to a file.

destination df_logzilla {
   file("/var/log/logzilla/syslog.log"

template("$HOST\t$FACILITY\t$LEVEL\t$TAG\t$YEAR-$MONTH-$DAY\t$HOUR:$MIN:$SEC\t$PROGRAM\t$MSG\n")
   );
};


On Wed, Mar 31, 2010 at 11:47 PM, Clayton Dukes <cdukes@gmail.com> wrote:


Hi Folks,
I'm trying to run a test to check insert rates.
If I run this command:

./loggen -r 5000 -D -I 10 127.0.0.1 514

The output shows:
average rate = 5441.60 msg/sec, count=54420, time=10.007, msg size=256,
bandwidth=1360.40 kB/sec

But, my stats don't show that many messages received:

syslog-ng[6660]: Log statistics; dropped=\'pipe(/dev/xconsole)=0\',
processed=\'center(queued)=24232\', processed=\'center(received)=8077,
processed=\'destination(df_logzilla)=8077\'

As you can see, it sent 55k messages, but I only received 8k.
Am I doing something wrong?

Here are my options in the syslog-ng config:
options {
      long_hostnames(off);
      log_msg_size(8192);
      flush_lines(1);
      log_fifo_size(16384);
      time_reopen(10);
      use_dns(yes);
      dns_cache(yes);
      use_fqdn(yes);
      keep_hostname(yes);
      chain_hostnames(no);
      perm(0644);
     stats_freq(60);

};


--
______________________________________________________________

Clayton Dukes
______________________________________________________________


--
______________________________________________________________

Clayton Dukes
______________________________________________________________


______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation:
http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.campin.net/syslog-ng/faq.html








--
pzolee

    

-- 
pzolee