[syslog-ng] Tests using loggen - not receiving all the packets

Clayton Dukes cdukes at gmail.com
Wed Apr 14 06:16:22 CEST 2010


Martin,
To answer your question regarding netcat:
It's better, but still not receiving all the data.

# netcat -u -p 514 -l > /tmp/logs

#loggen -r 600 -D -I 30 127.0.0.1 514
average rate = 607.28 msg/sec, count=18219, time=30.010, msg size=256,
bandwidth=151.82 kB/sec

# wc -l /tmp/logs
13134 /tmp/logs

What am I doing wrong?



On Tue, Apr 13, 2010 at 10:41 PM, Clayton Dukes <cdukes at gmail.com> wrote:
> I just happened to re-read my original post. Is it purely coincidental
> that no matter what the message rate was that I sent (5kmps in the
> first test vs 600mps in the second), the result was a log file of only
> around 8k messages?
>
>
>
> On Tue, Apr 13, 2010 at 10:30 PM, Clayton Dukes <cdukes at gmail.com> wrote:
>> Finally getting a chance to revisit this.
>> I'm still seeing the problem.
>>
>> If I run loggen like so:
>> /www/svn/loggen -r 600 -D -I 30 127.0.0.1 514
>> average rate = 607.51 msg/sec, count=18226, time=30.012, msg size=256,
>> bandwidth=151.88 kB/sec
>>
>> I only get around 8k messages:
>> wc -l /var/log/logzilla/syslog.log
>> 8740 /var/log/logzilla/syslog.log
>>
>>
>> I've tried bumping up flush_lines and the fifo but neither seemed to
>> make much of a difference.
>>
>> Here's my config:
>> options {
>>      long_hostnames(off);
>>      log_msg_size(8192);
>>      flush_lines(1); # Note: I've tried this up to 1000
>>      log_fifo_size(35535);
>>      time_reopen(10);
>>      use_dns(yes);
>>      dns_cache(yes);
>>      use_fqdn(yes);
>>      keep_hostname(yes);
>>      chain_hostnames(no);
>> };
>>
>> destination df_logzilla {
>>   file("/var/log/logzilla/syslog.log"
>>   template("$HOST\t$FACILITY\t$LEVEL\t$TAG\t$YEAR-$MONTH-$DAY\t$HOUR:$MIN:$SEC\t$PROGRAM\t$MSG\n")
>>   );
>> };
>>
>> log {
>>   source(s_all);
>>      destination(df_logzilla);
>> };
>> On Thu, Apr 1, 2010 at 9:33 AM, Martin Holste <mcholste at gmail.com> wrote:
>>> What do you get if you send the loggen data to a simple netcat session with
>>> its output redirected to a flat file?  Do you see all 55k messages using wc
>>> -l?
>>>
>>> On Thu, Apr 1, 2010 at 6:51 AM, Clayton Dukes <cdukes at gmail.com> wrote:
>>>>
>>>> I should have mentioned that this is logging directly to a file.
>>>>
>>>> destination df_logzilla {
>>>>    file("/var/log/logzilla/syslog.log"
>>>>
>>>> template("$HOST\t$FACILITY\t$LEVEL\t$TAG\t$YEAR-$MONTH-$DAY\t$HOUR:$MIN:$SEC\t$PROGRAM\t$MSG\n")
>>>>    );
>>>> };
>>>>
>>>>
>>>> On Wed, Mar 31, 2010 at 11:47 PM, Clayton Dukes <cdukes at gmail.com> wrote:
>>>>>
>>>>> Hi Folks,
>>>>> I'm trying to run a test to check insert rates.
>>>>> If I run this command:
>>>>>
>>>>> ./loggen -r 5000 -D -I 10 127.0.0.1 514
>>>>>
>>>>> The output shows:
>>>>> average rate = 5441.60 msg/sec, count=54420, time=10.007, msg size=256,
>>>>> bandwidth=1360.40 kB/sec
>>>>>
>>>>> But, my stats don't show that many messages received:
>>>>>
>>>>> syslog-ng[6660]: Log statistics; dropped=\'pipe(/dev/xconsole)=0\',
>>>>> processed=\'center(queued)=24232\', processed=\'center(received)=8077,
>>>>> processed=\'destination(df_logzilla)=8077\'
>>>>>
>>>>> As you can see, it sent 55k messages, but I only received 8k.
>>>>> Am I doing something wrong?
>>>>>
>>>>> Here are my options in the syslog-ng config:
>>>>> options {
>>>>>       long_hostnames(off);
>>>>>       log_msg_size(8192);
>>>>>       flush_lines(1);
>>>>>       log_fifo_size(16384);
>>>>>       time_reopen(10);
>>>>>       use_dns(yes);
>>>>>       dns_cache(yes);
>>>>>       use_fqdn(yes);
>>>>>       keep_hostname(yes);
>>>>>       chain_hostnames(no);
>>>>>       perm(0644);
>>>>>      stats_freq(60);
>>>>>
>>>>> };
>>>>>
>>>>>
>>>>> --
>>>>> ______________________________________________________________
>>>>>
>>>>> Clayton Dukes
>>>>> ______________________________________________________________
>>>>
>>>>
>>>>
>>>> --
>>>> ______________________________________________________________
>>>>
>>>> Clayton Dukes
>>>> ______________________________________________________________
>>>>
>>>>
>>>> ______________________________________________________________________________
>>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>>> Documentation:
>>>> http://www.balabit.com/support/documentation/?product=syslog-ng
>>>> FAQ: http://www.campin.net/syslog-ng/faq.html
>>>>
>>>>
>>>
>>>
>>
>>
>>
>> --
>> ______________________________________________________________
>>
>> Clayton Dukes
>> ______________________________________________________________
>>
>
>
>
> --
> ______________________________________________________________
>
> Clayton Dukes
> ______________________________________________________________
>



-- 
______________________________________________________________

Clayton Dukes
______________________________________________________________


More information about the syslog-ng mailing list