[syslog-ng] syslog-ng performance tuning

[Rémi BUISSON]

I compiled version 2.1.14 but nothing has changed.

I removed all my configuration and put configuration mentionned on this 
blog: 
http://bazsi.blogs.balabit.com/2007/12/syslog-ng-fun-with-performance.html

syslog-ng-server:~# loggen -s 150 -r 100000 -S 127.0.0.1 2000
average rate = 65539.50 msg/sec, count=655395

syslog-ng-client:~# loggen -r 100000 -s 150 -i -S xxx.xxx.xxx.xxx 2000
average rate = 22832.30 msg/sec, count=228323

I wone 2 000 msg/sec upgrading my kernel to 2.6.26.

Is there any TCP sysctl flag I can enable to make TCP connection to 
syslog server better that you have in mind ?


Rémi BUISSON wrote:
> Siem,
>
> Thanks for trying helping me.
>
> My ulimit value was unlimited.
> All my processes write <log$pid>m characters</log> so each process 
> have its own n unique lines.
>
> I added a destination for my local5 which is the file /root/test.log.
>
> I tried: ./test_syslog.pl -p 5 -n 100 -m 1000
>
> on log client:
> # wc -l /root/test.log
> 500 test.log
>
> on log server:
> # wc -l test.log
> 0 test.log
>
> Then:
>  ./test_syslog.pl -p 1000 -n 1000 -m 1000
>
> on log client:
> # wc -l /root/test.log
> 756688 test.log
>
> on log server:
> # wc -l test.log
> 9042 test.log
>
> The client outputs:
> ...
> Finished 9857!
> ...
> Finished 10904!
> ...
>
> So randomly near the firsts and lasts processes spawned:
>
> client# grep 10904 test.log | wc -l
> 0
> client# grep 9857 test.log | wc -l
> 1000
>
> server# grep 9857 test.log | wc -l
> 4
>
> Sample of log:
> Feb 15 10:01:05 xxxx logger: 
> <log9857>000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 
> 0000000000000000000000000</log>
>
> So, clearly the log server do not receive all logs but the client do 
> not seem to be able to process a large amount of logging message.
>
> Each test result number is nearly the same. It's good to see there is 
> no random in my tests ;-)
>
> Do you see the thing which make it not working ?
>
> Siem Korteweg wrote:
>> Remi,
>>
>> just to make sure. Do your ulimit settings allow you to spawn the p (1000)
>> processes in paralel?
>>
>> Considering your test. Did each instance of the test program write it's own
>> unique lines and can you see whether some processes did not make it to syslog
>> or that all processes produced partial logging?
>>
>> regards,
>>
>> Siem Korteweg
>>
>> -----Oorspronkelijk bericht-----
>> Van: syslog-ng-bounces at lists.balabit.hu namens Rémi BUISSON
>> Verzonden: vr 12-2-2010 17:51
>> Aan: syslog-ng at lists.balabit.hu
>> Onderwerp: [syslog-ng] syslog-ng performance tuning
>>  
>> Hi everybody,
>>
>> I'have an issue with syslog-ng configuration.
>> I would like to centralize my logs on one server.
>>
>> I've a lot of logs to send. I don't know how many but I can estimate it 
>> to 500GB per day from decades of servers.
>> But, it writes only 25 GB per day.
>> For some reasons I work on a debian etchnhalf environnement.
>> So, I'm working with syslog-ng 2.0.0.
>>
>> I wrote a perl program which spawn p "logger -p local5.info" processes
>> and send n lines of m characters.
>>
>> I'have tested with:
>> p: 1 000
>> n: 1 000
>> m: 1 000
>>
>> Instead of having 1 000 000 lines in my logs I have nearly 10 000 lines !
>> But my test was not revelant because normal logs where not stopped. So, 
>> maybe normal.
>>
>>   
>> ------------------------------------------------------------------------
>>
>> ______________________________________________________________________________
>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
>> FAQ: http://www.campin.net/syslog-ng/faq.html
>>
>>   
>
> -- 
> Rémi BUISSON - IT Engineer F-Secure Storage & Digital Content 7, rue 
> Raymond Manaud
> 33524 BORDEAUX Bruges Cedex
> FRANCE 
> http://www.f-secure.com/export/system/modules/com.fsecure.frontend.newbrand/resources/css/_ui/images/f-secure.png 
>
> ------------------------------------------------------------------------
>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.campin.net/syslog-ng/faq.html
>
>   

-- 
Rémi BUISSON - IT Engineer F-Secure Storage & Digital Content 7, rue 
Raymond Manaud
33524 BORDEAUX Bruges Cedex
FRANCE 
http://www.f-secure.com/export/system/modules/com.fsecure.frontend.newbrand/resources/css/_ui/images/f-secure.png 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.balabit.hu/pipermail/syslog-ng/attachments/20100216/426efea3/attachment.htm 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/png
Size: 3477 bytes
Desc: not available
Url : http://lists.balabit.hu/pipermail/syslog-ng/attachments/20100216/426efea3/attachment.png 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: f-secure.png
Type: image/png
Size: 3477 bytes
Desc: not available
Url : http://lists.balabit.hu/pipermail/syslog-ng/attachments/20100216/426efea3/attachment-0001.png