[syslog-ng] syslog-ng performance tuning
Rémi BUISSON
rbuisson at steek.com
Mon Feb 22 10:15:36 CET 2010
Yes, of course, please find configuration files in attachment.
This is customed debian based configuration so maybe you will have to
tune it.
The syslog server logs local2 (custom program which logs on local2) and
apache error/access logs on local0.
For apache you will need this kind of lines in your vhost configuration
files:
ErrorLog "|/usr/bin/logger -p local0.error -t www.test.com"
CustomLog "|/usr/bin/logger -p local0.info -t www.test.com" combined
The server will log apache logs on :
/logs/www.test.com/<ip_client>-access.log
/logs/www.test.com/<ip_client>-error.log
If it cans help you I attached my logrotate configuration too.
It rotates every 40 days and tar the month rotated.
So you will have every apache logs gzipped in one tar.gz.
Regards.
fedora fedora wrote:
> Do you think you can you share your modified config? thanks
>
> On Fri, Feb 19, 2010 at 2:27 AM, Rémi BUISSON <rbuisson at steek.com
> <mailto:rbuisson at steek.com>> wrote:
>
> Hi,
>
> For those who are interested in, I solved my issue.
>
> The problem was I had too many filter rules.
> Using macros, I reduce about 600 rules to 3.
>
> Now I get my syslog server working and no more lost messages.
>
> Rémi
>
>
> Rémi BUISSON wrote:
>> I compiled version 2.1.14 but nothing has changed.
>>
>> I removed all my configuration and put configuration mentionned
>> on this blog:
>> http://bazsi.blogs.balabit.com/2007/12/syslog-ng-fun-with-performance.html
>>
>> syslog-ng-server:~# loggen -s 150 -r 100000 -S 127.0.0.1 2000
>> average rate = 65539.50 msg/sec, count=655395
>>
>> syslog-ng-client:~# loggen -r 100000 -s 150 -i -S xxx.xxx.xxx.xxx
>> 2000
>> average rate = 22832.30 msg/sec, count=228323
>>
>> I wone 2 000 msg/sec upgrading my kernel to 2.6.26.
>>
>> Is there any TCP sysctl flag I can enable to make TCP connection
>> to syslog server better that you have in mind ?
>>
>>
>> Rémi BUISSON wrote:
>>> Siem,
>>>
>>> Thanks for trying helping me.
>>>
>>> My ulimit value was unlimited.
>>> All my processes write <log$pid>m characters</log> so each
>>> process have its own n unique lines.
>>>
>>> I added a destination for my local5 which is the file
>>> /root/test.log.
>>>
>>> I tried: ./test_syslog.pl <http://test_syslog.pl> -p 5 -n 100 -m
>>> 1000
>>>
>>> on log client:
>>> # wc -l /root/test.log
>>> 500 test.log
>>>
>>> on log server:
>>> # wc -l test.log
>>> 0 test.log
>>>
>>> Then:
>>> ./test_syslog.pl <http://test_syslog.pl> -p 1000 -n 1000 -m 1000
>>>
>>> on log client:
>>> # wc -l /root/test.log
>>> 756688 test.log
>>>
>>> on log server:
>>> # wc -l test.log
>>> 9042 test.log
>>>
>>> The client outputs:
>>> ...
>>> Finished 9857!
>>> ...
>>> Finished 10904!
>>> ...
>>>
>>> So randomly near the firsts and lasts processes spawned:
>>>
>>> client# grep 10904 test.log | wc -l
>>> 0
>>> client# grep 9857 test.log | wc -l
>>> 1000
>>>
>>> server# grep 9857 test.log | wc -l
>>> 4
>>>
>>> Sample of log:
>>> Feb 15 10:01:05 xxxx logger:
>>> <log9857>000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
>>> 0000000000000000000000000</log>
>>>
>>> So, clearly the log server do not receive all logs but the
>>> client do not seem to be able to process a large amount of
>>> logging message.
>>>
>>> Each test result number is nearly the same. It's good to see
>>> there is no random in my tests ;-)
>>>
>>> Do you see the thing which make it not working ?
>>>
>>> Siem Korteweg wrote:
>>>> Remi,
>>>>
>>>> just to make sure. Do your ulimit settings allow you to spawn the p (1000)
>>>> processes in paralel?
>>>>
>>>> Considering your test. Did each instance of the test program write it's own
>>>> unique lines and can you see whether some processes did not make it to syslog
>>>> or that all processes produced partial logging?
>>>>
>>>> regards,
>>>>
>>>> Siem Korteweg
>>>>
>>>> -----Oorspronkelijk bericht-----
>>>> Van: syslog-ng-bounces at lists.balabit.hu <mailto:syslog-ng-bounces at lists.balabit.hu> namens Rémi BUISSON
>>>> Verzonden: vr 12-2-2010 17:51
>>>> Aan: syslog-ng at lists.balabit.hu <mailto:syslog-ng at lists.balabit.hu>
>>>> Onderwerp: [syslog-ng] syslog-ng performance tuning
>>>>
>>>> Hi everybody,
>>>>
>>>> I'have an issue with syslog-ng configuration.
>>>> I would like to centralize my logs on one server.
>>>>
>>>> I've a lot of logs to send. I don't know how many but I can estimate it
>>>> to 500GB per day from decades of servers.
>>>> But, it writes only 25 GB per day.
>>>> For some reasons I work on a debian etchnhalf environnement.
>>>> So, I'm working with syslog-ng 2.0.0.
>>>>
>>>> I wrote a perl program which spawn p "logger -p local5.info <http://local5.info>" processes
>>>> and send n lines of m characters.
>>>>
>>>> I'have tested with:
>>>> p: 1 000
>>>> n: 1 000
>>>> m: 1 000
>>>>
>>>> Instead of having 1 000 000 lines in my logs I have nearly 10 000 lines !
>>>> But my test was not revelant because normal logs where not stopped. So,
>>>> maybe normal.
>>>>
>>>>
>>>> ------------------------------------------------------------------------
>>>>
>>>> ______________________________________________________________________________
>>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>>> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
>>>> FAQ: http://www.campin.net/syslog-ng/faq.html
>>>>
>>>>
>>>
>>> --
>>> Rémi BUISSON - IT Engineer F-Secure Storage & Digital Content 7,
>>> rue Raymond Manaud
>>> 33524 BORDEAUX Bruges Cedex
>>> FRANCE
>>> http://www.f-secure.com/export/system/modules/com.fsecure.frontend.newbrand/resources/css/_ui/images/f-secure.png
>>>
>>> ------------------------------------------------------------------------
>>>
>>> ______________________________________________________________________________
>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
>>> FAQ: http://www.campin.net/syslog-ng/faq.html
>>>
>>>
>>
>> --
>> Rémi BUISSON - IT Engineer F-Secure Storage & Digital Content 7,
>> rue Raymond Manaud
>> 33524 BORDEAUX Bruges Cedex
>> FRANCE
>> http://www.f-secure.com/export/system/modules/com.fsecure.frontend.newbrand/resources/css/_ui/images/f-secure.png
>>
>> ------------------------------------------------------------------------
>>
>> ______________________________________________________________________________
>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
>> FAQ: http://www.campin.net/syslog-ng/faq.html
>>
>>
>
> --
> Rémi BUISSON - IT Engineer F-Secure Storage & Digital Content 7,
> rue Raymond Manaud
> 33524 BORDEAUX Bruges Cedex
> FRANCE
> http://www.f-secure.com/export/system/modules/com.fsecure.frontend.newbrand/resources/css/_ui/images/f-secure.png
>
>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation:
> http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.campin.net/syslog-ng/faq.html
>
>
>
--
Rémi BUISSON - IT Engineer F-Secure Storage & Digital Content 7, rue
Raymond Manaud
33524 BORDEAUX Bruges Cedex
FRANCE
http://www.f-secure.com/export/system/modules/com.fsecure.frontend.newbrand/resources/css/_ui/images/f-secure.png
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.balabit.hu/pipermail/syslog-ng/attachments/20100222/5155841a/attachment-0001.htm
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/png
Size: 3477 bytes
Desc: not available
Url : http://lists.balabit.hu/pipermail/syslog-ng/attachments/20100222/5155841a/attachment-0004.png
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/png
Size: 3477 bytes
Desc: not available
Url : http://lists.balabit.hu/pipermail/syslog-ng/attachments/20100222/5155841a/attachment-0005.png
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/png
Size: 3477 bytes
Desc: not available
Url : http://lists.balabit.hu/pipermail/syslog-ng/attachments/20100222/5155841a/attachment-0006.png
-------------- next part --------------
A non-text attachment was scrubbed...
Name: f-secure.png
Type: image/png
Size: 3477 bytes
Desc: not available
Url : http://lists.balabit.hu/pipermail/syslog-ng/attachments/20100222/5155841a/attachment-0007.png
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: syslog-ng.conf-client
Url: http://lists.balabit.hu/pipermail/syslog-ng/attachments/20100222/5155841a/attachment-0003.txt
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: syslog-ng.conf-server
Url: http://lists.balabit.hu/pipermail/syslog-ng/attachments/20100222/5155841a/attachment-0004.txt
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: apache2-syslog-ng
Url: http://lists.balabit.hu/pipermail/syslog-ng/attachments/20100222/5155841a/attachment-0005.txt
More information about the syslog-ng
mailing list