<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
I compiled version 2.1.14 but nothing has changed.<br>
<br>
I removed all my configuration and put configuration mentionned on this
blog:
<a class="moz-txt-link-freetext" href="http://bazsi.blogs.balabit.com/2007/12/syslog-ng-fun-with-performance.html">http://bazsi.blogs.balabit.com/2007/12/syslog-ng-fun-with-performance.html</a><br>
<br>
syslog-ng-server:~# loggen -s 150 -r 100000 -S 127.0.0.1 2000<br>
average rate = 65539.50 msg/sec, count=655395<br>
<br>
syslog-ng-client:~# loggen -r 100000 -s 150 -i -S xxx.xxx.xxx.xxx 2000<br>
average rate = 22832.30 msg/sec, count=228323<br>
<br>
I wone 2 000 msg/sec upgrading my kernel to 2.6.26.<br>
<br>
Is there any TCP sysctl flag I can enable to make TCP connection to
syslog server better that you have in mind ?<br>
<br>
<br>
Rémi BUISSON wrote:
<blockquote cite="mid:4B7911C7.2020603@steek.com" type="cite">
<meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
Siem,<br>
<br>
Thanks for trying helping me.<br>
<br>
My ulimit value was unlimited.<br>
All my processes write <log$pid>m characters</log> so each
process have its own n unique lines.<br>
<br>
I added a destination for my local5 which is the file /root/test.log.<br>
<br>
I tried: ./test_syslog.pl -p 5 -n 100 -m 1000<br>
<br>
on log client:<br>
# wc -l /root/test.log<br>
500 test.log<br>
<br>
on log server:<br>
# wc -l test.log <br>
0 test.log<br>
<br>
Then:<br>
./test_syslog.pl -p 1000 -n 1000 -m 1000<br>
<br>
on log client:<br>
# wc -l /root/test.log<br>
756688 test.log<br>
<br>
on log server:<br>
# wc -l test.log <br>
9042 test.log<br>
<br>
The client outputs:<br>
...<br>
Finished 9857!<br>
...<br>
Finished 10904!<br>
...<br>
<br>
So randomly near the firsts and lasts processes spawned:<br>
<br>
client# grep 10904 test.log | wc -l<br>
0<br>
client# grep 9857 test.log | wc -l<br>
1000<br>
<br>
server# grep 9857 test.log | wc -l<br>
4<br>
<br>
Sample of log:<br>
Feb 15 10:01:05 xxxx logger:
<log9857>000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000</log><br>
<br>
So, clearly the log server do not receive all logs but the client do
not seem to be able to process a large amount of logging message.<br>
<br>
Each test result number is nearly the same. It's good to see there is
no random in my tests ;-)<br>
<br>
Do you see the thing which make it not working ?<br>
<br>
Siem Korteweg wrote:
<blockquote
cite="mid:61F7C813E194BE4B978C9F1A16165CBE06B75F@EX01.QNH.local"
type="cite">
<pre wrap="">Remi,
just to make sure. Do your ulimit settings allow you to spawn the p (1000)
processes in paralel?
Considering your test. Did each instance of the test program write it's own
unique lines and can you see whether some processes did not make it to syslog
or that all processes produced partial logging?
regards,
Siem Korteweg
-----Oorspronkelijk bericht-----
Van: <a moz-do-not-send="true" class="moz-txt-link-abbreviated"
href="mailto:syslog-ng-bounces@lists.balabit.hu">syslog-ng-bounces@lists.balabit.hu</a> namens Rémi BUISSON
Verzonden: vr 12-2-2010 17:51
Aan: <a moz-do-not-send="true" class="moz-txt-link-abbreviated"
href="mailto:syslog-ng@lists.balabit.hu">syslog-ng@lists.balabit.hu</a>
Onderwerp: [syslog-ng] syslog-ng performance tuning
Hi everybody,
I'have an issue with syslog-ng configuration.
I would like to centralize my logs on one server.
I've a lot of logs to send. I don't know how many but I can estimate it
to 500GB per day from decades of servers.
But, it writes only 25 GB per day.
For some reasons I work on a debian etchnhalf environnement.
So, I'm working with syslog-ng 2.0.0.
I wrote a perl program which spawn p "logger -p local5.info" processes
and send n lines of m characters.
I'have tested with:
p: 1 000
n: 1 000
m: 1 000
Instead of having 1 000 000 lines in my logs I have nearly 10 000 lines !
But my test was not revelant because normal logs where not stopped. So,
maybe normal.
</pre>
<pre wrap=""><hr size="4" width="90%">
______________________________________________________________________________
Member info: <a moz-do-not-send="true" class="moz-txt-link-freetext"
href="https://lists.balabit.hu/mailman/listinfo/syslog-ng">https://lists.balabit.hu/mailman/listinfo/syslog-ng</a>
Documentation: <a moz-do-not-send="true" class="moz-txt-link-freetext"
href="http://www.balabit.com/support/documentation/?product=syslog-ng">http://www.balabit.com/support/documentation/?product=syslog-ng</a>
FAQ: <a moz-do-not-send="true" class="moz-txt-link-freetext"
href="http://www.campin.net/syslog-ng/faq.html">http://www.campin.net/syslog-ng/faq.html</a>
</pre>
</blockquote>
<br>
<div class="moz-signature">-- <br>
<meta http-equiv="Content-type" content="text/html;">
<style type="text/css">
div.sign
{
width: 230px;
padding: 5px;
}
span.name
{
font-family: sans-serif;
font-size: 10pt;
font-weight: bold;
color: #8f9ed5;
}
span.company
{
display: block;
font-family: sans-serif;
font-size: 9pt;
font-weight: bold;
color: #7d7ddf;
}
span.address
{
display: block;
font-family: sans-serif;
font-size: 9pt;
font-weight: bold;
color: #5d5d5d;
}
span.job
{
font-family: sans-serif;
font-size: 9pt;
font-weight: bold;
color: #5d5d5d;
}
div.sign img
{
float: right;
width: 100px;
}
</style>
<title>Signature</title>
<div class="sign"> <span class="name">Rémi BUISSON</span> - <span
class="job">IT Engineer</span> <span class="company">F-Secure Storage
& Digital Content</span> <span class="address">7, rue Raymond
Manaud<br>
33524 BORDEAUX Bruges Cedex<br>
FRANCE</span> <img src="cid:part1.01070709.08090005@steek.com"
alt="http://www.f-secure.com/export/system/modules/com.fsecure.frontend.newbrand/resources/css/_ui/images/f-secure.png">
</div>
</div>
<pre wrap="">
<hr size="4" width="90%">
______________________________________________________________________________
Member info: <a class="moz-txt-link-freetext" href="https://lists.balabit.hu/mailman/listinfo/syslog-ng">https://lists.balabit.hu/mailman/listinfo/syslog-ng</a>
Documentation: <a class="moz-txt-link-freetext" href="http://www.balabit.com/support/documentation/?product=syslog-ng">http://www.balabit.com/support/documentation/?product=syslog-ng</a>
FAQ: <a class="moz-txt-link-freetext" href="http://www.campin.net/syslog-ng/faq.html">http://www.campin.net/syslog-ng/faq.html</a>
</pre>
</blockquote>
<br>
<div class="moz-signature">-- <br>
<meta http-equiv="Content-type" content="text/html;">
<style type="text/css">
div.sign
{
width: 230px;
padding: 5px;
}
span.name
{
font-family: sans-serif;
font-size: 10pt;
font-weight: bold;
color: #8f9ed5;
}
span.company
{
display: block;
font-family: sans-serif;
font-size: 9pt;
font-weight: bold;
color: #7d7ddf;
}
span.address
{
display: block;
font-family: sans-serif;
font-size: 9pt;
font-weight: bold;
color: #5d5d5d;
}
span.job
{
font-family: sans-serif;
font-size: 9pt;
font-weight: bold;
color: #5d5d5d;
}
div.sign img
{
float: right;
width: 100px;
}
</style>
<title>Signature</title>
<div class="sign"> <span class="name">Rémi BUISSON</span> - <span
class="job">IT Engineer</span> <span class="company">F-Secure Storage
& Digital Content</span> <span class="address">7, rue Raymond
Manaud<br>
33524 BORDEAUX Bruges Cedex<br>
FRANCE</span> <img src="cid:part2.02060009.03040904@steek.com"
alt="http://www.f-secure.com/export/system/modules/com.fsecure.frontend.newbrand/resources/css/_ui/images/f-secure.png">
</div>
</div>
</body>
</html>