On Wed, 2007-12-12 at 13:40 +0100, FILEUX arnaud wrote:
Dear all,
I realized some bench tests in order to determine the limits supported in tcp connection with a syslog-ng (client/server mode) and what is the optimal configuration (buffer size,etc..).
My setup is as follows:
2 syslog-ng (client A - server B) on dedicated servers with 8Go RAM and quadripro CPU.
Client A is configured as follow:
destination to_serverB { tcp("x.x.x.x" port(1999)); }; destination to_clientA { file("/var/log/syslog-ng.log"); };
Server B: destination all { file("/var/log/syslog-ng.log"); };
On the client A i have a a C program (named stressyslog) who send send 10.000 to 500.000 messages on 10 to 100 threads and the delay between sending is 1ms to 10 ms to Client A with : syslog(LOG_NOTICE, "go %d pThread for %d messages and %d ms", iThreads, iMessages, iDelay);
I make a diff between clientA and serverB and i lose some lines on ServerB (delai less than 5ms and thread greater than 50)
example: delay: 2ms ; messages: 50000; thread 100;
Server B has 4947888 lines in syslog-ng.log Client A has 5000087 lines in syslog-ng.log
Another problem is syslog-ng never alert if some line are lost beteween Client A and Server B.
I have tried to increase the buffer size (10.000 and 100.000) but the same problem.
It'd be useful to know where the message is lost. There are multiple possibilities: 1) on the local log transport between your application and syslog-ng (e.g. /dev/log) are you using unix-dgram or unix-stream ? 2) by syslog-ng itself, if the message does not fit into the output buffer it'll be dropped, but these drops are counted, check the "log statistics" log line, it includes the dropped counter 3) if you are using tcp() then the TCP connection will not lose messages 4) the receiving syslog-ng may drop a message if the fifo of the destination file gets full, but this also shows up in the log statistics message of the server syslog-ng If you have identified where message gets lost, we might be able to help you. -- Bazsi