[syslog-ng]losing messages using syslog-ng-1.4.7 & libol-0.2.20

[Balazs Scheidler]

> I am using syslog-ng 1.4.7 and libol 0.2.20 and am attempting to do some
> tests to make sure that I can get the level of performance that I need.  I
> am having the worst trouble though trying to isolate whether a problem is
> occuring in either syslog-ng or within the networking code of the machines
> that I am using.  I've managed to hack things in the network code for
> some of the machines to get better performance for some of the tests, but
> we have dropped down the requirements now.
> 
> I am sending along some test code that should compile on either red hat
> 6.2, red hat 7.0, and hopefully others.  (Haven't tried) The problem I am
> running into is that I lose anywhere between 30 and  800 messages.  As the
> message number goes up and number of clients, I lose more messages.
> 
> Some results I have gotten, with 100 threads connecting to the server, I
> can send 60,000 messages in about 30 seconds without a problem (200
> bytes), but when I increase the number to 300 threads, and I have it
> sending for 30 seconds for a total of 36,000 messages, I start to lose
> messages.  Notice in the first case I was sending more messages and had it
> sleeping for less time between instances, in the second case it is
> sleeping longer and sending less messages per second per thread.
> 
> I have tried it with multiple different machines and types of varying
> speeds, and it happens on all of them.  For the latest run, I have for teh
> sending machine an AMD athlon 1Ghz w 256M RAM, with an ATA 100 Hard drive
> running on ATA66 running red hat 7.0.  For the server, I have a PII 300Mhz
> with 180M RAM, a ATA33 hard drive running without acceleration.  The
> server is running red hat 6.2.  I have tried the numbers with the sender
> and receiver switched, but I lose more messages that way.
> 
> If you look in the code at logconf.h, you can change the #defines.  The
> logtest.c just creates the threads.  log_thread.c has the send_messages
> function that simulates a syslog-ng client.  If anyone can provide any
> help, it would be greatly appreciated, since I have to be able to show
> this working before I can get the full sign-off to use it for our the
> applications we are building.

Can you send me your syslog-ng config file on server? What I'd be especially
interested is your log_fifo_size() setting.

-- 
Bazsi
PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1
     url: http://www.balabit.hu/pgpkey.txt