Performance analysis (was Re: UDP packet loss with syslog-ng)
I've run a series of tests against our log server, using loggen-generated logging rates of 100, 1000, 2000, 4000, 8000, 16000, and 32000 messages/sec, and measured the results for each rate using socket buffer sizes ranging from 128KB to 16M. The results show, essentially, what buffer size you need to meet a target rate of message logging. So I'm happy now, and I'm getting the sort of rates I expect to be able to support. I figured other folks might like the data. I've put the raw data online as well as a graph of the results: http://drop.io/syslog_ng The graph shows so_rcvbuf() size along the X axis, and packet loss along the Y axis.
Cool, thanks for the numbers. So it took about a 16 MB buffer for you to get no drops at 16k mps? On Fri, Oct 15, 2010 at 11:04 PM, Lars Kellogg-Stedman <lars@oddbit.com> wrote:
I've run a series of tests against our log server, using loggen-generated logging rates of 100, 1000, 2000, 4000, 8000, 16000, and 32000 messages/sec, and measured the results for each rate using socket buffer sizes ranging from 128KB to 16M. The results show, essentially, what buffer size you need to meet a target rate of message logging.
So I'm happy now, and I'm getting the sort of rates I expect to be able to support. I figured other folks might like the data. I've put the raw data online as well as a graph of the results:
The graph shows so_rcvbuf() size along the X axis, and packet loss along the Y axis. ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html
Cool, thanks for the numbers. So it took about a 16 MB buffer for you to get no drops at 16k mps?
Yeah. Is that typical? I have no idea.
On Sat, 2010-10-16 at 15:54 -0400, Lars Kellogg-Stedman wrote:
Cool, thanks for the numbers. So it took about a 16 MB buffer for you to get no drops at 16k mps?
Yeah. Is that typical? I have no idea.
depends on the size of the incoming frames & latency imposed by the destinations. -- Bazsi
participants (3)
-
Balazs Scheidler
-
Lars Kellogg-Stedman
-
Martin Holste