Calibration of the syslog-ng server
Hi, I want to collect about 1500 hosts. I've estimated that it represents about 8 000 000 messages (1 Gb) daily. My syslog-ng server, which will be dedicated to this, will run on solaris 8. Have you got any idea about needed cpu number and memory ? Thanks for your answers. JM Loubet
On Thu, Jun 12, 2003 at 05:33:24PM +0200, Loubet Jean-Michel wrote:
I want to collect about 1500 hosts. I've estimated that it represents about 8 000 000 messages (1 Gb) daily.
My syslog-ng server, which will be dedicated to this, will run on solaris 8.
Have you got any idea about needed cpu number and memory ?
The biggest issues with large log volumes have two broad issues: 1) network throughput 2) disk subsystem throughput Bursts of messages will probably be the biggest enemy of getting logs committed to disk. People regularly report issues with UDP receive buffers filling and the OS dropping packets silently. Using TCP everywhere possible and/or increase your UDP receive buffer size is a good start. As for hardware, lots of memory is good for filesystem caching, on Solaris all disk I/O goes through memory. I'd say get some nice 10k RPM lvd scsi disks, balanced with raid across several spindles (with parity/mirroring of some sort since it's logs, maybe raid 5 or raid 1+0). Two cpus might be good, you'd hope one CPU handles most system stuff and the second does syslog-ng. If Solaris does one thing well it's scaling almost linearly with additional processors. Lots of people put a couple syslog servers behind a load balancer, which makes configuration simple all around (no syslog configs for clients or servers have to know anything about the load balancer). If your peak periods are too bursty that might end up a requirement. See: http://www.campin.net/syslog-ng/faq.html#how_fast -- Nate Campi http://www.campin.net
JM, I am currently working on getting a syslog server up and working. In my testing, I have pointed most of my mail servers, about 8 of them, to my log server and I have not noticed any proformance hits. I am using an older Winchester raid array for my space for logs. I believe that is where most of the proformance issues will come from. Each of the mail servers are sending about 400megs of logs daily. I hae not noticed any problems so far. I am not using solaris as my OS, but an older dual P3, and my load averages not next to nothing. Drive speed will be the most important. Hope this helps --Keith .On Thu, 12 Jun 2003, Loubet Jean-Michel wrote: Date: Thu, 12 Jun 2003 17:33:24 +0200 From: Loubet Jean-Michel <JeanMichel.Loubet@atosorigin.com> Reply-To: syslog-ng@lists.balabit.hu To: syslog-ng@lists.balabit.hu Subject: [syslog-ng]Calibration of the syslog-ng server Hi, I want to collect about 1500 hosts. I've estimated that it represents about 8 000 000 messages (1 Gb) daily. My syslog-ng server, which will be dedicated to this, will run on solaris 8. Have you got any idea about needed cpu number and memory ? Thanks for your answers. JM Loubet _______________________________________________ syslog-ng maillist - syslog-ng@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html --
participants (3)
-
Keith Olmstead
-
Loubet Jean-Michel
-
Nate Campi