[syslog-ng] syslog-ng stuck on one CPU but need to share it with multiple CPUs.
Mukarram Syed
msyed at walmart.com
Thu Dec 2 00:06:50 CET 2010
Interesting...I think I get the idea.
That way you could split the syslog-ng.conf file into multiple files, running multiple syslog-ng processes and not run into the 5000 line limit problem in the syslog-ng.conf that I faced a few months ago.
Thanks
# mukarram
From: syslog-ng-bounces at lists.balabit.hu [mailto:syslog-ng-bounces at lists.balabit.hu] On Behalf Of Patrick H.
Sent: Wednesday, December 01, 2010 3:00 PM
To: Syslog-ng users' and developers' mailing list
Subject: Re: [syslog-ng] syslog-ng stuck on one CPU but need to share it with multiple CPUs.
The way I do it is with separate init scripts and separate configs. The scripts launch the additional processes with args like '-f /etc/syslog-ng/syslog-ng.515.conf -R /var/lib/syslog-ng/syslog-ng.515.persist -p /var/lib/syslog-ng/syslog-ng.515.pid'. That config then has a tcp source driver listening on port 515, and the master process just sends traffic to localhost:515. You could use pipes or whatever transport you want, I just chose TCP.
-Patrick
Sent: Wed Dec 01 2010 15:35:36 GMT-0700 (Mountain Standard Time)
From: Mukarram Syed <msyed at walmart.com><mailto:msyed at walmart.com>
To: 'Syslog-ng users' and developers' mailing list' <syslog-ng at lists.balabit.hu><mailto:syslog-ng at lists.balabit.hu>
Subject: Re: [syslog-ng] syslog-ng stuck on one CPU but need to share it with multiple CPUs.
Thanks Patrick.
Any leads as to how you would do this?
Thanks
# mukarram
From: syslog-ng-bounces at lists.balabit.hu<mailto:syslog-ng-bounces at lists.balabit.hu> [mailto:syslog-ng-bounces at lists.balabit.hu] On Behalf Of Patrick H.
Sent: Wednesday, December 01, 2010 2:32 PM
To: Syslog-ng users' and developers' mailing list
Subject: Re: [syslog-ng] syslog-ng stuck on one CPU but need to share it with multiple CPUs.
I (and others) got around this problem by launching multiple syslog-ng processes and having a master process do very basic filtering and routing to the child processes.
I think I remember hearing that syslog-ng 3.2 was going to use threads (at least for destination drivers) but dont quote me on that.
-Patrick
Sent: Wed Dec 01 2010 14:57:28 GMT-0700 (Mountain Standard Time)
From: Mukarram Syed <msyed at walmart.com><mailto:msyed at walmart.com>
To: 'syslog-ng at lists.balabit.hu<mailto:syslog-ng at lists.balabit.hu>' <syslog-ng at lists.balabit.hu><mailto:syslog-ng at lists.balabit.hu>
Subject: [syslog-ng] syslog-ng stuck on one CPU but need to share it with multiple CPUs.
Hi,
I have syslog-ng version syslog-ng-2.1.4-1.el5 running on RHEL5.
When I run the top, syslog-ng process runs on only one CPU, but this is a 24 Core server.
How do I make syslog-ng share the load across several CPUs or cores rather than have it stuck in one CPU?
Any suggestions are appreciated.
TOP output for syslog-ng:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
11304 root 18 0 2023m 1.9g 1252 R 95.7 2.7 22658:29 syslog-ng
MPSTAT output:
01:16:53 PM CPU %user %nice %sys %iowait %irq %soft %steal %idle intr/s
01:16:54 PM all 6.37 0.00 1.38 0.00 0.00 0.17 0.00 92.08 5073.74
01:16:54 PM 0 90.91 0.00 4.04 0.00 1.01 3.03 0.00 1.01 4942.42 - my guess is this is using syslog-ng
01:16:54 PM 1 0.00 0.00 1.00 0.00 0.00 0.00 0.00 99.00 0.00
01:16:54 PM 2 14.00 0.00 3.00 0.00 0.00 0.00 0.00 83.00 0.00
01:16:54 PM 3 0.00 0.00 1.00 0.00 0.00 0.00 0.00 99.00 0.00
01:16:54 PM 4 0.00 0.00 0.00 0.00 0.00 0.00 0.00 100.00 0.00
01:16:54 PM 5 12.00 0.00 6.00 0.00 0.00 0.00 0.00 82.00 0.00
01:16:54 PM 6 11.11 0.00 4.04 0.00 0.00 0.00 0.00 84.85 0.00
01:16:54 PM 7 0.00 0.00 1.01 0.00 0.00 0.00 0.00 98.99 0.00
01:16:54 PM 8 0.00 0.00 1.00 0.00 0.00 0.00 0.00 99.00 0.00
01:16:54 PM 9 11.11 0.00 2.02 0.00 0.00 0.00 0.00 86.87 0.00
01:16:54 PM 10 1.00 0.00 6.00 1.00 0.00 1.00 0.00 91.00 122.22
01:16:54 PM 11 0.00 0.00 1.01 0.00 0.00 0.00 0.00 98.99 0.00
01:16:54 PM 12 0.00 0.00 0.00 0.00 0.00 0.00 0.00 100.00 0.00
01:16:54 PM 13 0.00 0.00 0.00 0.00 0.00 0.00 0.00 100.00 0.00
01:16:54 PM 14 0.00 0.00 0.00 0.00 0.00 0.00 0.00 100.00 0.00
01:16:54 PM 15 0.00 0.00 1.01 0.00 0.00 0.00 0.00 98.99 0.00
01:16:54 PM 16 0.00 0.00 0.00 0.00 0.00 0.00 0.00 100.00 0.00
01:16:54 PM 17 0.00 0.00 0.00 0.00 0.00 0.00 0.00 100.00 0.00
01:16:54 PM 18 0.00 0.00 0.00 0.00 0.00 0.00 0.00 100.00 0.00
01:16:54 PM 19 10.20 0.00 1.02 0.00 0.00 0.00 0.00 88.78 0.00
01:16:54 PM 20 0.00 0.00 0.00 0.00 0.00 0.00 0.00 100.00 5.05
01:16:54 PM 21 0.00 0.00 0.00 0.00 0.00 0.00 0.00 100.00 0.00
01:16:54 PM 22 0.00 0.00 0.00 0.00 0.00 0.00 0.00 100.00 0.00
01:16:54 PM 23 0.00 0.00 0.00 0.00 0.00 0.00 0.00 100.00 5.05
Thanks
# mukarram
Mukarram Syed
________________________________
______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.campin.net/syslog-ng/faq.html
________________________________
______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.campin.net/syslog-ng/faq.html
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.balabit.hu/pipermail/syslog-ng/attachments/20101201/b7163ddb/attachment-0001.htm
More information about the syslog-ng
mailing list