Interesting…I think I get
the idea.
That way you could split the
syslog-ng.conf file into multiple files, running multiple syslog-ng processes and
not run into the 5000 line limit problem in the syslog-ng.conf that I faced a
few months ago.
Thanks
# mukarram
From: syslog-ng-bounces@lists.balabit.hu
[mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Patrick H.
Sent: Wednesday, December 01, 2010 3:00 PM
To: Syslog-ng users' and developers' mailing list
Subject: Re: [syslog-ng] syslog-ng stuck on one CPU but need to share it
with multiple CPUs.
The
way I do it is with separate init scripts and separate configs. The scripts
launch the additional processes with args like '-f
/etc/syslog-ng/syslog-ng.515.conf -R /var/lib/syslog-ng/syslog-ng.515.persist
-p /var/lib/syslog-ng/syslog-ng.515.pid'. That config then has a tcp source
driver listening on port 515, and the master process just sends traffic to
localhost:515. You could use pipes or whatever transport you want, I just chose
TCP.
-Patrick
Sent: Wed Dec 01 2010 15:35:36 GMT-0700 (Mountain Standard Time)
From: Mukarram Syed <msyed@walmart.com>
To: 'Syslog-ng users' and developers' mailing list' <syslog-ng@lists.balabit.hu>
Subject: Re: [syslog-ng] syslog-ng stuck on one CPU but need to share it with
multiple CPUs.
Thanks Patrick.
Any leads as to how you would do
this?
Thanks
# mukarram
From: syslog-ng-bounces@lists.balabit.hu
[mailto:syslog-ng-bounces@lists.balabit.hu]
On Behalf Of Patrick H.
Sent: Wednesday, December 01, 2010 2:32 PM
To: Syslog-ng users' and developers' mailing list
Subject: Re: [syslog-ng] syslog-ng stuck on one CPU but need to share it
with multiple CPUs.
I
(and others) got around this problem by launching multiple syslog-ng processes
and having a master process do very basic filtering and routing to the child
processes.
I think I remember hearing that syslog-ng 3.2 was going to use threads (at
least for destination drivers) but dont quote me on that.
-Patrick
Sent: Wed Dec 01 2010 14:57:28 GMT-0700 (Mountain Standard Time)
From: Mukarram Syed <msyed@walmart.com>
To: 'syslog-ng@lists.balabit.hu'
<syslog-ng@lists.balabit.hu>
Subject: [syslog-ng] syslog-ng stuck on one CPU but need to share it
with multiple CPUs.
Hi,
I have syslog-ng version syslog-ng-2.1.4-1.el5 running on RHEL5.
When I run the top, syslog-ng process runs on only one CPU, but
this is a 24 Core server.
How do I make syslog-ng share the load across several CPUs or
cores rather than have it stuck in one CPU?
Any suggestions are appreciated.
TOP output for syslog-ng:
PID USER PR NI
VIRT RES SHR S %CPU %MEM TIME+
COMMAND
11304 root 18 0 2023m
1.9g 1252 R 95.7 2.7 22658:29 syslog-ng
MPSTAT output:
01:16:53 PM CPU %user
%nice %sys %iowait %irq
%soft %steal %idle intr/s
01:16:54 PM all 6.37
0.00 1.38 0.00
0.00 0.17 0.00
92.08 5073.74
01:16:54 PM 0
90.91 0.00 4.04
0.00 1.01 3.03 0.00
1.01 4942.42 – my guess is
this is using syslog-ng
01:16:54 PM 1
0.00 0.00 1.00 0.00
0.00 0.00 0.00
99.00 0.00
01:16:54 PM 2
14.00 0.00 3.00
0.00 0.00 0.00
0.00 83.00 0.00
01:16:54 PM 3
0.00 0.00 1.00
0.00 0.00 0.00
0.00 99.00 0.00
01:16:54 PM 4
0.00 0.00 0.00
0.00 0.00 0.00 0.00
100.00 0.00
01:16:54 PM 5
12.00 0.00 6.00
0.00 0.00 0.00
0.00 82.00 0.00
01:16:54 PM 6
11.11 0.00 4.04
0.00 0.00 0.00
0.00 84.85 0.00
01:16:54 PM 7
0.00 0.00 1.01
0.00 0.00 0.00
0.00 98.99 0.00
01:16:54 PM 8
0.00 0.00 1.00
0.00 0.00 0.00
0.00 99.00 0.00
01:16:54 PM 9
11.11 0.00 2.02
0.00 0.00 0.00
0.00 86.87 0.00
01:16:54 PM 10
1.00 0.00 6.00
1.00 0.00 1.00
0.00 91.00 122.22
01:16:54 PM 11
0.00 0.00 1.01
0.00 0.00 0.00
0.00 98.99 0.00
01:16:54 PM 12
0.00 0.00 0.00
0.00 0.00 0.00 0.00
100.00 0.00
01:16:54 PM 13
0.00 0.00 0.00
0.00 0.00 0.00 0.00
100.00 0.00
01:16:54 PM 14
0.00 0.00 0.00
0.00 0.00 0.00 0.00
100.00 0.00
01:16:54 PM 15
0.00 0.00 1.01
0.00 0.00 0.00
0.00 98.99 0.00
01:16:54 PM 16
0.00 0.00 0.00
0.00 0.00 0.00 0.00
100.00 0.00
01:16:54 PM 17
0.00 0.00 0.00
0.00 0.00 0.00 0.00
100.00 0.00
01:16:54 PM 18
0.00 0.00 0.00
0.00 0.00 0.00 0.00
100.00 0.00
01:16:54 PM 19 10.20
0.00 1.02 0.00
0.00 0.00 0.00
88.78 0.00
01:16:54 PM 20
0.00 0.00 0.00
0.00 0.00 0.00 0.00
100.00 5.05
01:16:54 PM 21
0.00 0.00 0.00
0.00 0.00 0.00 0.00
100.00 0.00
01:16:54 PM 22
0.00 0.00 0.00
0.00 0.00 0.00 0.00
100.00 0.00
01:16:54 PM 23
0.00 0.00 0.00
0.00 0.00 0.00 0.00
100.00 5.05
Thanks
# mukarram
Mukarram Syed