Syslog-ng fails with more than one program destination
I was testing with 1.3.13 and found it fails with a kill -HUP when more than one "program" destination is used. Syslog-ng never restarts and starts using all the cpu. I tested this on both Redhat 6.1 and FreeBSD 3.3. When I remove all but one program destination, it works fine. It also works fine under syslog-ng 1.2.3. I assume I can have more than one program destination but maybe not. Here is the debug output. It never closes the fd and the cpu goes to about 97% binding fd 3, inetaddr: 0.0.0.0, port: 514 io.c: Preparing fd 3 for reading io.c: listening on fd 4 io.c: Preparing fd 6 for writing io.c: Preparing fd 7 for writing io.c: Preparing fd 8 for writing syslog-ng version 1.3.13 starting io.c: Preparing fd 5 for writing Garbage collecting while idle... Objects alive: 68, garbage collected: 4 Garbage collecting while idle... Objects alive: 68, garbage collected: 14 Garbage collecting while idle... Objects alive: 68, garbage collected: 28 Garbage collecting while idle... Objects alive: 68, garbage collected: 37 SIGHUP received, restarting syslog-ng Marking fd 3 for closing. Marking fd 5 for closing. Marking fd 8 for closing. Here are the files from the Redhat system: syslog-ng.conf ## options { long_hostnames(off); sync(0); log_fifo_size(10); }; source src { unix-stream("/dev/log"); internal(); }; source net { udp(); }; # destination all { file ("/var/log/all" sync(0)); }; destination net { file ("/var/log/net" sync(100)); }; # destination authenfail {program ("/etc/scripts/authenfail.sh"); }; destination authendeny {program ("/etc/scripts/authendeny.sh"); }; destination poolmess {program ("/etc/scripts/poolmess.sh"); }; # filter f_authen { match ("PIX-6-10900"); }; filter f_authenfail { match ("PIX-6-109006"); }; filter f_authendeny { match ("PIX-6-109009"); }; filter f_poolfail { match ("laddr 10.10.01.254"); }; # log { source(src); destination(all); }; log { source(net); destination(net); }; # log { source(net); filter(f_authenfail); destination(authenfail); }; log { source(net); filter(f_authendeny); destination(authendeny); }; log { source(net); filter(f_poolfail); destination(poolmess); }; Here are the scripts: authendeny.sh #!/bin/sh while read LINE; do echo $LINE | mail -s "Athentication Denied" root done authenfail.sh #!/bin/sh while read LINE; do # use awk to get the second field with a ' as a delimiter NAME=`echo $LINE | awk -F\' '{print $2}'` echo $NAME echo $LINE | mail -s "Athentication Failed for $NAME" root done poolmess.sh #!/bin/sh while read LINE; do echo $LINE | mail -s "Possible pool overload" pixadmin done
Hi, On Mon, Jan 31, 2000 at 02:50:13PM -0500, George Madden wrote:
I was testing with 1.3.13 and found it fails with a kill -HUP when more than one "program" destination is used. Syslog-ng never restarts and starts using all the cpu. I tested this on both Redhat 6.1 and FreeBSD 3.3. When I remove all but one program destination, it works fine. It also works fine under syslog-ng 1.2.3. I assume I can have more than one program destination but maybe not.
Thanks for the report. I'll check this out. -- Bazsi PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1 url: http://www.balabit.hu/pgpkey.txt
On Mon, Jan 31, 2000 at 02:50:13PM -0500, George Madden wrote:
I was testing with 1.3.13 and found it fails with a kill -HUP when more than one "program" destination is used. Syslog-ng never restarts and starts using all the cpu. I tested this on both Redhat 6.1 and FreeBSD 3.3. When I remove all but one program destination, it works fine. It also works fine under syslog-ng 1.2.3. I assume I can have more than one program destination but maybe not.
Here is the debug output. It never closes the fd and the cpu goes to about 97%
binding fd 3, inetaddr: 0.0.0.0, port: 514 io.c: Preparing fd 3 for reading io.c: listening on fd 4 io.c: Preparing fd 6 for writing io.c: Preparing fd 7 for writing io.c: Preparing fd 8 for writing syslog-ng version 1.3.13 starting io.c: Preparing fd 5 for writing Garbage collecting while idle... Objects alive: 68, garbage collected: 4 Garbage collecting while idle... Objects alive: 68, garbage collected: 14 Garbage collecting while idle... Objects alive: 68, garbage collected: 28 Garbage collecting while idle... Objects alive: 68, garbage collected: 37 SIGHUP received, restarting syslog-ng Marking fd 3 for closing. Marking fd 5 for closing. Marking fd 8 for closing.
I couldn't reproduce this problem on my Debian Potato system, everything works fine. I'm releasing my current version as 1.3.14, check if the error still occurs. -- Bazsi PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1 url: http://www.balabit.hu/pgpkey.txt
I tried 1.3.14 and it has the same problem. It seems to be load related. The machine I'm running this on receives about 1000 messages a minute. When I stop logging to this computer from the other machines it restarts fine. Is there any other debugging you'd like me to try? -George ----- Original Message ----- From: "Balazs Scheidler" <bazsi@balabit.hu> To: <syslog-ng@venus.terrasoft.hu> Sent: Friday, February 04, 2000 2:01 PM Subject: Re: [syslog-ng] Syslog-ng fails with more than one program destination
On Mon, Jan 31, 2000 at 02:50:13PM -0500, George Madden wrote:
I was testing with 1.3.13 and found it fails with a kill -HUP when more than one "program" destination is used. Syslog-ng never restarts and starts using all the cpu. I tested this on both Redhat 6.1 and FreeBSD 3.3. When I remove all but one program destination, it works fine. It also works fine under syslog-ng 1.2.3. I assume I can have more than one program destination but maybe not.
Here is the debug output. It never closes the fd and the cpu goes to about 97%
binding fd 3, inetaddr: 0.0.0.0, port: 514 io.c: Preparing fd 3 for reading io.c: listening on fd 4 io.c: Preparing fd 6 for writing io.c: Preparing fd 7 for writing io.c: Preparing fd 8 for writing syslog-ng version 1.3.13 starting io.c: Preparing fd 5 for writing Garbage collecting while idle... Objects alive: 68, garbage collected: 4 Garbage collecting while idle... Objects alive: 68, garbage collected: 14 Garbage collecting while idle... Objects alive: 68, garbage collected: 28 Garbage collecting while idle... Objects alive: 68, garbage collected: 37 SIGHUP received, restarting syslog-ng Marking fd 3 for closing. Marking fd 5 for closing. Marking fd 8 for closing.
I couldn't reproduce this problem on my Debian Potato system, everything works fine. I'm releasing my current version as 1.3.14, check if the error
still occurs.
-- Bazsi PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1 url: http://www.balabit.hu/pgpkey.txt
_______________________________________________ syslog-ng maillist - syslog-ng@lists.balabit.hu http://lists.balabit.hu/mailman/listinfo/syslog-ng
On Sun, Feb 06, 2000 at 08:36:23AM -0500, George Madden wrote:
I tried 1.3.14 and it has the same problem. It seems to be load related. The machine I'm running this on receives about 1000 messages a minute. When I stop logging to this computer from the other machines it restarts fine. Is there any other debugging you'd like me to try?
-George
hmmm... I'll try to increase the load. I've just arrived home and catching up messages. (I was skiing in Czech Republic last week :)
----- Original Message ----- From: "Balazs Scheidler" <bazsi@balabit.hu> To: <syslog-ng@venus.terrasoft.hu> Sent: Friday, February 04, 2000 2:01 PM Subject: Re: [syslog-ng] Syslog-ng fails with more than one program destination
On Mon, Jan 31, 2000 at 02:50:13PM -0500, George Madden wrote:
I was testing with 1.3.13 and found it fails with a kill -HUP when more than one "program" destination is used. Syslog-ng never restarts and starts using all the cpu. I tested this on both Redhat 6.1 and FreeBSD 3.3. When I remove all but one program destination, it works fine. It also works fine under syslog-ng 1.2.3. I assume I can have more than one program destination but maybe not.
Here is the debug output. It never closes the fd and the cpu goes to about 97%
binding fd 3, inetaddr: 0.0.0.0, port: 514 io.c: Preparing fd 3 for reading io.c: listening on fd 4 io.c: Preparing fd 6 for writing io.c: Preparing fd 7 for writing io.c: Preparing fd 8 for writing syslog-ng version 1.3.13 starting io.c: Preparing fd 5 for writing Garbage collecting while idle... Objects alive: 68, garbage collected: 4 Garbage collecting while idle... Objects alive: 68, garbage collected: 14 Garbage collecting while idle... Objects alive: 68, garbage collected: 28 Garbage collecting while idle... Objects alive: 68, garbage collected: 37 SIGHUP received, restarting syslog-ng Marking fd 3 for closing. Marking fd 5 for closing. Marking fd 8 for closing.
I couldn't reproduce this problem on my Debian Potato system, everything works fine. I'm releasing my current version as 1.3.14, check if the error still occurs.
-- Bazsi PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1 url: http://www.balabit.hu/pgpkey.txt
I tried 1.3.14 and it has the same problem. It seems to be load related. The machine I'm running this on receives about 1000 messages a minute. When I stop logging to this computer from the other machines it restarts fine. Is there any other debugging you'd like me to try?
-George
hmmm... I'll try to increase the load. I've just arrived home and catching up messages. (I was skiing in Czech Republic last week :)
This patch should fix the problem: --- main.c 1999/12/18 22:25:38 1.22 +++ main.c 2000/02/13 15:07:57 @@ -131,7 +131,7 @@ void sig_child(int signo) { - while (waitpid(-1, NULL, WNOHANG) != -1) + while (waitpid(-1, NULL, WNOHANG) > 0) ; signal(SIGCHLD, sig_child); } -- Bazsi PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1 url: http://www.balabit.hu/pgpkey.txt
Balazs, The patch worked fine. I also tried 1.3.15 and it appears to work fine. I've had it running for about 12 hours now and it appears to be running well. I'd like to let it go for a week or so to test it properly. Thanks. ----- Original Message ----- From: "Balazs Scheidler" <bazsi@balabit.hu> To: <syslog-ng@lists.balabit.hu> Sent: Sunday, February 13, 2000 10:21 AM Subject: Re: [syslog-ng] Syslog-ng fails with more than one program destination
I tried 1.3.14 and it has the same problem. It seems to be load related. The machine I'm running this on receives about 1000 messages a minute. When I stop logging to this computer from the other machines it restarts fine. Is there any other debugging you'd like me to try?
-George
hmmm... I'll try to increase the load. I've just arrived home and catching up messages. (I was skiing in Czech Republic last week :)
This patch should fix the problem:
--- main.c 1999/12/18 22:25:38 1.22 +++ main.c 2000/02/13 15:07:57 @@ -131,7 +131,7 @@
void sig_child(int signo) { - while (waitpid(-1, NULL, WNOHANG) != -1) + while (waitpid(-1, NULL, WNOHANG) > 0) ; signal(SIGCHLD, sig_child); }
-- Bazsi PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1 url: http://www.balabit.hu/pgpkey.txt
_______________________________________________ syslog-ng maillist - syslog-ng@lists.balabit.hu http://lists.balabit.hu/mailman/listinfo/syslog-ng
participants (2)
-
Balazs Scheidler
-
George Madden