syslg-ng hangs on damnsmalllinux boot
Hello, i try to run syslog-ng 2.0.0 compiled with eventlog-0.25 and glib-2.12.4 on damnsmalllinux 3.0.1 (knoppix variant based on linux 2.4.26). It is started in a script like this which is called from bootlocal.sh sudo /opt/syslog-ng/sbin/syslog-ng \ --cfgfile=/opt/syslog-ng/etc/syslog-ng.conf \ --chroot=/data/var/log \ --user=65534 \ --group=65534 unfortunately it will hang causing mischief, like programs blocking on it, so the boot script does not complete and no login is possible at all. it can only be killed with "-9". The "messages" and "syslog" logfiles look like this, so the chroot etc has worked correctly, the config is valid etc etc: Nov 22 00:19:22 s_all@box syslog-ng[682]: syslog-ng starting up; version='2.0.0' Nov 22 00:19:22 s_all@box kernel: Linux version 2.4.26 (root@Knoppix) (gcc versi on 2.95.4 20011002 (Debian prerelease)) #1 SMP Sa Apr 17 19:33:42 CEST 2004 [blaaaaaaaaaaaaah] Nov 22 00:19:22 s_all@box kernel: Linux agpgart interface v0.99 (c) Jeff Hartmann But hereafter nothing happens anymore :-((( If it is started manually later on it seems to work. I tried starting it with --debug, but that didn't say anything at all, although i ran configure with --enable-debug (is that relevant?) I applied the patch from http://www.mail-archive.com/debian-bugs-dist@lists.debian.org/msg265412.html to prevent blocking on tty's but that didn't help. In the config file i then set use_dns(no) and commented out the usertty() destination, so it is only logging to files. no external programs, pipes etc are used. the only source used is this: source s_all { # message generated by Syslog-NG internal(); # standard Linux log source (this is the default place for the syslog() # function to send logs to) unix-dgram("/dev/log"); unix-dgram("/data/tftpboot/dev/log"); # messages from the kernel file("/proc/kmsg" log_prefix("kernel: ")); # use the next line if you want to receive remote UDP logging messages # (this is equivalent to the "-r" syslogd flag) udp(); tcp(); }; any help would be appreciated!!! regards, -- Heiko@Blume.AG
On Tue, 2006-11-21 at 23:27 +0100, Heiko Blume wrote:
Hello,
i try to run syslog-ng 2.0.0 compiled with eventlog-0.25 and glib-2.12.4 on damnsmalllinux 3.0.1 (knoppix variant based on linux 2.4.26).
It is started in a script like this which is called from bootlocal.sh
sudo /opt/syslog-ng/sbin/syslog-ng \ --cfgfile=/opt/syslog-ng/etc/syslog-ng.conf \ --chroot=/data/var/log \ --user=65534 \ --group=65534
unfortunately it will hang causing mischief, like programs blocking on it, so the boot script does not complete and no login is possible at all. it can only be killed with "-9".
You don't have two /proc/kmsg sources or klogd and syslog-ng running in parallel, do you? -- Bazsi
Hello, telling from the process table there are no other logd's (see below). But i have to admit that i do not know how to check for the "/proc/kmsg" sources you mention. i haven't been into logging like this before :-) however, removing the file("/proc/kmsg" log_prefix("kernel: ")); from the source has fixed the hanging problem. but why? i certainly don't want to miss any messages... thanks & regards, heiko root 1 0 0 00:22 ? 00:00:04 init [2] root 2 1 0 00:22 ? 00:00:00 [keventd] root 3 1 0 00:22 ? 00:00:00 [ksoftirqd_CPU0] root 4 1 0 00:22 ? 00:00:00 [kswapd] root 5 1 0 00:22 ? 00:00:00 [bdflush] root 6 1 0 00:22 ? 00:00:00 [kupdated] root 12 1 0 00:22 ? 00:00:00 [khubd] root 688 1 0 00:23 ? 00:00:00 /usr/sbin/sshd root 1065 1 0 00:23 ? 00:00:00 /opt/fcron/sbin/fcron -b root 1072 1 0 00:23 tty1 00:00:00 -bash root 1073 1 0 00:23 tty2 00:00:00 /sbin/getty 115200 tty2 root 1074 1 0 00:23 tty3 00:00:00 /sbin/getty 115200 tty3 root 1075 1 0 00:23 tty4 00:00:00 /sbin/getty 115200 tty4 root 1103 688 0 00:24 ? 00:00:00 sshd: nx [priv] root 1124 688 0 00:24 ? 00:00:00 sshd: dsl [priv] root 1279 1276 0 00:24 ttyp0 00:00:00 bash nobody 1294 1 0 12:46 ? 00:00:00 /opt/syslog-ng/sbin/syslog-ng --cfgfile=/opt/syslog-ng/etc/syslog-ng.conf --chroot=/data/var/log --user=65534 --group=65534
On Tue, 2006-11-21 at 23:27 +0100, Heiko Blume wrote:
Hello,
i try to run syslog-ng 2.0.0 compiled with eventlog-0.25 and glib-2.12.4 on damnsmalllinux 3.0.1 (knoppix variant based on linux 2.4.26).
It is started in a script like this which is called from bootlocal.sh
sudo /opt/syslog-ng/sbin/syslog-ng \ --cfgfile=/opt/syslog-ng/etc/syslog-ng.conf \ --chroot=/data/var/log \ --user=65534 \ --group=65534
unfortunately it will hang causing mischief, like programs blocking on it, so the boot script does not complete and no login is possible at all. it can only be killed with "-9".
You don't have two /proc/kmsg sources or klogd and syslog-ng running in parallel, do you?
-- Bazsi
_______________________________________________ syslog-ng maillist - syslog-ng@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html
-- Heiko@Blume.AG Cisco Certified Network Professional Cisco Certified Design Professional Juniper Certified Internet Specialist SUN Certified System Administrator Office: +49.30/4426309 FAX: +49.30/48494354 Mobile: +49.178/6662342 www: http://www.blume.ag/IT/ PHY: Knaackstrasse 6, 10405 Berlin, DE
On Wed, 2006-11-22 at 12:53 +0100, Heiko Blume wrote:
Hello,
telling from the process table there are no other logd's (see below). But i have to admit that i do not know how to check for the "/proc/kmsg" sources you mention. i haven't been into logging like this before :-)
however, removing the
file("/proc/kmsg" log_prefix("kernel: "));
from the source has fixed the hanging problem. but why? i certainly don't want to miss any messages...
something else is reading /proc/kmsg in addition to syslog-ng, and in this case, the one losing the race condition blocks, until the next kernel message. -- Bazsi
hello, thank you very much for your help! no more hangs... i couldn't figure out what else may be listening, though. regards, heiko
On Wed, 2006-11-22 at 12:53 +0100, Heiko Blume wrote:
Hello,
telling from the process table there are no other logd's (see below). But i have to admit that i do not know how to check for the "/proc/kmsg" sources you mention. i haven't been into logging like this before :-)
however, removing the
file("/proc/kmsg" log_prefix("kernel: "));
from the source has fixed the hanging problem. but why? i certainly don't want to miss any messages...
something else is reading /proc/kmsg in addition to syslog-ng, and in this case, the one losing the race condition blocks, until the next kernel message.
-- Bazsi
_______________________________________________ syslog-ng maillist - syslog-ng@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html
-- Heiko@Blume.AG Cisco Certified Network Professional Cisco Certified Design Professional Juniper Certified Internet Specialist SUN Certified System Administrator Office: +49.30/4426309 FAX: +49.30/48494354 Mobile: +49.178/6662342 www: http://www.blume.ag/IT/ PHY: Knaackstrasse 6, 10405 Berlin, DE
hello, i ran a lsof right before starting syslog-ng in the boot script, but i can't find a reference to /proc/kmsg, output below. the same after boot has completed. if i run syslog-ng with /pro/kmsg enables as a source lsof does show it, so lsof is working... so it looks like some it only reading /proc/kmsg once/shortly and terminates afterwards!? regards, heiko COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME init 1 root cwd DIR 1,0 1024 2 / init 1 root rtd DIR 1,0 1024 2 / init 1 root txt REG 1,0 46840 2501 /etc/init init 1 root mem REG 1,0 837 421 /etc/localtime init 1 root 10u FIFO 1,0 173 /dev/initctl keventd 2 root cwd DIR 1,0 1024 2 / keventd 2 root rtd DIR 1,0 1024 2 / keventd 2 root txt unknown /proc/2/exe ksoftirqd 3 root cwd DIR 1,0 1024 2 / ksoftirqd 3 root rtd DIR 1,0 1024 2 / ksoftirqd 3 root txt unknown /proc/3/exe kswapd 4 root cwd DIR 1,0 1024 2 / kswapd 4 root rtd DIR 1,0 1024 2 / kswapd 4 root txt unknown /proc/4/exe bdflush 5 root cwd DIR 1,0 1024 2 / bdflush 5 root rtd DIR 1,0 1024 2 / bdflush 5 root txt unknown /proc/5/exe kupdated 6 root cwd DIR 1,0 1024 2 / kupdated 6 root rtd DIR 1,0 1024 2 / kupdated 6 root txt unknown /proc/6/exe khubd 12 root cwd DIR 1,0 1024 2 / khubd 12 root rtd DIR 1,0 1024 2 / khubd 12 root txt unknown /proc/12/exe rc 623 root cwd DIR 1,0 1024 2 / rc 623 root rtd DIR 1,0 1024 2 / rc 623 root txt REG 240,0 690668 238140 /KNOPPIX/bin/bash rc 623 root mem REG 240,0 92174 1714634 /KNOPPIX/lib/ld-2.3.2.so rc 623 root mem REG 240,0 238192 1720746 /KNOPPIX/lib/libncurses.so.5.3 rc 623 root mem REG 240,0 9796 1719266 /KNOPPIX/lib/libdl-2.3.2.so rc 623 root mem REG 240,0 1230864 1716224 /KNOPPIX/lib/libc-2.3.2.so rc 623 root mem REG 240,0 27252 1721344 /KNOPPIX/lib/libnss_compat-2.3.2.so rc 623 root mem REG 240,0 73260 1721036 /KNOPPIX/lib/libnsl-2.3.2.so rc 623 root mem REG 240,0 32304 1722680 /KNOPPIX/lib/libnss_nis-2.3.2.so rc 623 root mem REG 240,0 34440 1721968 /KNOPPIX/lib/libnss_files-2.3.2.so rc 623 root 0u CHR 4,0 16 /dev/console rc 623 root 1u CHR 4,0 16 /dev/console rc 623 root 2u CHR 4,0 16 /dev/console rc 623 root 255r REG 240,0 1723 1076588 /KNOPPIX/etc/init.d/rc S99bootlo 658 root cwd DIR 1,0 1024 2 / S99bootlo 658 root rtd DIR 1,0 1024 2 / S99bootlo 658 root txt REG 240,0 690668 238140 /KNOPPIX/bin/bash S99bootlo 658 root mem REG 240,0 92174 1714634 /KNOPPIX/lib/ld-2.3.2.so S99bootlo 658 root mem REG 240,0 238192 1720746 /KNOPPIX/lib/libncurses.so.5.3 S99bootlo 658 root mem REG 240,0 9796 1719266 /KNOPPIX/lib/libdl-2.3.2.so S99bootlo 658 root mem REG 240,0 1230864 1716224 /KNOPPIX/lib/libc-2.3.2.so S99bootlo 658 root mem REG 240,0 27252 1721344 /KNOPPIX/lib/libnss_compat-2.3.2.so S99bootlo 658 root mem REG 240,0 73260 1721036 /KNOPPIX/lib/libnsl-2.3.2.so S99bootlo 658 root mem REG 240,0 32304 1722680 /KNOPPIX/lib/libnss_nis-2.3.2.so S99bootlo 658 root mem REG 240,0 34440 1721968 /KNOPPIX/lib/libnss_files-2.3.2.so S99bootlo 658 root 0u CHR 4,0 16 /dev/console S99bootlo 658 root 1u CHR 4,0 16 /dev/console S99bootlo 658 root 2u CHR 4,0 16 /dev/console S99bootlo 658 root 255r REG 0,7 3831 3039 /ramdisk/opt/bootlocal.sh ntpd 681 root cwd DIR 1,0 1024 2 / ntpd 681 root rtd DIR 1,0 1024 2 / ntpd 681 root txt REG 0,7 35580 3311 /ramdisk/opt/openntpd/sbin/ntpd ntpd 681 root mem REG 240,0 92174 1714634 /KNOPPIX/lib/ld-2.3.2.so ntpd 681 root mem REG 240,0 1230864 1716224 /KNOPPIX/lib/libc-2.3.2.so ntpd 681 root mem REG 240,0 27252 1721344 /KNOPPIX/lib/libnss_compat-2.3.2.so ntpd 681 root mem REG 240,0 73260 1721036 /KNOPPIX/lib/libnsl-2.3.2.so ntpd 681 root mem REG 240,0 32304 1722680 /KNOPPIX/lib/libnss_nis-2.3.2.so ntpd 681 root mem REG 240,0 34440 1721968 /KNOPPIX/lib/libnss_files-2.3.2.so ntpd 681 root 0r CHR 1,3 214 /dev/null ntpd 681 root 1u CHR 4,0 16 /dev/console ntpd 681 root 2u CHR 4,0 16 /dev/console ntpd 681 root 3r CHR 1,9 412 /dev/urandom ntpd 681 root 4u unix 0xdf491a90 15111 socket ntpd 684 _ntp cwd DIR 0,7 40 3301 /ramdisk/var/empty ntpd 684 _ntp rtd DIR 0,7 40 3301 /ramdisk/var/empty ntpd 684 _ntp txt REG 0,7 35580 3311 /ramdisk/opt/openntpd/sbin/ntpd ntpd 684 _ntp mem REG 240,0 92174 1714634 /KNOPPIX/lib/ld-2.3.2.so ntpd 684 _ntp mem REG 240,0 1230864 1716224 /KNOPPIX/lib/libc-2.3.2.so ntpd 684 _ntp mem REG 240,0 27252 1721344 /KNOPPIX/lib/libnss_compat-2.3.2.so ntpd 684 _ntp mem REG 240,0 73260 1721036 /KNOPPIX/lib/libnsl-2.3.2.so ntpd 684 _ntp mem REG 240,0 32304 1722680 /KNOPPIX/lib/libnss_nis-2.3.2.so ntpd 684 _ntp mem REG 240,0 34440 1721968 /KNOPPIX/lib/libnss_files-2.3.2.so ntpd 684 _ntp 0u CHR 1,3 214 /dev/null ntpd 684 _ntp 1u CHR 1,3 214 /dev/null ntpd 684 _ntp 2u CHR 1,3 214 /dev/null ntpd 684 _ntp 3r CHR 1,9 412 /dev/urandom ntpd 684 _ntp 4u IPv4 15115 UDP 10.24.254.40:32768->10.27.1.110:ntp ntpd 684 _ntp 5u unix 0xdff0d060 15112 socket ntpd 684 _ntp 6u IPv4 15116 UDP 10.24.254.40:32769->10.27.1.111:ntp lsof 695 root cwd DIR 1,0 1024 2 / lsof 695 root rtd DIR 1,0 1024 2 / lsof 695 root txt REG 8,1 120697 35485 /data/bin/lsof lsof 695 root mem REG 240,0 92174 1714634 /KNOPPIX/lib/ld-2.3.2.so lsof 695 root mem REG 240,0 1230864 1716224 /KNOPPIX/lib/libc-2.3.2.so lsof 695 root 0u CHR 4,0 16 /dev/console lsof 695 root 1w REG 8,1 0 14 /data/lsof.out lsof 695 root 2u CHR 4,0 16 /dev/console lsof 695 root 3r DIR 0,2 0 1 /proc lsof 695 root 4r DIR 0,2 0 45547528 /proc/695/fd lsof 695 root 5w FIFO 0,5 15509 pipe lsof 695 root 6r FIFO 0,5 15510 pipe lsof 696 root cwd DIR 1,0 1024 2 / lsof 696 root rtd DIR 1,0 1024 2 / lsof 696 root txt REG 8,1 120697 35485 /data/bin/lsof lsof 696 root mem REG 240,0 92174 1714634 /KNOPPIX/lib/ld-2.3.2.so lsof 696 root mem REG 240,0 1230864 1716224 /KNOPPIX/lib/libc-2.3.2.so lsof 696 root 4r FIFO 0,5 15509 pipe lsof 696 root 7w FIFO 0,5 15510 pipe
On Wed, 2006-11-22 at 12:53 +0100, Heiko Blume wrote:
Hello,
telling from the process table there are no other logd's (see below). But i have to admit that i do not know how to check for the "/proc/kmsg" sources you mention. i haven't been into logging like this before :-)
however, removing the
file("/proc/kmsg" log_prefix("kernel: "));
from the source has fixed the hanging problem. but why? i certainly don't want to miss any messages...
something else is reading /proc/kmsg in addition to syslog-ng, and in this case, the one losing the race condition blocks, until the next kernel message.
-- Bazsi
_______________________________________________ syslog-ng maillist - syslog-ng@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html
-- Heiko@Blume.AG Cisco Certified Network Professional Cisco Certified Design Professional Juniper Certified Internet Specialist SUN Certified System Administrator Office: +49.30/4426309 FAX: +49.30/48494354 Mobile: +49.178/6662342 www: http://www.blume.ag/IT/ PHY: Knaackstrasse 6, 10405 Berlin, DE
On Thu, 2006-11-23 at 12:49 +0100, Heiko Blume wrote:
hello,
i ran a lsof right before starting syslog-ng in the boot script, but i can't find a reference to /proc/kmsg, output below. the same after boot has completed. if i run syslog-ng with /pro/kmsg enables as a source lsof does show it, so lsof is working...
so it looks like some it only reading /proc/kmsg once/shortly and terminates afterwards!?
it is possible, dmesg for instance might cause the kernel ring buffer to be emptied. -- Bazsi
aaah! i think i found it: bash-2.05b# grep dmesg /etc/init.d/* /etc/init.d/bootmisc.sh:dmesg > /var/log/dmesg i'll remove that and try again. thanks again for your help! regards, heiko
On Thu, 2006-11-23 at 12:49 +0100, Heiko Blume wrote:
hello,
i ran a lsof right before starting syslog-ng in the boot script, but i can't find a reference to /proc/kmsg, output below. the same after boot has completed. if i run syslog-ng with /pro/kmsg enables as a source lsof does show it, so lsof is working...
so it looks like some it only reading /proc/kmsg once/shortly and terminates afterwards!?
it is possible, dmesg for instance might cause the kernel ring buffer to be emptied.
-- Bazsi
_______________________________________________ syslog-ng maillist - syslog-ng@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html
-- Heiko@Blume.AG Cisco Certified Network Professional Cisco Certified Design Professional Juniper Certified Internet Specialist SUN Certified System Administrator Office: +49.30/4426309 FAX: +49.30/48494354 Mobile: +49.178/6662342 www: http://www.blume.ag/IT/ PHY: Knaackstrasse 6, 10405 Berlin, DE
On Thu, 2006-11-23 at 15:31 +0100, Heiko Blume wrote:
aaah!
i think i found it:
bash-2.05b# grep dmesg /etc/init.d/* /etc/init.d/bootmisc.sh:dmesg > /var/log/dmesg
i'll remove that and try again.
You should move that before starting syslog-ng. -- Bazsi
participants (2)
-
Balazs Scheidler
-
Heiko Blume