no logging after HUP (1.5.13)
Because of the template feature I want to use syslog-ng 1.5 on a FreeBSD 4.2 and 4.3 platform. While testing 1.5.13 I found a problem when sending a hangup signal to syslog-ng. The restart message is shown but nothing follows until syslog-ng is stopped and started again. My question is how to solve this problem? The following output of the messages file appeared: Oct 30 11:06:49 humble syslog-ng[5018]: syslog-ng version 1.5.13 starting Oct 30 11:08:56 humble su: BAD SU user to root on /dev/ttyp7 Oct 30 11:09:28 humble su: user to root on /dev/ttyp7 Oct 30 11:16:49 humble syslog-ng[5018]: STATS: dropped 0 Oct 30 11:25:09 humble su: BAD SU user to root on /dev/ttyp7 Oct 30 11:26:05 humble su: user to root on /dev/ttyp3 Oct 30 11:26:49 humble syslog-ng[5018]: STATS: dropped 0 Oct 30 11:27:00 humble syslog-ng[5018]: SIGHUP received, restarting syslog-ng Oct 30 11:35:36 humble syslog-ng[5109]: syslog-ng version 1.5.13 starting Oct 30 11:35:49 humble syslog-ng[5109]: SIGHUP received, restarting syslog-ng
Arnold Pulleman writes:
Because of the template feature I want to use syslog-ng 1.5 on a FreeBSD 4.2 and 4.3 platform. While testing 1.5.13 I found a problem when sending a hangup signal to syslog-ng. The restart message is shown but nothing follows until syslog-ng is stopped
On my NetBSD 1.5.x box, it's even worse - syslog-ng 1.5.13 dumps core when it gets HUP'd, due to this assertion: assertion "s->ref" failed: file "center.c", line 129 The core file helpfully has a message on the stack saying it couldn't open /dev/klog, device busy, and I confirmed with ktrace that syslog-ng is closing its other sources in my config file, but not the file("/dev/klog") source. When I take the file("/dev/klog") source out of the config, it doesn't crash anymore. A bug in the class for file() ? Balazs, let me know if you need any more diagnostics.
Ed Ravin writes:
The core file helpfully has a message on the stack saying it couldn't open /dev/klog, device busy, and I confirmed with ktrace that syslog-ng is closing its other sources in my config file, but not the file("/dev/klog") source.
When I take the file("/dev/klog") source out of the config, it doesn't crash anymore. A bug in the class for file() ?
Curious, I'm wandering through the source code and I've found the destroyer routine for the other kinds of log sources (af_inet, af_unix), but not the "file" class. Anybody else know what might be going on in that code?
syslog-ng 1.5.10 died again on me (NetBSD 1.5.1). The core dump has an apparent infinite error loop, which eventually segfaulted. The bottom of the stack is shown below, you'll have to just imagine the other 8979 lines of the stack trace. I realize now that my last bug report was also about 1.5.10, not 1.5.13, so I'm switching to 1.5.13 and we'll see what happens. [...] #8980 0xf9e1 in msg_vformat () #8981 0xfa7d in fatal () #8982 0x104f8 in ol_object_check () #8983 0x2bc3 in do_add_source_name (c=0x23108, logmsg=0x1850c) at sources.c:198 #8984 0xb7d4 in write_internal_message (level=4, length=43, data=0xbfbfc7f0 "ol_object_check: Unexpected marked object!\n") at afinter.c:61 #8985 0xf9e1 in msg_vformat () #8986 0xfa7d in fatal () ---Type <return> to continue, or q <return> to quit--- #8987 0x104f8 in ol_object_check () #8988 0x2bc3 in do_add_source_name (c=0x23108, logmsg=0x1890c) at sources.c:198 #8989 0xb7d4 in write_internal_message (level=4, length=43, data=0xbfbfcc7c "ol_object_check: Unexpected marked object!\n") at afinter.c:61 #8990 0xf9e1 in msg_vformat () #8991 0xfa7d in fatal () #8992 0x104f8 in ol_object_check () #8993 0x2bc3 in do_add_source_name (c=0x23108, logmsg=0x1864c) at sources.c:198 #8994 0xb7d4 in write_internal_message (level=4, length=43, data=0xbfbfd108 "ol_object_check: Unexpected marked object!\n160, port 65520\n@XÑ¿¿xÕ¿¿\2100\002") at afinter.c:61 #8995 0xf9e1 in msg_vformat () #8996 0xfa7d in fatal () #8997 0x104f8 in ol_object_check () #8998 0x2bc3 in do_add_source_name (c=0x23108, logmsg=0x1884c) at sources.c:198 #8999 0xb7d4 in write_internal_message (level=4, length=67, data=0xbfbfd594 "dont_free_live_resource: About to garbage collect a live resource!\n") at afinter.c:61 #9000 0xf9e1 in msg_vformat () #9001 0xfa7d in fatal () #9002 0x10b23 in dont_free_live_resource () #9003 0x10a36 in do_resource_free () ---Type <return> to continue, or q <return> to quit--- #9004 0x1002e in gc_sweep () #9005 0x10117 in gc () #9006 0x101aa in gc_maybe () #9007 0xdde4 in io_iter () #9008 0x1c71 in main_loop (backend=0x18008) at main.c:208 #
No one ever responded to my message about this - am I the only one with this problem? I'll restate the issue: I am setting up syslog-ng-1.5.13 on NetBSD 1.5.1. If I have this statement in the configuration file: source kernel { file("/dev/klog"); }; then when I send syslog-NG a HUP, it dies with a core file. The interesting part of the backtrace is: #0 0x480eff17 in kill () #1 0x480ef0f1 in abort () #2 0x480b2d3a in __assert () #3 0x804adc3 in do_distribute_log (c=0x80649c8, msg=0x8064b0c) at center.c:129 #4 0x804aa75 in do_add_source_name (c=0x8067388, logmsg=0x8064b0c) at sources.c:238 #5 0x8050ac0 in write_internal_message (level=3, length=55, data=0xbfbfd544 "Error opening file /dev/klog for reading (Device busy)\n\bp±\006\bp±\006\bp±\006\bp±\006\bp±\006\bp±\006\bp±\006\bp±\006\bp±\006\bp±\006\b\200±\006\b\200±\006\b\200±\006\b\200±\006\bp±\006\bp±\006\b Â\005\b Â\005\b Â\005\b Â\005\b Â\005\b Â\005\bp±\006\bp±\006\b`Â\005\b`Â\005\bp±\006\bp±\006\b\212\025") at afinter.c:61 I ran syslog-ng under ktrace - sure enough, when it receives the HUP, it closes all its log sources except for /dev/klog, and then it tries to re-open all of them, including /dev/klog. Since it already has /dev/klog open, and NetBSD seems to let only one process access klog at a time, the kernel returns "device busy" and that provokes the core dump. Browsing around some more in the code, I see that afinet.c has methods to close a connection, but affile.c does not even have any appearances of the word "close". I though about fixing it myself, but it looks like I'd have to add callbacks and some other fun stuff into the class for the log source driver, and it's a bit above my head at the moment. This is a shame because on the machines I switched to syslog-ng, I can't log kernel errors anymore (unless I want to do some horrid thing like run regular syslogd as well to feed the messages into syslog-ng). Is anyone else using syslog-ng using the file() source? If so, what happens if you HUP the daemon? If it doesn't crash, I'll bet that at the very least it starts a file descriptor leak (which you could confirm by running lsof before and after the HUP). -- Ed Ed Ravin writes:
On my NetBSD 1.5.x box, it's even worse - syslog-ng 1.5.13 dumps core when it gets HUP'd, due to this assertion:
assertion "s->ref" failed: file "center.c", line 129
The core file helpfully has a message on the stack saying it couldn't open /dev/klog, device busy, and I confirmed with ktrace that syslog-ng is closing its other sources in my config file, but not the file("/dev/klog") source.
When I take the file("/dev/klog") source out of the config, it doesn't crash anymore. A bug in the class for file() ?
On Fri, Jan 04, 2002 at 07:57:25PM -0500, Ed Ravin wrote:
No one ever responded to my message about this - am I the only one with this problem? I'll restate the issue:
I am setting up syslog-ng-1.5.13 on NetBSD 1.5.1. If I have this statement in the configuration file:
source kernel { file("/dev/klog"); };
then when I send syslog-NG a HUP, it dies with a core file. The interesting part of the backtrace is:
Thanks for the report, I'll look into it. -- Bazsi PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1
so, did anyone got this solved? i m running 1.4.14 on a FreeBSD 4.3. after newsyslog does it's hourly cleanup, it will do a SIGHUP to the syslog. after that, syslog fails to log anything. any help? benny On Tue, Oct 30, 2001 at 12:06:09PM +0100, Arnold Pulleman wrote: | Because of the template feature I want to use syslog-ng 1.5 on a FreeBSD 4.2 and 4.3 platform. While testing 1.5.13 I found a problem when sending a hangup signal to syslog-ng. The restart message is shown but nothing follows until syslog-ng is stopped and started again. My question is how to solve this problem? | | The following output of the messages file appeared: | Oct 30 11:06:49 humble syslog-ng[5018]: syslog-ng version 1.5.13 starting | Oct 30 11:08:56 humble su: BAD SU user to root on /dev/ttyp7 | Oct 30 11:09:28 humble su: user to root on /dev/ttyp7 | Oct 30 11:16:49 humble syslog-ng[5018]: STATS: dropped 0 | Oct 30 11:25:09 humble su: BAD SU user to root on /dev/ttyp7 | Oct 30 11:26:05 humble su: user to root on /dev/ttyp3 | Oct 30 11:26:49 humble syslog-ng[5018]: STATS: dropped 0 | Oct 30 11:27:00 humble syslog-ng[5018]: SIGHUP received, restarting syslog-ng | Oct 30 11:35:36 humble syslog-ng[5109]: syslog-ng version 1.5.13 starting | Oct 30 11:35:49 humble syslog-ng[5109]: SIGHUP received, restarting syslog-ng | | | | | _______________________________________________ | syslog-ng maillist - syslog-ng@lists.balabit.hu | https://lists.balabit.hu/mailman/listinfo/syslog-ng -- I don't want Perl to be beautiful, I want you to write beautiful programs in Perl. --Larry Wall, Culture of Perl, August 1997
participants (4)
-
Arnold Pulleman
-
Balazs Scheidler
-
Benny Chee
-
Ed Ravin