[Bug 159] New: on signal HUP program destinations are not closed
https://bugzilla.balabit.com/show_bug.cgi?id=159 Summary: on signal HUP program destinations are not closed Product: syslog-ng Version: 3.3.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi@balabit.hu ReportedBy: erempel@uvic.ca Type of the Report: --- Estimated Hours: 0.0 In previous versions of syslog-ng, when a HUP signal was sent to syslog-ng it would close all of its destinations and reopen them. In version 3.3.4 (perhaps earlier ones) the destination to programs is not closed, but a new one is opened. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.
Yep, this is effecting ELSA users, as it uses a program() destination. I just found this today as well. One must manually kill any child procs after issue a HUP to load a new config. On Tue, Feb 7, 2012 at 1:58 PM, <bugzilla@bugzilla.balabit.com> wrote:
https://bugzilla.balabit.com/show_bug.cgi?id=159
Summary: on signal HUP program destinations are not closed Product: syslog-ng Version: 3.3.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi@balabit.hu ReportedBy: erempel@uvic.ca Type of the Report: --- Estimated Hours: 0.0
In previous versions of syslog-ng, when a HUP signal was sent to syslog-ng it would close all of its destinations and reopen them.
In version 3.3.4 (perhaps earlier ones) the destination to programs is not closed, but a new one is opened.
-- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes. ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
https://bugzilla.balabit.com/show_bug.cgi?id=159 --- Comment #1 from Balazs Scheidler <bazsi@balabit.hu> 2012-03-11 14:31:50 --- Are you sure it affects 3.3.4? Here's a patch that should've fixed this, but that's already included in 3.3.4: Author: Gergely Nagy <algernon@balabit.hu> 2011-12-02 15:37:34 Committer: Balazs Scheidler <bazsi@balabit.hu> 2011-12-21 13:38:26 Parent: 79fc32703615e045fca4c9aaf85b7ab2915e7ed4 (csvparser: fixed csv-parser() when applied to multiple log paths) Child: 511e939f4199af3ae9aaa4a14b77c7f82863c16f (logsource: fixed potential division by zero exception) Branches: master, pipe-owner-group, remotes/algernon/debian/3.3, remotes/algernon/debian/autobuilt/3.3, remotes/algernon/feature/3.3/ivykis/upstream, remotes/algernon/feature/3.3/mongodb/flat-insert, remotes/algernon/feature/3.4/tf-uuid, remotes/algernon/patched/3.3, remotes/algernon/sandbox/3.4, remotes/algernon/upstream/dfsg/3.3, remotes/algernon/upstream/mirror/3.3, remotes/algernon/upstream/mirror/3.4, remotes/github/master, remotes/github/pipe-owner-group, remotes/origin/master Follows: v3.3.3 Precedes: debian/v3.3.3.dfsg-1_mhp2+g511e939, v3.3.4 afprog: Terminate the child process on deinit. When reloading the configuration, terminate the child process of a program destination, for the following reasons: it will be restarted anyway by afprog_dd_reopen(), so there's no harm done. But if dash is used as /bin/sh, simply closing the stdin of the process will not make it stop. That, in turn, results in us starting the same program over and over again on each SIGHUP. So instead of doing this, send a SIGTERM to the child on deinit, too. Reported-by: Thomas Wollner <tw@wollner-net.de> Signed-off-by: Gergely Nagy <algernon@balabit.hu> -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.
https://bugzilla.balabit.com/show_bug.cgi?id=159 --- Comment #2 from Evan Rempel <erempel@uvic.ca> 2012-03-12 02:48:34 --- Yes, I am sure that it is 3.3.4 I am also sure that the standard input to the program is not closed. Syslog-ng may send a signal, but my application ignores all signals and just reads the input until EOF. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.
https://bugzilla.balabit.com/show_bug.cgi?id=159 Gergely Nagy <algernon@balabit.hu> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |algernon@balabit.hu AssignedTo|bazsi@balabit.hu |algernon@balabit.hu -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.
https://bugzilla.balabit.com/show_bug.cgi?id=159 Gergely Nagy <algernon@balabit.hu> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |ASSIGNED -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.
https://bugzilla.balabit.com/show_bug.cgi?id=159 Gergely Nagy <algernon@balabit.hu> changed: What |Removed |Added ---------------------------------------------------------------------------- Target Milestone|--- |3.3.6 -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.
https://bugzilla.balabit.com/show_bug.cgi?id=159 --- Comment #3 from Gergely Nagy <algernon@balabit.hu> 2012-06-08 16:20:00 --- I think I see what happens. We do restart the program, and that's something I don't think I can change within the scope of 3.3, but, I can make it so that apart from sending a SIGTERM, the program's stdin gets closed too. This way, a new program would start upon SIGHUP, but we'd close the stdin of the original too. There's a small race there, though, between the old shutting down, and the new starting, and there's two ways I can imagine how to handle that: 1) Make the app smart, and wait for the old one to exit. 2) Make syslog-ng wait for the application to quit The second can easily lead to a deadlock, though, so I'd rather not go there. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.
https://bugzilla.balabit.com/show_bug.cgi?id=159 --- Comment #4 from Evan Rempel <erempel@uvic.ca> 2012-06-08 17:22:13 --- I don't think that the race condition is any different than already exists. Currently when sig HUP is sent to syslog-ng, all outputs are changed to be new instances of the outputs. Old files are closed and new files are opened, old sockets are closed and new sockets are opened, old programs are sent a signal (I forget which one) and a new program is started. In none of these cases does starting the new output coordinate with any actions from the olde one. OS disk buffers need to be flushed on file close. TCP buffers at the recipient end need to be consumed and processed, and program destinations need to finish processing what they have buffered. What I am saying is that all programs running as program destinations MUST be written in a way that coordinates between the old closing down instance and the new starting up instance. So I request that you just add the closing of the programs stdin and require/recommend that the authors of the programs make them smart enough to hand thei stop/start conditions. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.
https://bugzilla.balabit.com/show_bug.cgi?id=159 --- Comment #5 from Gergely Nagy <algernon@balabit.hu> 2012-06-08 17:45:37 --- (In reply to comment #4)
I don't think that the race condition is any different than already exists. So I request that you just add the closing of the programs stdin and require/recommend that the authors of the programs make them smart enough to hand thei stop/start conditions.
Mhm. That's what I'll be doing. Currently digging my way through a few layers of code to figure out where and how to add the functionality properly. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.
https://bugzilla.balabit.com/show_bug.cgi?id=159 --- Comment #6 from Gergely Nagy <algernon@balabit.hu> 2012-06-22 12:21:47 --- I believe I have something that works, but the solution isn't as pretty as I would've liked, so it's not on master yet. Meanwhile, can you try this patch: http://git.madhouse-project.org/debian/syslog-ng/patch/?id=a8e8fbef59eeaf5d3... ? (It should cleanly apply to syslog-ng 3.3.5, too) It works for me, all my test programs work: those that ignore signals, get their FDs closed, those that don't, get killed by SIGTERM. But I'd love to have it tested by someone else too, to make sure it works and solves the problem this time, for sure. Thanks in advance! -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.
Yes, that patch fixes the issue for my environment. Thanks, bugzilla@wwwold.balabit.com wrote:
https://bugzilla.balabit.com/show_bug.cgi?id=159
--- Comment #6 from Gergely Nagy <algernon@balabit.hu> 2012-06-22 12:21:47 --- I believe I have something that works, but the solution isn't as pretty as I would've liked, so it's not on master yet. Meanwhile, can you try this patch: http://git.madhouse-project.org/debian/syslog-ng/patch/?id=a8e8fbef59eeaf5d3... ? (It should cleanly apply to syslog-ng 3.3.5, too)
It works for me, all my test programs work: those that ignore signals, get their FDs closed, those that don't, get killed by SIGTERM. But I'd love to have it tested by someone else too, to make sure it works and solves the problem this time, for sure.
Thanks in advance!
-- Evan Rempel erempel@uvic.ca Senior Systems Administrator 250.721.7691 Unix Services, University Systems, University of Victoria
https://bugzilla.balabit.com/show_bug.cgi?id=159 Gergely Nagy <algernon@balabit.hu> changed: What |Removed |Added ---------------------------------------------------------------------------- Resolution| |FIXED Status|ASSIGNED |RESOLVED --- Comment #7 from Gergely Nagy <algernon@balabit.hu> 2012-06-26 15:05:48 --- Fix applied to 3.3's master. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.
participants (3)
-
bugzilla＠bugzilla.balabit.com
-
Evan Rempel
-
Martin Holste