My syslog-ng got into an infinite loop when it didn't have permission to write to a file destination. It tried to log to that same destination the fact that it failed to log. Is there a recommended way to configure syslog-ng to avoid this? Maybe I could route messages from Syslog itself to some relatively reliable destination or just turn them off? If not, what would be a good enhancement to the program to avoid loops? I had the problem with syslog-ng 2.0.2. -- Bryan Henderson San Jose, California
you could have syslog-ng log messages from itself (source internal) to a separate file and use something like logwatch (or other log monitoring programs) to watch for errors in the file and report issues. On Thu, 2007-05-31 at 00:01 +0000, Bryan Henderson wrote:
My syslog-ng got into an infinite loop when it didn't have permission to write to a file destination. It tried to log to that same destination the fact that it failed to log.
Is there a recommended way to configure syslog-ng to avoid this? Maybe I could route messages from Syslog itself to some relatively reliable destination or just turn them off?
If not, what would be a good enhancement to the program to avoid loops?
I had the problem with syslog-ng 2.0.2.
-- Hiren Patel ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ This e-mail and its contents are subject to the Telkom SA Limited e-mail legal notice available at http://www.telkom.co.za/TelkomEMailLegalNotice.PDF ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
On Thu, May 31, 2007 at 08:36:08AM +0200, Hiren Patel wrote:
you could have syslog-ng log messages from itself (source internal) to a separate file and use something like logwatch (or other log monitoring programs) to watch for errors in the file and report issues.
...and if you're worried about that not being able to write to the file, pipe the internal source directly to swatch/SEC/logwatch/whatever and alert from there. Of course if you alert via email and send via a local MTA and your local disk is fried, it still might not get out. Make sure you're protecting against real-world failure scenarios and that you account for them properly. Something like this should work rather reliably: syslog-ng -> pipe into log parser -> remote SMTP or trap or nagios passive alert
On Thu, 2007-05-31 at 00:01 +0000, Bryan Henderson wrote:
My syslog-ng got into an infinite loop when it didn't have permission to write to a file destination. It tried to log to that same destination the fact that it failed to log.
Is there a recommended way to configure syslog-ng to avoid this? Maybe I could route messages from Syslog itself to some relatively reliable destination or just turn them off?
If not, what would be a good enhancement to the program to avoid loops?
I had the problem with syslog-ng 2.0.2.
-- Hiren Patel
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ This e-mail and its contents are subject to the Telkom SA Limited e-mail legal notice available at http://www.telkom.co.za/TelkomEMailLegalNotice.PDF ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ _______________________________________________ syslog-ng maillist - syslog-ng@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html
-- Nate "They [Computers] can rattle off the Manhattan telephone directory unerringly time after time, which no human can do, but they cannot begin to distinguish one face from another, as babies can do." - LEE DEMBART, in New York Times
On Thu, 2007-05-31 at 12:53 -0700, Nate Campi wrote:
On Thu, May 31, 2007 at 08:36:08AM +0200, Hiren Patel wrote:
you could have syslog-ng log messages from itself (source internal) to a separate file and use something like logwatch (or other log monitoring programs) to watch for errors in the file and report issues.
...and if you're worried about that not being able to write to the file, pipe the internal source directly to swatch/SEC/logwatch/whatever and alert from there.
Of course if you alert via email and send via a local MTA and your local disk is fried, it still might not get out. Make sure you're protecting against real-world failure scenarios and that you account for them properly. Something like this should work rather reliably:
syslog-ng -> pipe into log parser -> remote SMTP or trap or nagios passive alert
and you could have the parser emailing you to notify you that everything is okay, that way, when the mailing system for some reason fails, you stand a chance of noticing that it hasn't mailed you an okay notification and investigate.
On Thu, 2007-05-31 at 00:01 +0000, Bryan Henderson wrote:
My syslog-ng got into an infinite loop when it didn't have permission to write to a file destination. It tried to log to that same destination the fact that it failed to log.
Is there a recommended way to configure syslog-ng to avoid this? Maybe I could route messages from Syslog itself to some relatively reliable destination or just turn them off?
If not, what would be a good enhancement to the program to avoid loops?
I had the problem with syslog-ng 2.0.2.
-- Hiren Patel
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ This e-mail and its contents are subject to the Telkom SA Limited e-mail legal notice available at http://www.telkom.co.za/TelkomEMailLegalNotice.PDF ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ _______________________________________________ syslog-ng maillist - syslog-ng@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html
-- Hiren Patel ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ This e-mail and its contents are subject to the Telkom SA Limited e-mail legal notice available at http://www.telkom.co.za/TelkomEMailLegalNotice.PDF ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
I think logging to a pipe is even more likely to fail (and thereby generate a logging loop) than logging to a file, which I already know is too risky. Avoiding loops is much more important than preserving the messages, so I will probably either stop logging of the "internal" source altogether, or route the messages to a "user" destination (i.e. write to a logged-in terminal). The "user" destination seems to be really simple and highly robust. But I might be able to do better with a small enhancement to the code. I'm thinking of something that drops any message generated internally while in the process of logging a message from source 'internal'. Another thing I've considered is a destination option that says, "never mind if it fails." Then in my configuration, I'd route all messages from source 'internal' to such a destination. -- Bryan Henderson San Jose, California
On Fri, 2007-06-01 at 15:38 +0000, Bryan Henderson wrote:
I think logging to a pipe is even more likely to fail (and thereby generate a logging loop) than logging to a file, which I already know is too risky. Avoiding loops is much more important than preserving the messages, so I will probably either stop logging of the "internal" source altogether, or route the messages to a "user" destination (i.e. write to a logged-in terminal). The "user" destination seems to be really simple and highly robust.
But I might be able to do better with a small enhancement to the code. I'm thinking of something that drops any message generated internally while in the process of logging a message from source 'internal'.
That's a good idea. I'll look into implementing this. -- Bazsi
participants (4)
-
Balazs Scheidler
-
bryanh＠giraffe-data.com
-
Hiren Patel
-
Nate Campi