Hi, I am looking to manage my output buffers. I have one destination that sometimes has trouble and I want to be able to manage the messages that got buffered while I fix my flaky destination. Is there any tool I can use to manage those buffered messages? If not, isn't there any exception handling that I can apply to a log such that errors or other failures for that log can be redirected to another destination? I want to avoid clogging up my output queue and especially to avoid losing any messages. Thanks! -eric This email message is intended for the use of the person to whom it has been sent, and may contain information that is confidential or legally protected. If you are not the intended recipient or have received this message in error, you are not authorized to copy, distribute, or otherwise use this message or its attachments. Please notify the sender immediately by return e-mail and permanently delete this message and any attachments. Verio, Inc. makes no warranty that this email is error or virus free. Thank you.
On Fri, 2010-03-05 at 17:49 -0500, Eric Snow wrote:
Hi,
I am looking to manage my output buffers. I have one destination that sometimes has trouble and I want to be able to manage the messages that got buffered while I fix my flaky destination. Is there any tool I can use to manage those buffered messages? If not, isn’t there any exception handling that I can apply to a log such that errors or other failures for that log can be redirected to another destination? I want to avoid clogging up my output queue and especially to avoid losing any messages. Thanks!
you mean you have disk based buffering? in the upcoming 3.1 version (due to be released within 2 weeks), there's a tool called dqtool to dump the contents of the queue file, although I'm not sure that's what you meant. there are further disk buffer related improvements in the queue for 3.2, and also a way to send the messages to a failed destination to an alternate server. also, syslog-ng is counting the number of messages in a given queue, using its statistics framework. it is the stored counter you can see for a given destination. this gets reported in the 'log statistics' message, or via the syslog-ng control socket (/opt/syslog-ng/var/syslog-ng.ctl). -- Bazsi
If disk based buffering allows me to yank the message at the front of the queue, so that it doesn't hold up the ones behind it, then yes that is what I want. The statistics and ability to look at the queue are good, but I need to be able to manipulate the queue. OR I need to be able to have some form of exception handling in my syslog-ng configuration. Something that will catch a failure in a log and allow me to handle that failure intelligently. -eric -----Original Message----- From: syslog-ng-bounces@lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Balazs Scheidler Sent: Saturday, March 06, 2010 2:09 AM To: Syslog-ng users' and developers' mailing list Subject: Re: [syslog-ng] queue management On Fri, 2010-03-05 at 17:49 -0500, Eric Snow wrote:
Hi,
I am looking to manage my output buffers. I have one destination that sometimes has trouble and I want to be able to manage the messages that got buffered while I fix my flaky destination. Is there any tool I can use to manage those buffered messages? If not, isn’t there any exception handling that I can apply to a log such that errors or other failures for that log can be redirected to another destination? I want to avoid clogging up my output queue and especially to avoid losing any messages. Thanks!
you mean you have disk based buffering? in the upcoming 3.1 version (due to be released within 2 weeks), there's a tool called dqtool to dump the contents of the queue file, although I'm not sure that's what you meant. there are further disk buffer related improvements in the queue for 3.2, and also a way to send the messages to a failed destination to an alternate server. also, syslog-ng is counting the number of messages in a given queue, using its statistics framework. it is the stored counter you can see for a given destination. this gets reported in the 'log statistics' message, or via the syslog-ng control socket (/opt/syslog-ng/var/syslog-ng.ctl). -- Bazsi ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html This email message is intended for the use of the person to whom it has been sent, and may contain information that is confidential or legally protected. If you are not the intended recipient or have received this message in error, you are not authorized to copy, distribute, or otherwise use this message or its attachments. Please notify the sender immediately by return e-mail and permanently delete this message and any attachments. Verio, Inc. makes no warranty that this email is error or virus free. Thank you.
Hi Eric, On Mon, 2010-03-08 at 12:12 -0500, Eric Snow wrote:
If disk based buffering allows me to yank the message at the front of the queue, so that it doesn't hold up the ones behind it, then yes that is what I want. The statistics and ability to look at the queue are good, but I need to be able to manipulate the queue.
I'm writing up this as a feature request to our current syslog-ng backlog. So far I've ended up with this short description: * explicitly remove items from the beginning of a queue (cancel messages) via some kind of query * stall and restart queueing on a given destination * administrative stop of a given destination while messages build up * administrative start of a given destination * use case: process spooled messages only during a configured timeframe Does the above cover your requirements? Do you have anything else to add?
OR
I need to be able to have some form of exception handling in my syslog-ng configuration. Something that will catch a failure in a log and allow me to handle that failure intelligently.
Your exact use case would help here as well. Thanks in advance. -- Bazsi
Bazsi, That is spot on. Thanks so much for taking the time on this. A tool like this would be very helpful! -eric -----Original Message----- From: syslog-ng-bounces@lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Balazs Scheidler Sent: Monday, March 29, 2010 4:48 AM To: Syslog-ng users' and developers' mailing list Subject: Re: [syslog-ng] queue management Hi Eric, On Mon, 2010-03-08 at 12:12 -0500, Eric Snow wrote:
If disk based buffering allows me to yank the message at the front of the queue, so that it doesn't hold up the ones behind it, then yes that is what I want. The statistics and ability to look at the queue are good, but I need to be able to manipulate the queue.
I'm writing up this as a feature request to our current syslog-ng backlog. So far I've ended up with this short description: * explicitly remove items from the beginning of a queue (cancel messages) via some kind of query * stall and restart queueing on a given destination * administrative stop of a given destination while messages build up * administrative start of a given destination * use case: process spooled messages only during a configured timeframe Does the above cover your requirements? Do you have anything else to add?
OR
I need to be able to have some form of exception handling in my syslog-ng configuration. Something that will catch a failure in a log and allow me to handle that failure intelligently.
Your exact use case would help here as well. Thanks in advance. -- Bazsi ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html This email message is intended for the use of the person to whom it has been sent, and may contain information that is confidential or legally protected. If you are not the intended recipient or have received this message in error, you are not authorized to copy, distribute, or otherwise use this message or its attachments. Please notify the sender immediately by return e-mail and permanently delete this message and any attachments. Verio, Inc. makes no warranty that this email is error or virus free. Thank you.
participants (2)
-
Balazs Scheidler
-
Eric Snow