[syslog-ng] Syslog-ng OSE Question.

[Malkin, Vlad]

Balazs,

Thank you very much!

--Vlad

-----Original Message-----
From: syslog-ng-bounces at lists.balabit.hu [mailto:syslog-ng-bounces at lists.balabit.hu] On Behalf Of Balazs Scheidler
Sent: Tuesday, November 19, 2013 5:24 AM
To: Syslog-ng users' and developers' mailing list
Subject: Re: [syslog-ng] Syslog-ng OSE Question.

On Mon, 2013-11-18 at 16:00 +0000, Malkin, Vlad wrote:
> Thank you for responding!
> 
> I'm not as much concerned with the reliability on-the-wire, TCP is good enough to detect most network issues.  There is a loose assumption that if the message gets to the stack on the server side, the server will be able to get the data, i.e. it in itself had not failed.  But leaving all that aside, what I'm really after is when syslog-ng client sends a message and it "believes" that it has been successfully sent, how could a call back to another process be inserted?  Should hard flow control be used between message producer and the syslog-ng, I'm assuming that the client will stop reading from the producer if it is unable to send, correct?
> 
> If we wanted to insert code in syslog-ng to call another process upon successful message send, do you know where that would need to be?

in the in-process acknowledgement path.

log_source_msg_ack() is the final ACK callback that increments the window and awakens the souce if it went to sleep because of the lack of window space.

But calling out to an external process is resource hungry, syslog-ng is able to process hundreds of thousands of messages per second, with a fork/exec like that I'd guess it wouldn't be more than a couple of thousands.

> 
> Thank you!
> --Vlad
> 
> -----Original Message-----
> From: syslog-ng-bounces at lists.balabit.hu 
> [mailto:syslog-ng-bounces at lists.balabit.hu] On Behalf Of Gergely Nagy
> Sent: Monday, November 18, 2013 3:49 AM
> To: Syslog-ng users' and developers' mailing list
> Subject: Re: [syslog-ng] Syslog-ng OSE Question.
> 
> "Malkin, Vlad" <Vlad.Malkin at viasat.com> writes:
> 
> > Is it possible to have syslog-ng client to invoke an external (to 
> > it) software to acknowledge when each individual log statement is 
> > sent to the server?
> 
> No, it's not possible to do that, but read below.
> 
> > Background: logs should not be lost. The system needs to store logs 
> > internally until they are transmitted to the server (use of TCP 
> > transport provides a level of reliable delivery). After each log 
> > statement is transmitted by syslog-ng client, it needs to be removed 
> > from internal store to preserve allocated space.
> 
> This is something that syslog-ng PE can do for you (albeit still not allowing you to use an external program for ACKing) by way of its RLTP protocol and disk buffering.
> 
> Other than that, using something like AMQP with durability and persistance on would be closest to what you want, but that still allows a narrow window where messages can get lost.
> 


______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq