[syslog-ng] bug report

[Balazs Scheidler]

On Fri, 2006-10-06 at 11:07 -0500, Matt Zagrabelny wrote:
> On Fri, 2006-10-06 at 11:13 +0200, Balazs Scheidler wrote:
> > On Thu, 2006-09-21 at 14:29 -0500, Matt Zagrabelny wrote:
> > > On Thu, 2006-09-21 at 20:53 +0200, Sandor Geller wrote:
> > > > Matt Zagrabelny wrote:
> > > > > does anyone on the list know the preferred way of submitting bug reports
> > > > > to the developers?
> > > > 
> > > > Why don't you send your report to the list? 
> > > 
> > > just wanted to make sure the bugreport wouldnt fall on deaf ears. ;)
> > > 
> > > the problem has to do with using the program() destination driver
> > > dropping syslog entries, or at least the program not receiving them. it
> > > may very well be a problem with the program i am using, but i thought i
> > > would pass it on to the experts.
> > > 
> > > what happens is that when the program() is executing a remote command,
> > > for example via ssh, any successive input on STDIN is dropped or not
> > > received or not sent or something. thus to test this phenomenon, the
> > > program() must get more than one line at a time (or perhaps, at least a
> > > second line before the remote program is done executing). exact timing
> > > of the failure hasnt been investigated yet. the test below uses the auth
> > > facilities because establishing an ssh connection sends more than one
> > > message to syslog.
> > 
> > syslog-ng 1.6.x does not ensure that input and output flows are
> > controlled, e.g. it might happen that more messages enter syslog-ng than
> > it is able to send out.
> > 
> > If the internal log message fifo overflows (you can control its size
> > using log_fifo_size) syslog-ng will start dropping messages, and it
> > reports such cases in the "STATS" message it generates every once in a
> > while.
> 
> i am not sure i agree...
> 
> the default setting for 'log_fifo_size' is 100, mine is set for 2048.
> the log path in my problem is getting *two* lines at a time.
> 
> again, the scenario is that the *second* line is always dropped while
> the destination driver's program is executing a remote command (such as
> 'ssh remote_ip sleep 5'). if the destination driver's program is
> executing a local command (such as 'sleep 5') then the second line is
> *not* dropped.
> 
> psedo-code for driver's program:
> 	open file handle
> 	while (input)
> 		write input to file
> 		ssh localhost sleep 5  << causes lines to be dropped
> 
> 		sleep 5 << replace with this line causes lines not to
>                            be dropped
> 	end
> 
> output just generated from destination driver's program:
> 

Hmm... If ssh allocates a pty (which it does not by default for remote
command executions), the input/output processing is asynchronous, and
some data might still be transferred to the server even if the program
there does not really read them (e.g. the pseudo terminal driver
consumes input on the server side into its internal buffer, even if the
program running on that virtual terminal does not read)

I'm not sure a similar thing would happen if a pty is not allocated, but
I would use strace on the ssh process and see what it reads on its
standard input. You could do something like:

ssh localhost sleep 5 < /dev/null

To be sure that ssh does not consume _any_ messages, originally destined
to the script.

-- 
Bazsi