[syslog-ng]syslog-ng loses messages

[Aaron watkins]

On Wed, 20 Oct 2004 14:45:50 +0100, Watkins, Aaron
<aaron.watkins@newsint.co.uk> wrote:
> 
> Dave Johnson syslog-ng@lists.balabit.hu
> Mon, 11 Oct 2004 14:43:24 -0500
> 
> ------------------------------------------------------------------------
> --------
> 
> The two questions are:
> 1) where in the process do messages 0,1 get lost
> 2) where in the process do messages x get lost when high load
> 

This is exactly the question. Did you manage to set up the test case
on your system. If so, what sort of results did you get?

> In theory, since using TCP this shouldn't be the problem... ?
> 

I am assured that we have a highly reliable network, and as you
mentioned, with TCP it shouldn't be an issue. The fact that it is
consistently two messages that get dropped implies that it is unlikely
to be a network problem. However, maybe it is dropped at the kernel
level?

> *--------------------------*
> * Can we try using "logger" as opposed to "event_pipe" in the while
> loop?
> * Can we `snoop -p -o /tmp/outout port 1515` on server2 and see if the
> packets get sent?
> * If that doesn't work, can we try without the templates?
> * Where are the messages lost?  system1 local files? system2 local
> files?  system2 pipe output?
> * Solarisms?  nscd running?  kernel-revs?
> 
> ----

It is worth noting that the system1 local copy of the events includes
these lost events; it is somewhere in between sys1 and sys2 that it
goes. Unfortunately, we have one of those corporate setups where I
don't have the privileges to be able to run most of the other options,
maybe someone else will have some ideas?


Thanks,

Aaron


> 
> Anybody else have any ideas?
> 
> On Fri, 8 Oct 2004 12:22:45 +0100, Aaron watkins
> <aaron.j.watkins@gmail.com> wrote:
> > Dave,
> >
> > > Dave Johnson syslog-ng@lists.balabit.hu
> > > Thu, 7 Oct 2004 10:47:14 -0500
> > >
> > > We have a couple solaris boxes that do logging with pipes and we log
> a
> > > large amount of data, thats why I'm interested in these tests.  I
> can
> > > see if I can setup a quick testbed next week.
> > >
> > > **********
> > >
> > > Otherwise, I'm wondering if the packets are throttled, what
> happens..  IE:
> > >
> > > ---------------------------
> > > cat > /tmp/smallsleep.c <<ENDHERE
> > > #include <unistd.h>
> > >
> > > main () {
> > > usleep(50000);  /* 20 hz */
> > > }
> > > ENDHERE
> > > gcc -o /tmp/smallsleep /tmp/smallsleep.c
> > > -----------------------------
> > > in the do -> done segment of your for loop add `/tmp/smallsleep`
> > >
> > > How does this impact lost messages?
> > >
> >
> > In the situation where the receiving server is killed while the
> > sending server has stopped sending messages (ie. during the sleep 1
> > command), there is no affect (ie. Message 0 and 1 are still lost).
> >
> > In the situation where the receiving server is killed while the
> > sending server is still sending messages, the throttling you suggested
> > reduces the number of messages that are lost to 2 (similar to the
> > first test).
> >
> > Test 1: Message 12 and 13 lost
> > Test 2: Message 25 and 26 lost
> >
> > It's an improvement, but unfortunately, its still not quite reliable
> > enough and when used in practice, I'm unsure if our events (which will
> > be asynchronous by their very nature) could possibly be throttled...
> >
> > Do you have any more ideas???
> >
> > Thanks,
> >
> >
> > Aaron Watkins
> > _______________________________________________
> > syslog-ng maillist  -  syslog-ng@lists.balabit.hu
> > https://lists.balabit.hu/mailman/listinfo/syslog-ng
> > Frequently asked questions at http://www.campin.net/syslog-ng/faq.html
> >
> >
> 
> The Newspaper Marketing Agency:  Opening Up Newspapers:
> 
> www.nmauk.co.uk
> 
> This e-mail and all attachments are confidential and may be privileged. If you have received this e-mail in error, notify the sender immediately. Do not use, disseminate, store or copy it in any way. Statements or opinions in this e-mail or any attachment are those of the author and are not necessarily agreed or authorised by News International (NI). NI Group may monitor emails sent or received for operational or business reasons as permitted by law. NI Group accepts no liability for viruses introduced by this e-mail or attachments. You should employ virus checking software. News International Limited, 1 Virginia St, London E98 1XY, is the holding company for the News International group and is registered in England No 81701
> 
>