[syslog-ng] Losing to much remote sent logs

[Hendrik Visage]

On Tue, Mar 6, 2012 at 5:20 PM, Christophe Brocas
<christophe.brocas at cnamts.fr> wrote:
> Le 06/03/2012 15:43, Martin Holste a écrit :
>>> For a benchmark, I have stressed (10 000 to 20 000msg/sec) a syslogd server
>>> which transmits all logs it received to a Syslog-NG server over udp. I was able
>>> to reach a score of 90% of lost messages.
>>>
>>> udp is very good way to have problem with your log management solution I think.
>> That doesn't sound right at all.  We get much better performance with
>> UDP: zero drops at around 15k/sec with a lot of bursting to over 20k.
> I was not as clear as required.
>
> On my one year old benchmark, the server sending the logs has a standard syslogd
> daemon and was hosted on a redhat vm.
>
> The sending server sent logs over udp.
>
> The receiving server was also on a redhat server vm but hosting a Syslog-ng
> 3.0.x daemon.
>
> [syslogd | RH on VM]  ---- UDP ------> [syslog-ng 3.0.x | RH on VM]

I believe that was what was mentioned elsewhere/previously that the
VMs might be a concern... especially for UDP.

See, UDP is an *un*reliable protocol, and it means that the buffers
etc. needs to be serviced in time, and there are now buffer/window
changes happening un the fly as with TCP.

> I reached a 90% loss rate for a total of 100000 messages sent, on a 6600msg/sec
> rate.
>
> Everything went well after :
> - installing the 3.0.x Syslog-NG on sending server
> - and using TCP to send logs
>
> Hope it helps
> Christophe
>
> --
> Christophe Brocas
>  CNAMTS/DDSI/MRSSI                     12, allées Haussmann 33300 Bordeaux
>  christophe.brocas at cnamts.fr           3072R/0x0661CBBA
>  fixe +33(0)5.57.85.53.55              mob +33(0)6.77.05.19.01
>
>
>
>
> *****************************************************
> "Le contenu de ce courriel et ses éventuelles pièces jointes sont confidentiels. Ils s'adressent exclusivement à la personne destinataire. Si cet envoi ne vous est pas destiné, ou si vous l'avez reçu par erreur, et afin de ne pas violer le secret des correspondances, vous ne devez pas le transmettre à d'autres personnes ni le reproduire. Merci de le renvoyer à l'émetteur et de le détruire.
>
> Attention : L'organisme de l'émetteur du message ne pourra être tenu responsable de l'altération du présent courriel. Il appartient au destinataire de vérifier que les messages et pièces jointes reçus ne contiennent pas de virus. Les opinions contenues dans ce courriel et ses éventuelles pièces jointes sont celles de l'émetteur. Elles ne reflètent pas la position de l'organisme sauf s'il en est disposé autrement dans le présent courriel."
> ******************************************************
>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>