[syslog-ng] udp tuning - where to look.

[Jim Hendrick]

Thanks everyone for the quick feedback!

I am asking our vmware folks to take a look from the "outside" for things
that might help (we may be having a shared resource issue for example)

I am working on a test environment to look into further tuning with kernel
settings.

I am also looking at application changes (since this is seen on both a
syslog-ng "collector" and a splunk "forwarder" I am looking at the kernel /
OS stuff first).

One possibility is that we could stand up a load balancer in front and
spread the load across multiple boxes.

Thanks - If / when I have any brilliant success I will pass it along :-)

Jim


On Wed, Jan 11, 2017 at 8:06 AM, Clayton Dukes <cdukes at logzilla.net> wrote:

> Jim,
>
> You can try some of the steps outlined here:
>
> http://demo.logzilla.net/help/performance_tuning/udp_buffer_tuning
>
>
>
> Since these are VM's, you may also want to look at the type of NIC you are
> using for the VM, for example:
>
> https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=
> displayKC&externalId=2019944
>
>
>
> And of course, make sure the VMware drivers are installed in the OS.
>
>
>
>
>
> *From: *syslog-ng <syslog-ng-bounces at lists.balabit.hu> on behalf of Jim
> Hendrick <james.r.hendrick at gmail.com>
> *Reply-To: *Syslog-ng users' and developers' mailing list <
> syslog-ng at lists.balabit.hu>
> *Date: *Tuesday, January 10, 2017 at 4:20 PM
> *To: *Syslog-ng users' and developers' mailing list <
> syslog-ng at lists.balabit.hu>
> *Subject: *[syslog-ng] udp tuning - where to look.
>
>
>
> Hi,
>
>
>
>   I am working an issue with UDP drops in a high EPS rate environment
> (several thousand steady) and struggling with making any changes in the
> loss rate. (according to netstat -su | grep error).
>
>
>
>   This is happening on two nodes, (one running an older syslog-ng 3.2 OSE,
> the other running a splunk forwarder) but I don't actually think it is
> making it out of the kernel to either application.
>
>
>
>   On the syslog-ng side syslog-ng-ctl stats shows *no* drops at all.
>
>
>
>   Increasing net.core.rmem_max and so_rcvbuf together all the way to 64
> MB did not seem to make any significant difference.
>
>
>
>   This is a RHEL 6 box with 16 GB and 4 cores (virtual - running in an ESX
> environment)
>
>
>
>   Are there other parameters, things I should be looking at?
>
>
>
> Thanks,
>
>
>
> Jim
>
>
>
>
>
>
>
>
>
>
>
>
>
> ____________________________________________________________
> __________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?
> product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20170111/d072bbc7/attachment-0001.html>