[syslog-ng] Support for Open Source Syslog-ng

Sat Apr 14 12:37:51 UTC 2018

Well, some very basic information is missing from your report, like
syslog-ng and OS versions.

The 750 filter rules seems like a lot, however 300 EPS is not that much. If
you are using expensive regexps for filtering that can potentially explain
why so much is dropped on the floor. Also, please look at the UDP
statistics and double-check that we are dropping messages before reception
into the syslog-ng process.

If this is a critical production deployment however, I would seriously
consider the paid for services. The Open Source version of syslog-ng is not
supported by Balabit's professional support team, community is usually
helping on a best effort basis. Results are available as patches against
the latest version, which you would have to compile and deploy, that is
usually not enough to address P1 type of problems.

Anyhow, if you post some more details, we would look into this, as time
allows.

Cheers,
Bazsi

-- 
Bazsi

On Fri, Apr 13, 2018 at 10:50 PM, Schoonover, Mark E HHHH <
Mark.Schoonover at cigna.com> wrote:

> Naveen,
>
>
>
> Need some additional information to help. How many messages per sec are
> arriving at your NG server? Can you post the section of the syslog-ng.conf
> file showing your sources?
>
>
>
> Regards,
>
>
>
> Mark Schoonover – KA6WKE - Infrastructure Engineering Manager
>
> ENE   : Tools, Instrumentation and Common Services Team
>
> Office: 32.8697° N, 116.9711° W - Phone : 770-261-7934 - Email :
> mark.schoonover at cigna.com
>
> *HPSM Team: ENE NMS Engineering*
>
>                   *[image: vet]*
>
>
>
> *Confidential, unpublished property of Cigna. Do not duplicate or
> distribute. Use and distribution limited solely to authorized personnel. ©
> Copyright 2018 Cigna.*
>
>
>
> *From:* syslog-ng [mailto:syslog-ng-bounces at lists.balabit.hu] *On Behalf
> Of *Naveen Bhalla (nbhalla)
> *Sent:* Friday, April 13, 2018 10:43 AM
> *To:* syslog-ng at lists.balabit.hu
> *Subject:* Re: [syslog-ng] Support for Open Source Syslog-ng
>
>
>
> Team,
>
>     Could you pls help us on below P1 situation for us?
>
>
>
>
>
>
>
> Regards,
>
>
>
> [image:
> http://wwwin.cisco.com/c/dam/cec/organizations/gmcc/services-tools/signaturetool/images/logo/logo_gradient.png]
>
> *Naveen Bhalla* | Manager.Technical Support
>
> CMS Platform Operations
>
>
>
> Cell:  +91-9880362157
>
> Desk: +91-80-44260795
>
>
>
> *From:* Naveen Bhalla (nbhalla)
> *Sent:* 13 April 2018 09:14 PM
> *To:* 'support at balabit.com' <support at balabit.com>
> *Subject:* Support for Open Source Syslog-ng
>
>
>
> Hello Support Team,
>
>      We have a situation in our platform where syslog-ng is dropping part
> of syslog traffic coming into our server. The syslog-ng has around 750
> match rules in its configuration. Based on these rules the syslogs are
> forwarded to the destinations. Also, there is one rule to write all the
> received syslogs to disk. We are receiving syslogs at the rate of 300 eps.
>
>
>
> The issue is that we are seeing that syslog-ng is not able to process the
> syslogs and forward them to the destinations. It is not writing to the disk
> also. We are seeing that there is a big delay after which some syslogs are
> getting written to the disk. We are seeing loss of UDP packets. The UDP
> buffer size is big enough.
>
>
>
> net.ipv4.tcp_rmem = 4096 4194304 16777216
>
> net.ipv4.tcp_wmem = 98304 4194304 16777216
>
> net.core.rmem_default = 234217728
>
> net.core.wmem_default = 234217728
>
> net.core.rmem_max =  234217728
>
> net.core.wmem_max = 234217728
>
> net.ipv4.tcp_window_scaling = 1
>
> net.ipv4.ip_local_port_range = 32768 61000
>
> fs.file-max = 2097152
>
> net.core.optmem_max = 40960
>
> net.core.netdev_max_backlog = 50000
>
> net.ipv4.udp_rmem_min = 8192
>
> net.ipv4.udp_wmem_min = 8192
>
>
>
> We need help to resolve this issue.
>
>
>
>
>
> We are using open-source syslog-ng in our setup.
>
>
>
>
>
>
>
> Regards,
>
>
>
> [image:
> http://wwwin.cisco.com/c/dam/cec/organizations/gmcc/services-tools/signaturetool/images/logo/logo_gradient.png]
>
> *Naveen Bhalla* | Manager.Technical Support
>
> CMS Platform Operations
>
>
>
> Cell:  +91-9880362157
>
> Desk: +91-80-44260795
>
>
>
> ------------------------------------------------------------
> ------------------
> CONFIDENTIALITY NOTICE: If you have received this email in error,
> please immediately notify the sender by e-mail at the address shown.
> This email transmission may contain confidential information.  This
> information is intended only for the use of the individual(s) or entity to
> whom it is intended even if addressed incorrectly.  Please delete it from
> your files if you are not the intended recipient.  Thank you for your
> compliance.  Copyright (c) 2018 Cigna
> ============================================================
> ==================
>
> ____________________________________________________________
> __________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?
> product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20180414/0fc84f6a/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image003.png
Type: image/png
Size: 2773 bytes
Desc: not available
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20180414/0fc84f6a/attachment-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.png
Type: image/png
Size: 12152 bytes
Desc: not available
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20180414/0fc84f6a/attachment-0003.png>