RE: [syslog-ng] Syslog-ng dropped packets.
Hi Bazsi We are still having issues with dropped packets under heavy load. We increased the size of rmem_default and rmem_max drastically but we are not seeing much of an improvement....Maybe about 50% or so. Currently, the sizes are $ > cat /proc/sys/net/core/rmem_default 786426 $ > cat /proc/sys/net/core/rmem_max 1572852 Do you have any other suggestions? We are running syslog-ng 1.6.11. All the logging is done over TCP (except 1 UDP source but that doesn't generate a lot of logging data). Thanks, -----Original Message----- From: syslog-ng-bounces@lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Balazs Scheidler Sent: May 26, 2006 2:05 AM To: Syslog-ng users' and developers' mailing list Subject: RE: [syslog-ng] Syslog-ng dropped packets. On Thu, 2006-05-25 at 06:27 -0700, Vaibhav Goel wrote:
We are using 1.6.5 (soon to upgrade to 1.6.11) in a production environment. Is 1.9.11 production ready? I recall a message from you saying that you are looking to release 2.0.0 within a month. I will install 1.9.11 on my QA server and start testing it once we deploy 1.6.11 to prod. I am actually not using /dev/xconsole... I have 2 production loghosts and one of them is more heavily used then the other. I am seeing dropped packets on that one during peak hours. That leads me to believe that syslog-ng. Is there a way to increase the incoming buffer in another way?
Are you using UDP or TCP transport? You could increase the socket receive buffer size. /proc/sys/net/core/rmem_default and /proc/sys/net/core/rmem_max -- Bazsi _______________________________________________ syslog-ng maillist - syslog-ng@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html
On Fri, 2006-07-14 at 06:21 -0700, Vaibhav Goel wrote:
Hi Bazsi
We are still having issues with dropped packets under heavy load. We increased the size of rmem_default and rmem_max drastically but we are not seeing much of an improvement....Maybe about 50% or so. Currently, the sizes are
$ > cat /proc/sys/net/core/rmem_default 786426 $ > cat /proc/sys/net/core/rmem_max 1572852
Do you have any other suggestions? We are running syslog-ng 1.6.11.
All the logging is done over TCP (except 1 UDP source but that doesn't generate a lot of logging data).
It would be important to know whether it is syslog-ng itself that drops messages or something else in the system. Do you have STATS lines in your logs? That shows how many lines were dropped by syslog-ng itself. It is also possible that it is the sending syslog-ng process that drops messages (in the case of TCP), you can verify this by checking the STATS line of the sending syslog-ng process. You can improve this situation somewhat by increasing log_fifo_size on the sender. 1.6.x sends messages on its TCP destination without flow control, which means that if the bandwidth of a TCP channel is less than the messages received, it could intentionally drop messages, that's what is shown in the DROPPED line. 2.0 supports flow control, which means that syslog-ng is trying to slow its inputs down to the rate of its output pipes. This could slow down applications at the end, but this is the only way to avoid dropping messages. -- Bazsi
participants (2)
-
Balazs Scheidler
-
Vaibhav Goel