The other thing I forgot to mention is that yes it is syslog-ng on the receving end that is dropping packets. We are logging a lot of web data on a loghost. We have no sending syslog-ng process. We have our application that connects directly to the syslog-ng port. It is a java application and uses log4j. We wrote a custom "TCP appender" to connect to port 514 and dump the log messages there. Now it is possible I guess that its dropping messages but unlikely. We can do more investigation in that. -----Original Message----- From: syslog-ng-bounces@lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Balazs Scheidler Sent: July 14, 2006 8:27 AM To: Syslog-ng users' and developers' mailing list Subject: RE: [syslog-ng] Syslog-ng dropped packets. On Fri, 2006-07-14 at 06:21 -0700, Vaibhav Goel wrote:

Hi Bazsi

We are still having issues with dropped packets under heavy load. We increased the size of rmem_default and rmem_max drastically but we are not seeing much of an improvement....Maybe about 50% or so. Currently, the sizes are

$ > cat /proc/sys/net/core/rmem_default 786426 $ > cat /proc/sys/net/core/rmem_max 1572852

Do you have any other suggestions? We are running syslog-ng 1.6.11.

All the logging is done over TCP (except 1 UDP source but that doesn't generate a lot of logging data).

It would be important to know whether it is syslog-ng itself that drops messages or something else in the system. Do you have STATS lines in your logs? That shows how many lines were dropped by syslog-ng itself. It is also possible that it is the sending syslog-ng process that drops messages (in the case of TCP), you can verify this by checking the STATS line of the sending syslog-ng process. You can improve this situation somewhat by increasing log_fifo_size on the sender. 1.6.x sends messages on its TCP destination without flow control, which means that if the bandwidth of a TCP channel is less than the messages received, it could intentionally drop messages, that's what is shown in the DROPPED line. 2.0 supports flow control, which means that syslog-ng is trying to slow its inputs down to the rate of its output pipes. This could slow down applications at the end, but this is the only way to avoid dropping messages. -- Bazsi _______________________________________________ syslog-ng maillist - syslog-ng@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html