On Thu, 2011-03-10 at 09:21 +0100, Zoltán Pallagi wrote:
Hi,
First of all, it seems that syslog-ng receives logs faster than your mssql server can process them that's the reason for dropping.
However, you are using udp source for receiving logs, it's not the best solution if you don't want to lose logs (UDP is not a lossless protocol and perhaps the kernel will also drop the unprocessed messages). You should use TCP instead of UDP.
If you use TCP, you can use flags(flow-control) in your server configuration. If the senders are also syslog-ng, you can use it on their configurations, too. flow-control will slow down (or block) receiving logs if syslog-ng cannot process (write out, forward and so on) the messages in time. It can prevent losing logs.
For example: log { source(s_gms); flags(flow-control); destination(d_mssql); };
Also, by increasing the buffer size, you may be able to process peaks, if otherwise your SQL server is fast enough to process at least the average load. The OSE features a memory based buffer that you can set using log_fifo_size(), the Premium one also has a disk based one. Also, the 3.2 release of syslog-ng contained a change that improves SQL performance a lot. (by enabling explicit-commits, instead of commit-by-insert which the older versions used by default). -- Bazsi