[syslog-ng] Dropped messages to MSSQL?

[Balazs Scheidler]

On Thu, 2011-03-10 at 09:21 +0100, Zoltán Pallagi wrote:
> Hi,
> 
> First of all, it seems that syslog-ng receives logs faster than your
> mssql server can process them that's the reason for dropping.
> 
> However, you are using udp source for receiving logs, it's not the
> best solution if you don't want to lose logs (UDP is not a lossless
> protocol and perhaps the kernel will also drop the unprocessed
> messages). 
> You should use TCP instead of UDP.
> 
> If you use TCP, you can use flags(flow-control) in your server
> configuration. If the senders are also syslog-ng, you can use it on
> their configurations, too.
> flow-control will slow down (or block) receiving logs if syslog-ng
> cannot process (write out, forward and so on) the messages in time. It
> can prevent losing logs.
> 
> For example:
> log {
> source(s_gms);
> flags(flow-control);
> destination(d_mssql);
> };

Also, by increasing the buffer size, you may be able to process peaks,
if otherwise your SQL server is fast enough to process at least the
average load.

The OSE features a memory based buffer that you can set using
log_fifo_size(), the Premium one also has a disk based one. Also, the
3.2 release of syslog-ng contained a change that improves SQL
performance a lot. (by enabling explicit-commits, instead of
commit-by-insert which the older versions used by default).


-- 
Bazsi