Thanks for all the info. The current method that our firewall management program uses to log messages into the current database is by multiple open connections to the database. syslog-ng is making one connection and trying to force everything down that one connection. So, my question is this: can syslog-ng be configured to make multiple connections to the SQL database to insert the data? Just so you have a comparison, our current product (which changes in the new version and why we need a different syslog product) has182 open connections open and that is from 8 agents. It stays up to speen by doing that. Thanks....<br>
<br><div class="gmail_quote">On Thu, Mar 10, 2011 at 11:06 AM, Martin Holste <span dir="ltr"><<a href="mailto:mcholste@gmail.com">mcholste@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
Feel free to contradict, but in my experience, if you have more than<br>
around 2k messages/second sustained, logging to any database directly<br>
puts you at very high risk of message drops. Flow control and other<br>
burst control mechanisms will not help if you have an unsustainable<br>
message rate.<br>
<div><div></div><div class="h5"><br>
On Thu, Mar 10, 2011 at 9:33 AM, John Kristoff <<a href="mailto:jtk@cymru.com">jtk@cymru.com</a>> wrote:<br>
> On Thu, 10 Mar 2011 09:21:56 +0100<br>
> Zoltán Pallagi <<a href="mailto:pzolee@balabit.hu">pzolee@balabit.hu</a>> wrote:<br>
><br>
>> If you use TCP, you can use flags(flow-control) in your server<br>
>> configuration. If the senders are also syslog-ng, you can use it on<br>
>> their configurations, too.<br>
>> flow-control will slow down (or block) receiving logs if syslog-ng<br>
>> cannot process (write out, forward and so on) the messages in time.<br>
>> It can prevent losing logs.<br>
><br>
> The one caveat with this approach seems to be that if you have multiple<br>
> destinations, then all destinations will block until the one stalled<br>
> destination is free. So for instance if the SQL destination is too<br>
> slow, and you're also logging to a file, using flow-control may cause<br>
> the file-based log to lose messages as well.<br>
><br>
> John<br>
> ______________________________________________________________________________<br>
> Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" target="_blank">https://lists.balabit.hu/mailman/listinfo/syslog-ng</a><br>
> Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" target="_blank">http://www.balabit.com/support/documentation/?product=syslog-ng</a><br>
> FAQ: <a href="http://www.campin.net/syslog-ng/faq.html" target="_blank">http://www.campin.net/syslog-ng/faq.html</a><br>
><br>
><br>
______________________________________________________________________________<br>
Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" target="_blank">https://lists.balabit.hu/mailman/listinfo/syslog-ng</a><br>
Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" target="_blank">http://www.balabit.com/support/documentation/?product=syslog-ng</a><br>
FAQ: <a href="http://www.campin.net/syslog-ng/faq.html" target="_blank">http://www.campin.net/syslog-ng/faq.html</a><br>
<br>
</div></div></blockquote></div><br>