My current issue: <div><br></div><div><div>syslog ~ % watch -d 'sudo syslog-ng-ctl stats | sort -rnk2 -t ";" | grep "_custom"'</div><div><br></div><div>dst.sql;d_mysql_example_custom#0;mysql,10.0.0.1,3306,syslog_production,custom_example_${HO;a;stored;<b>1000</b></div>
<div>dst.sql;d_mysql_example_custom#0;mysql,10.0.0.1,3306,syslog_production,custom_example_${HO;a;dropped;0</div><div>dst.file;d_app_example_custom#0;/logs/example/custom.log;o;stored;0</div><div>dst.file;d_app_example_custom#0;/logs/example/custom.log;o;processed;351305</div>
<div>dst.file;d_app_example_custom#0;/logs/example/custom.log;o;dropped;0</div><div>destination;d_mysql_example_custom;;a;processed;331953</div><div>destination;d_app_example_custom;;a;processed;351305</div></div><div><br>
</div><div>It just stops to read the source after a random time(1-2-3hours) with 1000 stored statements. There are no problems at mysql destination. My current configuration: <a href="https://gist.github.com/9f5619573d2f3e9f071c">https://gist.github.com/9f5619573d2f3e9f071c</a></div>
<div><br></div><div>I've already tried to tune all the values, it doesn't seem to help.</div><div><br></div><div>Also I'm not able to enable debug logs due to <a href="https://bugzilla.balabit.com/show_bug.cgi?id=208">https://bugzilla.balabit.com/show_bug.cgi?id=208</a></div>
<div><br></div><div><br></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Mon, Nov 5, 2012 at 2:32 PM, Gergely Nagy <span dir="ltr"><<a href="mailto:algernon@balabit.hu" target="_blank">algernon@balabit.hu</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="im">Anton Koldaev <<a href="mailto:koldaevav@gmail.com">koldaevav@gmail.com</a>> writes:<br>
<br>
> I wanted to know if syslog-ng developers has some tools like mysqltuner or<br>
> just a shell scripts to check syslog-ng configuration and get some<br>
> recommendations on tuning?<br>
<br>
</div>My bottleneck is usually not syslog-ng, so I use perf/tuning tools to<br>
whatever is on the other end (be that a database, filesystem or<br>
network). To see how much I need to tune the various syslog-ng buffers,<br>
I do load testing in a simulated environment, and base my settings on<br>
the number of dropped messages, and tune both the receiving end and<br>
syslog-ng until the drop count gets to zero during peak-like loads.<br>
<br>
So far, this method worked remarkably well, but most of my setups have<br>
reasonably low incoming log volume, most time is spent post-processing<br>
them, which I usually do outside of syslog-ng.<br>
<div class="im"><br>
> For example if I'm using flow-control+multiple destinations it can stop<br>
> reading the source at any time and I have no idea when and why it's<br>
> happening and which value should I tune.<br>
<br>
</div>It would be nice if syslog-ng would log an info (so that I don't need to<br>
enable debug logging on a live system) level message when flow-control<br>
kicks in (and when it stops). For bonus points, if it could tell what<br>
triggered it, and which source it applies to, that'd be great.<br>
<br>
I don't think we can do this yet, though.<br>
<span class="HOEnZb"><font color="#888888"><br>
--<br>
|8]<br>
<br>
______________________________________________________________________________<br>
Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" target="_blank">https://lists.balabit.hu/mailman/listinfo/syslog-ng</a><br>
Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" target="_blank">http://www.balabit.com/support/documentation/?product=syslog-ng</a><br>
FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" target="_blank">http://www.balabit.com/wiki/syslog-ng-faq</a><br>
<br>
</font></span></blockquote></div><br><br clear="all"><div><br></div>-- <br>Best regards,<br>Koldaev Anton<br>
</div>