My current issue: syslog ~ % watch -d 'sudo syslog-ng-ctl stats | sort -rnk2 -t ";" | grep "_custom"' dst.sql;d_mysql_example_custom#0;mysql,10.0.0.1,3306,syslog_production,custom_example_${HO;a;stored; *1000* dst.sql;d_mysql_example_custom#0;mysql,10.0.0.1,3306,syslog_production,custom_example_${HO;a;dropped;0 dst.file;d_app_example_custom#0;/logs/example/custom.log;o;stored;0 dst.file;d_app_example_custom#0;/logs/example/custom.log;o;processed;351305 dst.file;d_app_example_custom#0;/logs/example/custom.log;o;dropped;0 destination;d_mysql_example_custom;;a;processed;331953 destination;d_app_example_custom;;a;processed;351305 It just stops to read the source after a random time(1-2-3hours) with 1000 stored statements. There are no problems at mysql destination. My current configuration: https://gist.github.com/9f5619573d2f3e9f071c I've already tried to tune all the values, it doesn't seem to help. Also I'm not able to enable debug logs due to https://bugzilla.balabit.com/show_bug.cgi?id=208 On Mon, Nov 5, 2012 at 2:32 PM, Gergely Nagy <algernon@balabit.hu> wrote:
Anton Koldaev <koldaevav@gmail.com> writes:
I wanted to know if syslog-ng developers has some tools like mysqltuner or just a shell scripts to check syslog-ng configuration and get some recommendations on tuning?
My bottleneck is usually not syslog-ng, so I use perf/tuning tools to whatever is on the other end (be that a database, filesystem or network). To see how much I need to tune the various syslog-ng buffers, I do load testing in a simulated environment, and base my settings on the number of dropped messages, and tune both the receiving end and syslog-ng until the drop count gets to zero during peak-like loads.
So far, this method worked remarkably well, but most of my setups have reasonably low incoming log volume, most time is spent post-processing them, which I usually do outside of syslog-ng.
For example if I'm using flow-control+multiple destinations it can stop reading the source at any time and I have no idea when and why it's happening and which value should I tune.
It would be nice if syslog-ng would log an info (so that I don't need to enable debug logging on a live system) level message when flow-control kicks in (and when it stops). For bonus points, if it could tell what triggered it, and which source it applies to, that'd be great.
I don't think we can do this yet, though.
-- |8]
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
-- Best regards, Koldaev Anton