My current issue: 

syslog ~ % watch -d 'sudo syslog-ng-ctl stats | sort -rnk2 -t ";" | grep "_custom"'

dst.sql;d_mysql_example_custom#0;mysql,10.0.0.1,3306,syslog_production,custom_example_${HO;a;stored;1000
dst.sql;d_mysql_example_custom#0;mysql,10.0.0.1,3306,syslog_production,custom_example_${HO;a;dropped;0
dst.file;d_app_example_custom#0;/logs/example/custom.log;o;stored;0
dst.file;d_app_example_custom#0;/logs/example/custom.log;o;processed;351305
dst.file;d_app_example_custom#0;/logs/example/custom.log;o;dropped;0
destination;d_mysql_example_custom;;a;processed;331953
destination;d_app_example_custom;;a;processed;351305

It just stops to read the source after a random time(1-2-3hours) with 1000 stored statements. There are no problems at mysql destination. My current configuration: https://gist.github.com/9f5619573d2f3e9f071c

I've already tried to tune all the values, it doesn't seem to help.

Also I'm not able to enable debug logs due to https://bugzilla.balabit.com/show_bug.cgi?id=208




On Mon, Nov 5, 2012 at 2:32 PM, Gergely Nagy <algernon@balabit.hu> wrote:
Anton Koldaev <koldaevav@gmail.com> writes:

> I wanted to know if syslog-ng developers has some tools like mysqltuner or
> just a shell scripts to check syslog-ng configuration and get some
> recommendations on tuning?

My bottleneck is usually not syslog-ng, so I use perf/tuning tools to
whatever is on the other end (be that a database, filesystem or
network). To see how much I need to tune the various syslog-ng buffers,
I do load testing in a simulated environment, and base my settings on
the number of dropped messages, and tune both the receiving end and
syslog-ng until the drop count gets to zero during peak-like loads.

So far, this method worked remarkably well, but most of my setups have
reasonably low incoming log volume, most time is spent post-processing
them, which I usually do outside of syslog-ng.

> For example if I'm using flow-control+multiple destinations it can stop
> reading the source at any time and I have no idea when and why it's
> happening and which value should I tune.

It would be nice if syslog-ng would log an info (so that I don't need to
enable debug logging on a live system) level message when flow-control
kicks in (and when it stops). For bonus points, if it could tell what
triggered it, and which source it applies to, that'd be great.

I don't think we can do this yet, though.

--
|8]

______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq




--
Best regards,
Koldaev Anton