[syslog-ng] Debugging tools for syslog-ng

[Anton Koldaev]

My current issue:

syslog ~ % watch -d 'sudo syslog-ng-ctl stats | sort -rnk2 -t ";" | grep
"_custom"'

dst.sql;d_mysql_example_custom#0;mysql,10.0.0.1,3306,syslog_production,custom_example_${HO;a;stored;
*1000*
dst.sql;d_mysql_example_custom#0;mysql,10.0.0.1,3306,syslog_production,custom_example_${HO;a;dropped;0
dst.file;d_app_example_custom#0;/logs/example/custom.log;o;stored;0
dst.file;d_app_example_custom#0;/logs/example/custom.log;o;processed;351305
dst.file;d_app_example_custom#0;/logs/example/custom.log;o;dropped;0
destination;d_mysql_example_custom;;a;processed;331953
destination;d_app_example_custom;;a;processed;351305

It just stops to read the source after a random time(1-2-3hours) with 1000
stored statements. There are no problems at mysql destination. My current
configuration: https://gist.github.com/9f5619573d2f3e9f071c

I've already tried to tune all the values, it doesn't seem to help.

Also I'm not able to enable debug logs due to
https://bugzilla.balabit.com/show_bug.cgi?id=208




On Mon, Nov 5, 2012 at 2:32 PM, Gergely Nagy <algernon at balabit.hu> wrote:

> Anton Koldaev <koldaevav at gmail.com> writes:
>
> > I wanted to know if syslog-ng developers has some tools like mysqltuner
> or
> > just a shell scripts to check syslog-ng configuration and get some
> > recommendations on tuning?
>
> My bottleneck is usually not syslog-ng, so I use perf/tuning tools to
> whatever is on the other end (be that a database, filesystem or
> network). To see how much I need to tune the various syslog-ng buffers,
> I do load testing in a simulated environment, and base my settings on
> the number of dropped messages, and tune both the receiving end and
> syslog-ng until the drop count gets to zero during peak-like loads.
>
> So far, this method worked remarkably well, but most of my setups have
> reasonably low incoming log volume, most time is spent post-processing
> them, which I usually do outside of syslog-ng.
>
> > For example if I'm using flow-control+multiple destinations it can stop
> > reading the source at any time and I have no idea when and why it's
> > happening and which value should I tune.
>
> It would be nice if syslog-ng would log an info (so that I don't need to
> enable debug logging on a live system) level message when flow-control
> kicks in (and when it stops). For bonus points, if it could tell what
> triggered it, and which source it applies to, that'd be great.
>
> I don't think we can do this yet, though.
>
> --
> |8]
>
>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation:
> http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>


-- 
Best regards,
Koldaev Anton
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.balabit.hu/pipermail/syslog-ng/attachments/20121105/6c7adb6b/attachment.htm