[syslog-ng] [review request] 3.6queue/f/stats-reset

[Balazs Scheidler]

On Mon, 2013-11-18 at 12:22 +0100, Gergely Nagy wrote:
> Balazs Scheidler <bazsi at balabit.hu> writes:
> 
> > User-visible changes:
> > =====================
> >
> > syslog-ng is keeping track on a number of statistics counters that can
> > be used to track what is happening.
> >
> > The detail level of these counters is controlled by the stats-level()
> > global option, which defaults to zero.
> >
> > syslog-ng reports these counters periodically every stats-freq()
> > seconds, or preferably on-demand by invoking "syslog-ng-ctl stats" on
> > the command line.
> >
> > At stats-level(3) a number of "dynamic" counters are enabled, which
> > track the number of messages on a per $HOST, per $PROGRAM and per
> > $HOST_FROM basis. This means that you can follow how many messages a
> > given $PROGRAM is generating in a period.
> >
> > The issue with dynamic counters is that they can consume a lot of memory
> > if  the rate of new $PRORGRAM/$HOST values is high.
> >
> > Previously the cleanup happened in a limited way at configuration reload
> > (e.g. SIGHUP) time, which is not really useful for controlling memory
> > usage by this feature.
> >
> > The current patch series introduces a stats-lifetime() option and
> > syslog-ng starts inspecting counters every once in a while and prune
> > those that are updated earlier than stats-lifetime().
> 
> So, this means that if I don't query the stats just before clearing,
> I'll potentially loose information? I mean, considering stats-freq(60)
> stats-lifetime(10), if I have a source that has messages available every
> 14 seconds, I'll loose some of  them.

yeah, stats-lifetime() should be set to a value to at least stats-freq()
(or the poll frequency of an external polling application) to avoid data
loss.

> 
> Would it be possible to dump the stats just before forgetting them?
> Even better, dump only the ones we're about to prune out. That'd make
> things a bit safer, I think.
> 

I'd like to make it possible to generate a structured log message when
stats-freq() happens. that can be the "dump" operation.

I'm thinking along the lines of

${.syslog-ng.stats[0].type}        == "processed"
${.syslog-ng.stats[0].source}      == "src.file"
${.syslog-ng.stats[0].instance}    == "input.log"
${.syslog-ng.stats[0].value}       == "232323"
${.syslog-ng.stats[0].value_delta} == "3"

And when formatted using $(format-json) this could become an actual
array.

we would need array support in name-value pairs for that though.

The point of the format is to make it as easy to generate graphs using
graphite or similar tools.

Once we have the structured format, we might as well reuse that to dump
pruned entries once pruning happens.

What do you think?


-- 
Bazsi