[syslog-ng] [review request] 3.6queue/f/stats-reset

[Evan Rempel]

I have a couple of issues with the statistics mechanism proposed, and with the existing mechanism.

First off, the statistics that are logged when stats-freq() is non zero does not contain all of the information
that "syslog-ng-ctl stats" provides. To get this information I must use "syslog-ng-ctl stats" but with the dropping
of the statistics counters, they may be dropped before I can get to them with syslog-ng-ctl and I would
loose some information.

To resolve that, I would like a flag that makes the stats-freq() log the same data as the syslog-ng-ctl gets. This can
be filtered and routed to monitoring code without loosing any stats.

The second issue is that there is no way to log the log_fifo_size for each destination. The number queued is not
very useful without also having the size of the queue. There is no way to pre-warn that messages might be dropped
until they are actually dropped, but by that time it is a little too late :-(

To resolve this, add a queue size item to the statistics dump


dst.tcp;d_syslog1#0;syslog.uvic.ca:514;a;dropped;0
dst.tcp;d_syslog1#0;syslog.uvic.ca:514;a;processed;109506432
dst.tcp;d_syslog1#0;syslog.uvic.ca:514;a;stored;0
dst.tcp;d_syslog1#0;syslog.uvic.ca:514;a;size;2000000

Thanks,

On 11/20/2013 06:48 AM, Jim Hendrick wrote:
>
> If desired, could a "zero-counters" be added to syslog-ng-ctl ?
>
> Not critical but maybe useful for troubleshooting.
>
> Jim
>
> Sent from my Verizon Wireless 4G LTE Smartphone
>
>
>
> -------- Original message --------
> From: Gergely Nagy <algernon at balabit.hu>
> Date: 11/20/2013 8:47 AM (GMT-05:00)
> To: Syslog-ng users' and developers' mailing list <syslog-ng at lists.balabit.hu>
> Cc: syslog-ng-dev-l at balabit.hu
> Subject: Re: [syslog-ng] [review request] 3.6queue/f/stats-reset
>
>
> Balazs Scheidler <bazsi77 at gmail.com> writes:
>
>  > On Mon, 2013-11-18 at 18:22 +0100, Gergely Nagy wrote:
>  >> Evan Rempel <erempel at uvic.ca> writes:
>  >>
>  >> > Perhaps the same logic can be used to identify those counters that have expired meaning that the stats-lifetime() has been exceeded,but
>  >> > instead of dropping the stat counter at that time, the counter is just flagged for dropping.
>  >> >
>  >> > Then it becomes the job of the statistics logging, based on the stats-freq() value, to log AND drop those counters that
>  >> > have been flagged as having exceeded the stats-lifetime().
>  >> >
>  >> > This way a counter is never dropped until it has exceeded the
>  >> > stats-lifetime() AND has been logged.
>  >>
>  >> That makes a lot of sense, but syslog-ng-ctl stats should not count (or
>  >> perhaps only with a flag), precisely to avoid the scenario you outlined
>  >> too.
>  >>
>  >
>  > This really makes sense, however my issue is that stats-freq() is often
>  > set to 0 which disables the Log statistics message currently. Relying
>  > the cleanup mechanism on that would disable cleanup altogether in a lot
>  > of setups.
>  >
>  > hmm... maybe we should use stats-freq() if that's nonzero and
>  > stats-lifetime() when it is zero. This way the cleanup mechanism would
>  > kick in, without the risk of losing messages.
>  >
>  > syslog-ng-ctl stats wouldn't prune counters.
>  >
>  > This way, stats-freq() would be the one which actually clears counters,
>  > we get rid of an additional iteration over the set of counters.
>  >
>  > This paves the way for an improved log statistics message that could be
>  > sent into graphite/logstash/whatever graphing application via standard
>  > syslog-ng destination drivers, and all this without the risk of losing
>  > counters.
>  >
>  > What do you think?
>
> Sounds very good to me. This is a big step towards what I need, and is
> simple enough to understand too.
>
> --
> |8]
>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>
>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>


-- 
Evan Rempel                                      erempel at uvic.ca
Senior Systems Administrator                        250.721.7691
Data Centre Services, University Systems, University of Victoria