<p dir="ltr">I havent followed this up, but this got integrated and is on mainline. The additional functionality is noted but not yet there.</p>
<div class="gmail_quote">On Nov 27, 2013 11:22 AM, "Balazs Scheidler" <<a href="mailto:bazsi@balabit.hu">bazsi@balabit.hu</a>> wrote:<br type="attribution"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi,<br>
<br>
I have now updated my branch. The new functionality (extended statistics<br>
log message and log-fifo-size() reporting through stats) is not yet<br>
created, however a set of code reorganization would make this change<br>
much easier.<br>
<br>
So the current patch set (published on the same branch, rebased since<br>
last time) uses the stats-freq() timer to prune unneeded counters, but<br>
only if they have already been reported by the statistics log message.<br>
<br>
This solves the losing-data issue that was discussed earlier.<br>
<br>
So this time, it is more of a code-review thing, could those interested<br>
in code review this in the coming days, prior to integration?<br>
<br>
Thanks.<br>
<br>
On Wed, 2013-11-20 at 14:47 +0100, Gergely Nagy wrote:<br>
> Balazs Scheidler <<a href="mailto:bazsi77@gmail.com">bazsi77@gmail.com</a>> writes:<br>
><br>
> > On Mon, 2013-11-18 at 18:22 +0100, Gergely Nagy wrote:<br>
> >> Evan Rempel <<a href="mailto:erempel@uvic.ca">erempel@uvic.ca</a>> writes:<br>
> >><br>
> >> > Perhaps the same logic can be used to identify those counters that have expired meaning that the stats-lifetime() has been exceeded,but<br>
> >> > instead of dropping the stat counter at that time, the counter is just flagged for dropping.<br>
> >> ><br>
> >> > Then it becomes the job of the statistics logging, based on the stats-freq() value, to log AND drop those counters that<br>
> >> > have been flagged as having exceeded the stats-lifetime().<br>
> >> ><br>
> >> > This way a counter is never dropped until it has exceeded the<br>
> >> > stats-lifetime() AND has been logged.<br>
> >><br>
> >> That makes a lot of sense, but syslog-ng-ctl stats should not count (or<br>
> >> perhaps only with a flag), precisely to avoid the scenario you outlined<br>
> >> too.<br>
> >><br>
> ><br>
> > This really makes sense, however my issue is that stats-freq() is often<br>
> > set to 0 which disables the Log statistics message currently. Relying<br>
> > the cleanup mechanism on that would disable cleanup altogether in a lot<br>
> > of setups.<br>
> ><br>
> > hmm... maybe we should use stats-freq() if that's nonzero and<br>
> > stats-lifetime() when it is zero. This way the cleanup mechanism would<br>
> > kick in, without the risk of losing messages.<br>
> ><br>
> > syslog-ng-ctl stats wouldn't prune counters.<br>
> ><br>
> > This way, stats-freq() would be the one which actually clears counters,<br>
> > we get rid of an additional iteration over the set of counters.<br>
> ><br>
> > This paves the way for an improved log statistics message that could be<br>
> > sent into graphite/logstash/whatever graphing application via standard<br>
> > syslog-ng destination drivers, and all this without the risk of losing<br>
> > counters.<br>
> ><br>
> > What do you think?<br>
><br>
> Sounds very good to me. This is a big step towards what I need, and is<br>
> simple enough to understand too.<br>
><br>
<br>
<br>
<br>
______________________________________________________________________________<br>
Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" target="_blank">https://lists.balabit.hu/mailman/listinfo/syslog-ng</a><br>
Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" target="_blank">http://www.balabit.com/support/documentation/?product=syslog-ng</a><br>
FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" target="_blank">http://www.balabit.com/wiki/syslog-ng-faq</a><br>
<br>
</blockquote></div>