[syslog-ng] Feature request - statistics show log_fifo_size

[Balazs Scheidler]

----- Original message -----
> I thing it would be very useful to use the statistics to determine how
> close the buffering has gotten to the limit of the log_fifo_size before
> messages start being dropped. This information can be used to;

hmm. IIRC there's a stored counter that measures how much messages are stored in a queue. I'm unable to check the source right now, and this is only a vague recollection, or might be a pe feature I stumbled upon.

> 
> - proactively change the configuration on a host.
> - calculate the rate of buffer consumption and predict how much time
> will elapse before dropping will start.   Alerts to system operators can
> be created prior to messages being dropped.
> 



> This information can be calculated from having the number of messages
> queued, and the log_fifo_size for the destination, or a percentage full
> could be used.
> 
> I would prefer the log_fifo_size.
> 
> I would also prefer to see all of the stats for a destination on a
> single line. Programaticaly it is easier to split the numbers up than it
> is the try to join them back together based on the first three fields of
> a stat line
> 
> dst.file;d_var_syslog#0;/var/log/syslog.20120908.000000;a;dropped;0
> dst.file;d_var_syslog#0;/var/log/syslog.20120908.000000;a;processed;326
> dst.file;d_var_syslog#0;/var/log/syslog.20120908.000000;a;stored;0
> 

huh, this is the stored counter I was referring to above. so it definitely exists, the question remains if it gets updated properly. 

> Something like
> 
> 
> dst.file;d_var_syslog#0;/var/log/syslog.20120908.000000;a;0;326;0;5000,1347143526
> 
> where the 4 numbers are   dropped, processed, stored, fifo_size and stamp
> or two different formats, one for source and one for destination
> 
> 
> src.file;local#2;/proc/kmsg;a;1549;1347143526
> dst.file;d_var_syslog#0;/var/log/syslog.20120908.000000;a;0;326;0;5000

this could probably be done, although a little problematic to extend, and would break applications that relied on the current format. perhaps a separate stats switch could trigger this alternate format.

> 
> where
> src has processed and stamp
> dst has dropped, processed, stored, fifo_size
> 
> Possibly the source could also show the flow-control state as well, or
> even better, the flow control duration, so if a source has been blocked
> for x number of seconds I can alert an operations center about the
> condition.

like the longest 'blocking period' since the start of syslog-ng. this could be monitored easily.

> 
> Thanks again for your consideration.
> 

good ideas in general, and shouldn't be difficult either. I'm somewhat overwhelmed these days.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.balabit.hu/pipermail/syslog-ng/attachments/20120909/4b503147/attachment.htm