<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html><head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="generator" content="Osso Notes">
<title></title></head>
<body>
<p>----- Original message -----
<br>> I thing it would be very useful to use the statistics to determine how
<br>> close the buffering has gotten to the limit of the log_fifo_size before
<br>> messages start being dropped. This information can be used to;
<br>
<br>hmm. IIRC there's a stored counter that measures how much messages are stored in a queue. I'm unable to check the source right now, and this is only a vague recollection, or might be a pe feature I stumbled upon.
<br>
<br>>
<br>> - proactively change the configuration on a host.
<br>> - calculate the rate of buffer consumption and predict how much time
<br>> will elapse before dropping will start.  Alerts to system operators can
<br>> be created prior to messages being dropped.
<br>>
<br>
<br>
<br>
<br>> This information can be calculated from having the number of messages
<br>> queued, and the log_fifo_size for the destination, or a percentage full
<br>> could be used.
<br>>
<br>> I would prefer the log_fifo_size.
<br>>
<br>> I would also prefer to see all of the stats for a destination on a
<br>> single line. Programaticaly it is easier to split the numbers up than it
<br>> is the try to join them back together based on the first three fields of
<br>> a stat line
<br>>
<br>> dst.file;d_var_syslog#0;/var/log/syslog.20120908.000000;a;dropped;0
<br>> dst.file;d_var_syslog#0;/var/log/syslog.20120908.000000;a;processed;326
<br>> dst.file;d_var_syslog#0;/var/log/syslog.20120908.000000;a;stored;0
<br>>
<br>
<br>huh, this is the stored counter I was referring to above. so it definitely exists, the question remains if it gets updated properly.
<br>
<br>> Something like
<br>>
<br>>
<br>> dst.file;d_var_syslog#0;/var/log/syslog.20120908.000000;a;0;326;0;5000,1347143526
<br>>
<br>> where the 4 numbers are  dropped, processed, stored, fifo_size and stamp
<br>> or two different formats, one for source and one for destination
<br>>
<br>>
<br>> src.file;local#2;/proc/kmsg;a;1549;1347143526
<br>> dst.file;d_var_syslog#0;/var/log/syslog.20120908.000000;a;0;326;0;5000
<br>
<br>this could probably be done, although a little problematic to extend, and would break applications that relied on the current format. perhaps a separate stats switch could trigger this alternate format.
<br>
<br>>
<br>> where
<br>> src has processed and stamp
<br>> dst has dropped, processed, stored, fifo_size
<br>>
<br>> Possibly the source could also show the flow-control state as well, or
<br>> even better, the flow control duration, so if a source has been blocked
<br>> for x number of seconds I can alert an operations center about the
<br>> condition.
<br>
<br>like the longest 'blocking period' since the start of syslog-ng. this could be monitored easily.
<br>
<br>>
<br>> Thanks again for your consideration.
<br>>
<br>
<br>good ideas in general, and shouldn't be difficult either. I'm somewhat overwhelmed these days.<br></p>
</body>
</html>