[syslog-ng] logging statistics

[Wolfram Schlich]

* Heinemann, Joerg (SBI Ruhr) <Joerg.Heinemann at sbi-ruhr.de> [2006-06-19 10:30]:
> Hi,

Hello Joerg!

> we are plannig to migrate our syslog infrastructure to syslog-ng.

Congratulations :)

> Is there a possibility to generate a daily logging statistics like our old kiwi syslogger?
> 
> Kiwi Syslog Daemon Statistics
> ---------------------------------------------------
> 24 hour period ending on: Sun, 11 Jun 2006 00:00:03
> Syslog Daemon started on: Sat, 13 May 2006 22:03:13
> Syslog Daemon uptime: 28 days, 1 hour, 56 minutes
> ---------------------------------------------------
> 
> + Messages received - Total: 129600953
> + Messages received - Last 24 hours: 2556443
> + Messages received - Since Midnight: 2418503
> + Messages received - Last hour: 92320
> + Messages received - This hour: 76835
> + Messages per hour - Average: 103317
> 
> + Messages forwarded: 0
> + Messages logged to disk: 4836834
> 
> + Errors - Logging to disk: 0
> + Errors - Invalid priority tag: 0
> + Errors - No priority tag: 0
> + Errors - Oversize message: 11

Well, syslog-ng has a global option stats_freq() -- see
http://www.balabit.com/products/syslog_ng/reference-2.0/syslog-ng.html/index.html#id2526962

Such a stats log message looks like this:
--8<--
syslog-ng[7359]: Log statistics; processed='source(SrcName)=NumMsgs', processed='destination(DstName)=NumMgs'
--8<--
If you have more than one source/destination, more ", processed=" entries
are appended.

Please keep in mind that if you have many sources/destinations (like I do),
the stats log message is chopped after log_msg_size() bytes, which is 8192
by default!

You could use those stats log messages to generate your own more
comprehensive stats.

You could even log those stats messages to a special logfile to
process them specially, like this:

--8<--
 options {
  stats_freq(600);
  log_msg_size(16384);
 };

 template t_stats {
  template("${YEAR}-${MONTH}-${DAY} ${HOUR}:${MIN}:${SEC} ${TZOFFSET}; ${MSG}\n");
  template_escape(no);
 };

 source s_local {
  unix-stream("/dev/log" max-connections(1000));
  internal();
 };

 destination d_stats {
  file("/var/log/syslog-ng/stats.log" template(t_stats));
 };

 filter f_stats {
  facility("syslog");
  priority("notice");
  program("^syslog-ng");
  match("Log statistics;");
 };

 log {
  source(s_local);
  filter(f_stats);
  destination(d_firewall);
 };
--8<--

Caution: this is untested -- I just hacked it together.
-- 
Regards,
Wolfram Schlich <wschlich at gentoo.org>
Gentoo Linux * http://dev.gentoo.org/~wschlich/