[syslog-ng] logging statistics

[Wolfram Schlich]

* Sandor Geller <wildy at balabit.hu> [2006-06-19 14:24]:
> Wolfram Schlich wrote:
> 
> > Well, syslog-ng has a global option stats_freq() -- see
> > http://www.balabit.com/products/syslog_ng/reference-2.0/syslog-ng.html/index.html#id2526962
> > 
> > Such a stats log message looks like this:
> > --8<--
> > syslog-ng[7359]: Log statistics; processed='source(SrcName)=NumMsgs', processed='destination(DstName)=NumMgs'
> > --8<--
> > If you have more than one source/destination, more ", processed=" entries
> > are appended.
> > 
> > Please keep in mind that if you have many sources/destinations (like I do),
> > the stats log message is chopped after log_msg_size() bytes, which is 8192
> > by default!
> 
> This behaviour is going to be changed in the future, there was a
> discussion about the possible solutions already. The current proposal is
> to create a separate logfile which will be simply appended, so the
> statistic information (and maybe other internal messages) won't be
> handled as the other logs.

Can you point me at the corresponding message(s)?
A URL from http://news.gmane.org/gmane.comp.syslog-ng would be nice :-)

> > You could use those stats log messages to generate your own more
> > comprehensive stats.
> > 
> > You could even log those stats messages to a special logfile to
> > process them specially, like this:
> > 
> > --8<--
> >  options {
> >   stats_freq(600);
> >   log_msg_size(16384);
> >  };
> > 
> >  template t_stats {
> >   template("${YEAR}-${MONTH}-${DAY} ${HOUR}:${MIN}:${SEC} ${TZOFFSET}; ${MSG}\n");
> >   template_escape(no);
> >  };
> > 
> >  source s_local {
> >   unix-stream("/dev/log" max-connections(1000));
> >   internal();
> >  };
> > 
> >  destination d_stats {
> >   file("/var/log/syslog-ng/stats.log" template(t_stats));
> >  };
> > 
> >  filter f_stats {
> >   facility("syslog");
> >   priority("notice");
> >   program("^syslog-ng");
> >   match("Log statistics;");
> >  };
> > 
> >  log {
> >   source(s_local);
> >   filter(f_stats);
> >   destination(d_firewall);
> >  };
> > --8<--
> > 
> > Caution: this is untested -- I just hacked it together.
> 
> Good solution, however moving the internal() into separate source is
> more optimal, especially when the host syslog-ng is running on generates
> a lot of logs.

Good point! I overlooked that in my hurry :-)
Thanks for pointing that out!

So, it should be like this:
--8<--
 options {
  stats_freq(600);
  log_msg_size(16384);
 };

 template t_stats {
  template("${YEAR}-${MONTH}-${DAY} ${HOUR}:${MIN}:${SEC} ${TZOFFSET}; ${MSG}\n");
  template_escape(no);
 };

 source s_internal {
  internal();
 };

 destination d_stats {
  file("/var/log/syslog-ng/stats.log" template(t_stats));
 };

 filter f_stats {
  facility("syslog");
  priority("notice");
  program("^syslog-ng");
  match("Log statistics;");
 };

 log {
  source(s_internal);
  filter(f_stats);
  destination(d_firewall);
 };
--8<--

Happy testing ;)
-- 
Regards,
Wolfram Schlich <wschlich at gentoo.org>
Gentoo Linux * http://dev.gentoo.org/~wschlich/