Hi, we are plannig to migrate our syslog infrastructure to syslog-ng. Is there a possibility to generate a daily logging statistics like our old kiwi syslogger? Kiwi Syslog Daemon Statistics --------------------------------------------------- 24 hour period ending on: Sun, 11 Jun 2006 00:00:03 Syslog Daemon started on: Sat, 13 May 2006 22:03:13 Syslog Daemon uptime: 28 days, 1 hour, 56 minutes --------------------------------------------------- + Messages received - Total: 129600953 + Messages received - Last 24 hours: 2556443 + Messages received - Since Midnight: 2418503 + Messages received - Last hour: 92320 + Messages received - This hour: 76835 + Messages per hour - Average: 103317 + Messages forwarded: 0 + Messages logged to disk: 4836834 + Errors - Logging to disk: 0 + Errors - Invalid priority tag: 0 + Errors - No priority tag: 0 + Errors - Oversize message: 11 Mit freundlichem Gruß / Best Regards Jörg Heinemann Services for Business IT Ruhr GmbH Communication & Network Services Operation WAN / LAN Bruchstraße 5, 45883 Gelsenkirchen, Germany Jörg Heinemann Tel.: +49 209 9456-7790 Fax: +49 209 9456-67790 joerg.heinemann@sbi-ruhr.de <mailto:joerg.heinemann@sbi-ruhr.de> http://www.sbi-ruhr.de <http://www.sbi-ruhr.de> Diese E-Mail enthält vertrauliche und/oder rechtlich geschützte Informationen. Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtümlich erhalten haben, informieren Sie bitte sofort den Absender und vernichten Sie diese Mail. Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser Mail ist nicht gestattet. This e-mail may contain confidential and/or privileged Information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorised copying, disclosure or distribution of the material in this e-mail is strictly forbidden.
* Heinemann, Joerg (SBI Ruhr) <Joerg.Heinemann@sbi-ruhr.de> [2006-06-19 10:30]:
Hi,
Hello Joerg!
we are plannig to migrate our syslog infrastructure to syslog-ng.
Congratulations :)
Is there a possibility to generate a daily logging statistics like our old kiwi syslogger?
Kiwi Syslog Daemon Statistics --------------------------------------------------- 24 hour period ending on: Sun, 11 Jun 2006 00:00:03 Syslog Daemon started on: Sat, 13 May 2006 22:03:13 Syslog Daemon uptime: 28 days, 1 hour, 56 minutes ---------------------------------------------------
+ Messages received - Total: 129600953 + Messages received - Last 24 hours: 2556443 + Messages received - Since Midnight: 2418503 + Messages received - Last hour: 92320 + Messages received - This hour: 76835 + Messages per hour - Average: 103317
+ Messages forwarded: 0 + Messages logged to disk: 4836834
+ Errors - Logging to disk: 0 + Errors - Invalid priority tag: 0 + Errors - No priority tag: 0 + Errors - Oversize message: 11
Well, syslog-ng has a global option stats_freq() -- see http://www.balabit.com/products/syslog_ng/reference-2.0/syslog-ng.html/index... Such a stats log message looks like this: --8<-- syslog-ng[7359]: Log statistics; processed='source(SrcName)=NumMsgs', processed='destination(DstName)=NumMgs' --8<-- If you have more than one source/destination, more ", processed=" entries are appended. Please keep in mind that if you have many sources/destinations (like I do), the stats log message is chopped after log_msg_size() bytes, which is 8192 by default! You could use those stats log messages to generate your own more comprehensive stats. You could even log those stats messages to a special logfile to process them specially, like this: --8<-- options { stats_freq(600); log_msg_size(16384); }; template t_stats { template("${YEAR}-${MONTH}-${DAY} ${HOUR}:${MIN}:${SEC} ${TZOFFSET}; ${MSG}\n"); template_escape(no); }; source s_local { unix-stream("/dev/log" max-connections(1000)); internal(); }; destination d_stats { file("/var/log/syslog-ng/stats.log" template(t_stats)); }; filter f_stats { facility("syslog"); priority("notice"); program("^syslog-ng"); match("Log statistics;"); }; log { source(s_local); filter(f_stats); destination(d_firewall); }; --8<-- Caution: this is untested -- I just hacked it together. -- Regards, Wolfram Schlich <wschlich@gentoo.org> Gentoo Linux * http://dev.gentoo.org/~wschlich/
Wolfram Schlich wrote:
Well, syslog-ng has a global option stats_freq() -- see http://www.balabit.com/products/syslog_ng/reference-2.0/syslog-ng.html/index...
Such a stats log message looks like this: --8<-- syslog-ng[7359]: Log statistics; processed='source(SrcName)=NumMsgs', processed='destination(DstName)=NumMgs' --8<-- If you have more than one source/destination, more ", processed=" entries are appended.
Please keep in mind that if you have many sources/destinations (like I do), the stats log message is chopped after log_msg_size() bytes, which is 8192 by default!
This behaviour is going to be changed in the future, there was a discussion about the possible solutions already. The current proposal is to create a separate logfile which will be simply appended, so the statistic information (and maybe other internal messages) won't be handled as the other logs.
You could use those stats log messages to generate your own more comprehensive stats.
You could even log those stats messages to a special logfile to process them specially, like this:
--8<-- options { stats_freq(600); log_msg_size(16384); };
template t_stats { template("${YEAR}-${MONTH}-${DAY} ${HOUR}:${MIN}:${SEC} ${TZOFFSET}; ${MSG}\n"); template_escape(no); };
source s_local { unix-stream("/dev/log" max-connections(1000)); internal(); };
destination d_stats { file("/var/log/syslog-ng/stats.log" template(t_stats)); };
filter f_stats { facility("syslog"); priority("notice"); program("^syslog-ng"); match("Log statistics;"); };
log { source(s_local); filter(f_stats); destination(d_firewall); }; --8<--
Caution: this is untested -- I just hacked it together.
Good solution, however moving the internal() into separate source is more optimal, especially when the host syslog-ng is running on generates a lot of logs. -- Sandor Geller wildy@balabit.hu
* Sandor Geller <wildy@balabit.hu> [2006-06-19 14:24]:
Wolfram Schlich wrote:
Well, syslog-ng has a global option stats_freq() -- see http://www.balabit.com/products/syslog_ng/reference-2.0/syslog-ng.html/index...
Such a stats log message looks like this: --8<-- syslog-ng[7359]: Log statistics; processed='source(SrcName)=NumMsgs', processed='destination(DstName)=NumMgs' --8<-- If you have more than one source/destination, more ", processed=" entries are appended.
Please keep in mind that if you have many sources/destinations (like I do), the stats log message is chopped after log_msg_size() bytes, which is 8192 by default!
This behaviour is going to be changed in the future, there was a discussion about the possible solutions already. The current proposal is to create a separate logfile which will be simply appended, so the statistic information (and maybe other internal messages) won't be handled as the other logs.
Can you point me at the corresponding message(s)? A URL from http://news.gmane.org/gmane.comp.syslog-ng would be nice :-)
You could use those stats log messages to generate your own more comprehensive stats.
You could even log those stats messages to a special logfile to process them specially, like this:
--8<-- options { stats_freq(600); log_msg_size(16384); };
template t_stats { template("${YEAR}-${MONTH}-${DAY} ${HOUR}:${MIN}:${SEC} ${TZOFFSET}; ${MSG}\n"); template_escape(no); };
source s_local { unix-stream("/dev/log" max-connections(1000)); internal(); };
destination d_stats { file("/var/log/syslog-ng/stats.log" template(t_stats)); };
filter f_stats { facility("syslog"); priority("notice"); program("^syslog-ng"); match("Log statistics;"); };
log { source(s_local); filter(f_stats); destination(d_firewall); }; --8<--
Caution: this is untested -- I just hacked it together.
Good solution, however moving the internal() into separate source is more optimal, especially when the host syslog-ng is running on generates a lot of logs.
Good point! I overlooked that in my hurry :-) Thanks for pointing that out! So, it should be like this: --8<-- options { stats_freq(600); log_msg_size(16384); }; template t_stats { template("${YEAR}-${MONTH}-${DAY} ${HOUR}:${MIN}:${SEC} ${TZOFFSET}; ${MSG}\n"); template_escape(no); }; source s_internal { internal(); }; destination d_stats { file("/var/log/syslog-ng/stats.log" template(t_stats)); }; filter f_stats { facility("syslog"); priority("notice"); program("^syslog-ng"); match("Log statistics;"); }; log { source(s_internal); filter(f_stats); destination(d_firewall); }; --8<-- Happy testing ;) -- Regards, Wolfram Schlich <wschlich@gentoo.org> Gentoo Linux * http://dev.gentoo.org/~wschlich/
Wolfram Schlich wrote:
This behaviour is going to be changed in the future, there was a discussion about the possible solutions already. The current proposal is to create a separate logfile which will be simply appended, so the statistic information (and maybe other internal messages) won't be handled as the other logs.
Can you point me at the corresponding message(s)? A URL from http://news.gmane.org/gmane.comp.syslog-ng would be nice :-)
There aren't written messages. The conversion took place between Bazsi and myself and the decision is up to Bazsi as he is the syslog-ng developer. I'm sure that he will make his comments about this topic when he returns from vacation. -- Sandor Geller wildy@balabit.hu
participants (3)
-
Heinemann, Joerg (SBI Ruhr)
-
Sandor Geller
-
Wolfram Schlich