Hello, Running with syslog-ng OSE 3.2.1 and wondering what exactly mean some of the parameters by STATS. Please find below an extract of the output we get (with the default stats_level) , hoping to receive some hints from you. global > payload_reallocs = ? global > msg_clones = ? global > sdata_updates = ? why center > received is 0 ? From OSE 3.2 manual, Chapter 14, it should be the "total number of messages received from every configured sources." BR, Ioan ### [root@myhost TOOLS]# /opt/syslog-ng/sbin/syslog-ng-ctl stats SourceName;SourceId;SourceInstance;State;Type;Number global;payload_reallocs;;a;processed;1643977 source;s_net;;a;processed;615301996 global;msg_clones;;a;processed;1230603992 destination;d_mysql1;;a;processed;247656 destination;d_mysql2;;a;processed;1354498 global;sdata_updates;;a;processed;0 center;;received;a;processed;0 center;;queued;a;processed;0 ###
On Fri, 2011-03-18 at 15:17 +0200, Ioan Indreias wrote:
Hello,
Running with syslog-ng OSE 3.2.1 and wondering what exactly mean some of the parameters by STATS.
Please find below an extract of the output we get (with the default stats_level) , hoping to receive some hints from you.
global > payload_reallocs = ?
The number of times the payload structure of a LogMessage needed to be reallocated. A high number of this probably means that you have a lot of name-value pairs, and the preallocation is not sufficient for your setup. It is not a big issue, however it causes the performance to be dropped somewhat. The heuristics to allocate a payload is: the length of the incoming message mulitplied by 2, and this is where all name-value pairs (MSG, PID, PROGRAM, and user-defined ones too) need to be stored. Each name-value pair has some overhead too. Basically if this is number is large, or even if it's non-zero, I'd like to know about it.
global > msg_clones = ?
The number of times a message needed to be cloned. messages need to be cloned in case a log path is changing the message (with a parser or rewrite rule for instance), since syslog-ng ensures that parallel branches of the log processing tree (root is the source, branches described by the log statements, forks is when the same message needs to go multiple directions) receives the same message.
global > sdata_updates = ?
Counts the number of times a new structured data member was added to a message, causing the SDATA array to be updated. This is a slower operation, therefore it is counted to make it possible to diagnose performance issues.
why center > received is 0 ? From OSE 3.2 manual, Chapter 14, it should be the "total number of messages received from every configured sources."
This is a bug, that I never got around to fixing yet.
BR, Ioan
### [root@myhost TOOLS]# /opt/syslog-ng/sbin/syslog-ng-ctl stats SourceName;SourceId;SourceInstance;State;Type;Number global;payload_reallocs;;a;processed;1643977 source;s_net;;a;processed;615301996 global;msg_clones;;a;processed;1230603992 destination;d_mysql1;;a;processed;247656 destination;d_mysql2;;a;processed;1354498 global;sdata_updates;;a;processed;0 center;;received;a;processed;0 center;;queued;a;processed;0 ### ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html
-- Bazsi
Hello Bazsi, Thank a lot for the info provided - I was happy that no dropped messages are reported - however I am now a bit worried about the payload_reallocs values. Could we do something to lower it? We have started a monitor script (in Cacti) and log the number of processed messages (received from the network) - the average for the last 3 days is 700 msg/sec with some peaks near 1000. Because I am expecting that these numbers will slowly increase I have started to check the STATS values. Just to complete my report, the average number of MySQL questions is near 150 qps. We are using 2 templates to route the MySQL queries to a pipe file and a "consumer" script which actually run the queries. Probably there are some rooms to optimize the configuration used, this list being a valuable source of information for us. Best regards, Ioan On Sun, Mar 20, 2011 at 8:32 PM, Balazs Scheidler <bazsi@balabit.hu> wrote:
On Fri, 2011-03-18 at 15:17 +0200, Ioan Indreias wrote:
Hello,
Running with syslog-ng OSE 3.2.1 and wondering what exactly mean some of the parameters by STATS.
Please find below an extract of the output we get (with the default stats_level) , hoping to receive some hints from you.
global > payload_reallocs = ?
The number of times the payload structure of a LogMessage needed to be reallocated. A high number of this probably means that you have a lot of name-value pairs, and the preallocation is not sufficient for your setup.
It is not a big issue, however it causes the performance to be dropped somewhat.
The heuristics to allocate a payload is: the length of the incoming message mulitplied by 2, and this is where all name-value pairs (MSG, PID, PROGRAM, and user-defined ones too) need to be stored. Each name-value pair has some overhead too.
Basically if this is number is large, or even if it's non-zero, I'd like to know about it.
global > msg_clones = ?
The number of times a message needed to be cloned. messages need to be cloned in case a log path is changing the message (with a parser or rewrite rule for instance), since syslog-ng ensures that parallel branches of the log processing tree (root is the source, branches described by the log statements, forks is when the same message needs to go multiple directions) receives the same message.
global > sdata_updates = ?
Counts the number of times a new structured data member was added to a message, causing the SDATA array to be updated. This is a slower operation, therefore it is counted to make it possible to diagnose performance issues.
why center > received is 0 ? From OSE 3.2 manual, Chapter 14, it should be the "total number of messages received from every configured sources."
This is a bug, that I never got around to fixing yet.
BR, Ioan
### [root@myhost TOOLS]# /opt/syslog-ng/sbin/syslog-ng-ctl stats SourceName;SourceId;SourceInstance;State;Type;Number global;payload_reallocs;;a;processed;1643977 source;s_net;;a;processed;615301996 global;msg_clones;;a;processed;1230603992 destination;d_mysql1;;a;processed;247656 destination;d_mysql2;;a;processed;1354498 global;sdata_updates;;a;processed;0 center;;received;a;processed;0 center;;queued;a;processed;0 ### ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html
-- Bazsi
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html
participants (2)
-
Balazs Scheidler
-
Ioan Indreias