<div dir="ltr">Balazs, I hope you can answer the following questions:<div>1. Is that formula correct?<br><div style><b>SUM(`syslog-ng-ctl stats | grep source | grep processed`) == SUM(`syslog-ng-ctl stats | grep source | grep processed`)</b></div>
<div style><br></div><div style>If I'm not dropping any messages by 'flags(final)' without sending to any destination and also I'm not sending the same logs to multiple destinations.</div><div style><br></div>
<div style>Where SUM - summarize messages count in each line.</div><div style><br></div><div style>2. I don't need 'dst.name...' and 'src.name...' to get the summ, right?</div><div style>Destination = dst.name1 + dst.name2 + ..., is that correct?</div>
<div style><br></div><div style>3. What should I check if I see more 'source' processed messages than 'destination' and there are no dropped messages?</div><div style><br></div><div style>4. What other global things is it good to monitor using syslog stats?</div>
<div style>Are there are any methods to see if syslog-ng has stopped reading the source?</div><div style><br></div><div style>For now I'm going to monitor:</div><div style>1. Message/per second rate - alert if it decreased significantly</div>
<div style>2. The number of stored messages - alert if the queue is always full (log_fifo_size)</div><div style>3. The number of dropped messages</div><div style>4. Timestamps of the latest message - alert if the timestamp is tool old.</div>
<div style>... any other ideas?</div></div><div style><br></div><div style><br></div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Fri, Dec 28, 2012 at 12:57 PM, Anton Koldaev <span dir="ltr"><<a href="mailto:koldaevav@gmail.com" target="_blank">koldaevav@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Thanks Balazs for the explanation.<div>Looks like they all are not very important for me to monitor.</div>
</div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><br><div class="gmail_quote">On Fri, Dec 28, 2012 at 9:44 AM, Balazs Scheidler <span dir="ltr"><<a href="mailto:bazsi77@gmail.com" target="_blank">bazsi77@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><u></u>
<div>
<p></p><div>----- Original message -----
<br>> Thanks Daniel. I'm already using "syslog-ng-ctl stats" with different
<br>> stats levels. I'm just confused with "center()":
<br></div>> *% sudo syslog-ng-ctl stats | grep center
<br>>
<br>> *
<br>> *center;;received;a;processed;0*
<br>> *center;;queued;a;processed;0*
<br>
<br>center has been broken since 3.2, because there's no 'center' anymore. IIRC I have removed that counter since then. If there's a usecase for center I may give a thought how to revive it.
<br>
<br>> *
<br>> *
<br><div>> Also maybe balabit guys are able to give us some information on global
<br>> stats:
<br></div>> *% sudo syslog-ng-ctl stats | grep global *
<br>> *global;sdata_updates;;a;processed;0*
<br>
<br>this one counts the number of 'slow' updates to structured data entries during processing. being zero means that you basically never add sdata, but either not change it at all, or only change entries that are already present.
<br>
<br>> *global;payload_reallocs;;a;processed;760*
<br>
<br>this counts the number of reallocs of the message payload. syslog-ng sizes the allocated buffer with a simple heuristics in the hope that parsing, rewrite rules will not cause it to grow. in your case syslog-ng had to do a realloc for 760 messages. if this happens to be close to all messages you processed, it's the cause for performance degradation.
<br>
<br>if it's a minority then you probably don't have to care.
<br>
<br>if the first one is true, I'd like to know about it.
<br>
<br>right now the alllocated size is twice the length of the incoming message.
<br>
<br>
<br>> *global;msg_clones;;a;processed;0*
<br>>
<br>
<br>again a performance monitoring counter, that tracks the number of clone operations.
<br><div>
<br>>
<br>> On Thu, Dec 27, 2012 at 6:07 PM, Daniel Neubacher
<br>> <<a href="mailto:daniel.neubacher@xing.com" target="_blank">daniel.neubacher@xing.com</a>
<br>> > wrote:
<br>>
<br>> > I guess u need „syslog-ng-ctl stats”. But u need to configure
<br></div>> > “stats_level(2);” in your options to get this running nicely.****
<br><div>> >
<br>> > And if you want to get logs per second I’ve attached my little bash
<br></div>> > script: ****
<br>> >
<br>> > #!/bin/bash****
<br>> >
<br>> > ** **
<br>> >
<br>> > while true****
<br>> >
<br>> > do ****
<br><div>> >
<br>> > for i in $(syslog-ng-ctl stats | grep src.tcp | grep proc |
<br></div>> > cut -d ";" -f6) ****
<br>> >
<br>> > do ****
<br>> >
<br>> > let tc1+=$i ****
<br>> >
<br>> > done ****
<br>> >
<br>> > let lps=tc1-tc2****
<br>> >
<br>> > test -z $tc2 || echo $lps****
<br>> >
<br>> > tc2=$tc1****
<br>> >
<br>> > tc1=0****
<br>> >
<br>> > sleep 1****
<br>> >
<br>> > done****
<br>> >
<br>> > ** **
<br>> >
<br>> > *Von:* <a href="mailto:syslog-ng-bounces@lists.balabit.hu" target="_blank">syslog-ng-bounces@lists.balabit.hu</a> [mailto:
<br>> > <a href="mailto:syslog-ng-bounces@lists.balabit.hu" target="_blank">syslog-ng-bounces@lists.balabit.hu</a>] *Im Auftrag von *Anton Koldaev
<br>> > *Gesendet:* Donnerstag, 27. Dezember 2012 10:18
<br>> > *An:* Syslog-ng users' and developers' mailing list
<br>> > *Betreff:* [syslog-ng] Statistics summary****
<br>> >
<br>> > ** **
<br><div>> >
<br>> > Is there a nice way to get total numbers of processed/stored/dropped
<br></div>> > messages in syslog-ng v3.3?****
<br>> >
<br>> > ** **
<br>> >
<br>> > From docs:****
<br>> >
<br>> > > The *center(received)* entry shows the total number of messages
<br><div>> > received from every configured sources.
<br></div>> > ****
<br>> >
<br>> > ** **
<br><div>> >
<br>> > But in my stats center() shows zero almost for every host. I see zero
<br></div>> > in some other examples in docs too.****
<br>> >
<br>> > ** **
<br>> >
<br>> > --
<br>> > Best regards,
<br>> > Koldaev Anton ****
<br><div>> >
<br>> >
<br>> > ______________________________________________________________________________
<br>> > Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" target="_blank">https://lists.balabit.hu/mailman/listinfo/syslog-ng</a>
<br>> > Documentation:
<br>> > <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" target="_blank">http://www.balabit.com/support/documentation/?product=syslog-ng</a>
<br>> > FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" target="_blank">http://www.balabit.com/wiki/syslog-ng-faq</a>
<br>> >
<br>> >
<br>> >
<br>>
<br>>
<br>> --
<br>> Best regards,
<br>> Koldaev Anton
<br><br></div><p></p>
</div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br>Best regards,<br>Koldaev Anton
</div>
</div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br>Best regards,<br>Koldaev Anton
</div>