[syslog-ng] Statistics summary

[Balazs Scheidler]

----- Original message -----
> > > *global;payload_reallocs;;a;processed;760*
>
> > this counts the number of reallocs of the message payload. syslog-ng
> sizes the allocated buffer with a simple heuristics in the hope that
> parsing, rewrite rules will not cause it to grow. in your case syslog-ng
> had to do a realloc for 760 messages. if this happens to be close to all
> messages you processed, it's the cause for performance degradation.
>
> > if the first one is true, I'd like to know about it.
>
> At one of my clients hosts(I don't have stats monitoring for all of them
> yet) I have about 45 msg/sec sent by syslog-ng client to syslog-ng server
> and about 4msg/sec reallocated. Does it sound like a problem?
>

not immediate problem, I just would like to know your use-case. what are these messages and what you do to them.

I was thinking about a more complicated heuristic when I implemented this code, but then I decided to defer them until I saw actual uses where something better is needed

>
>
> On Sun, Dec 30, 2012 at 11:31 PM, Anton Koldaev <koldaevav at gmail.com>
> wrote:
>
> > Balazs, I hope you can answer the following questions:
> > 1. Is that formula correct?
> > *SUM(`syslog-ng-ctl stats | grep source | grep processed`) ==
> > SUM(`syslog-ng-ctl stats | grep source | grep processed`)*
> >
> > If I'm not dropping any messages by 'flags(final)' without sending to
> > any destination and also I'm not sending the same logs to multiple
> > destinations.
> >
> > Where SUM - summarize messages count in each line.
> >
> > 2. I don't need 'dst.name...' and 'src.name...' to get the summ, right?
> > Destination = dst.name1 + dst.name2 + ..., is that correct?
> >
> > 3. What should I check if I see more 'source' processed messages than
> > 'destination' and there are no dropped messages?
> >
> > 4. What other global things is it good to monitor using syslog stats?
> > Are there are any methods to see if syslog-ng has stopped reading the
> > source?
> >
> > For now I'm going to monitor:
> > 1. Message/per second rate - alert if it decreased significantly
> > 2. The number of stored messages - alert if the queue is always full
> > (log_fifo_size)
> > 3. The number of dropped messages
> > 4. Timestamps of the latest message - alert if the timestamp is tool
> > old. ... any other ideas?
> >
> >
> >
> >
> > On Fri, Dec 28, 2012 at 12:57 PM, Anton Koldaev
> > <koldaevav at gmail.com>wrote:
> >
> > > Thanks Balazs for the explanation.
> > > Looks like they all are not very important for me to monitor.
> > >
> > >
> > > On Fri, Dec 28, 2012 at 9:44 AM, Balazs Scheidler
> > > <bazsi77 at gmail.com>wrote:
> > >
> > > > **
> > > >
> > > > ----- Original message -----
> > > > > Thanks Daniel. I'm already using "syslog-ng-ctl stats" with
> > > > > different stats levels. I'm just confused with "center()":
> > > > > *% sudo syslog-ng-ctl stats | grep center
> > > > >
> > > > > *
> > > > > *center;;received;a;processed;0*
> > > > > *center;;queued;a;processed;0*
> > > >
> > > > center has been broken since 3.2, because there's no 'center'
> > > > anymore. IIRC I have removed that counter since then. If there's a
> > > > usecase for center I may give a thought how to revive it.
> > > >
> > > > > *
> > > > > *
> > > > > Also maybe balabit guys are able to give us some information on
> > > > > global stats:
> > > > > *% sudo syslog-ng-ctl stats | grep global *
> > > > > *global;sdata_updates;;a;processed;0*
> > > >
> > > > this one counts the number of 'slow' updates to structured data
> > > > entries during processing. being zero means that you basically
> > > > never add sdata, but either not change it at all, or only change
> > > > entries that are already present.
> > > >
> > > > > *global;payload_reallocs;;a;processed;760*
> > > >
> > > > this counts the number of reallocs of the message payload.
> > > > syslog-ng sizes the allocated buffer with a simple heuristics in
> > > > the hope that parsing, rewrite rules will not cause it to grow. in
> > > > your case syslog-ng had to do a realloc for 760 messages. if this
> > > > happens to be close to all messages you processed, it's the cause
> > > > for performance degradation.
> > > >
> > > > if it's a minority then you probably don't have to care.
> > > >
> > > > if the first one is true, I'd like to know about it.
> > > >
> > > > right now the alllocated size is twice the length of the incoming
> > > > message.
> > > >
> > > >
> > > > > *global;msg_clones;;a;processed;0*
> > > > >
> > > >
> > > > again a performance monitoring counter, that tracks the number of
> > > > clone operations.
> > > >
> > > > >
> > > > > On Thu, Dec 27, 2012 at 6:07 PM, Daniel Neubacher
> > > > > <daniel.neubacher at xing.com
> > > > > > wrote:
> > > > >
> > > > > > I guess u need „syslog-ng-ctl stats”.   But u need to configure
> > > > > > “stats_level(2);” in your options to get this running
> > > > > > nicely.****
> > > > > >
> > > > > > And if you want to get logs per second I’ve attached my little
> > > > > > bash script: ****
> > > > > >
> > > > > > #!/bin/bash****
> > > > > >
> > > > > > ** **
> > > > > >
> > > > > > while true****
> > > > > >
> > > > > > do ****
> > > > > >
> > > > > > for i in $(syslog-ng-ctl stats | grep src.tcp | grep proc |
> > > > > > cut -d ";" -f6) ****
> > > > > >
> > > > > > do ****
> > > > > >
> > > > > > let tc1+=$i ****
> > > > > >
> > > > > > done ****
> > > > > >
> > > > > > let lps=tc1-tc2****
> > > > > >
> > > > > > test -z $tc2 || echo $lps****
> > > > > >
> > > > > > tc2=$tc1****
> > > > > >
> > > > > > tc1=0****
> > > > > >
> > > > > > sleep 1****
> > > > > >
> > > > > > done****
> > > > > >
> > > > > > ** **
> > > > > >
> > > > > > *Von:* syslog-ng-bounces at lists.balabit.hu [mailto:
> > > > > > syslog-ng-bounces at lists.balabit.hu] *Im Auftrag von *Anton
> > > > > > Koldaev *Gesendet:* Donnerstag, 27. Dezember 2012 10:18
> > > > > > *An:* Syslog-ng users' and developers' mailing list
> > > > > > *Betreff:* [syslog-ng] Statistics summary****
> > > > > >
> > > > > > ** **
> > > > > >
> > > > > > Is there a nice way to get total numbers of
> > > > > > processed/stored/dropped messages in syslog-ng v3.3?****
> > > > > >
> > > > > > ** **
> > > > > >
> > > > > > From docs:****
> > > > > >
> > > > > > > The *center(received)* entry shows the total number of
> > > > > > > messages
> > > > > > received from every configured sources.
> > > > > > ****
> > > > > >
> > > > > > ** **
> > > > > >
> > > > > > But in my stats center() shows zero almost for every host. I
> > > > > > see
> > > > zero
> > > > > > in some other examples in docs too.****
> > > > > >
> > > > > > ** **
> > > > > >
> > > > > > --
> > > > > > Best regards,
> > > > > > Koldaev Anton ****
> > > > > >
> > > > > >
> > > > > >
> > > > ______________________________________________________________________________
> > > >
> > > > > > Member info:
> > > > > > https://lists.balabit.hu/mailman/listinfo/syslog-ng
> > > > > > Documentation:
> > > > > > http://www.balabit.com/support/documentation/?product=syslog-ng
> > > > > > FAQ: http://www.balabit.com/wiki/syslog-ng-faq
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Best regards,
> > > > > Koldaev Anton
> > > >
> > > >
> > >
> > >
> > > --
> > > Best regards,
> > > Koldaev Anton
> > >
> >
> >
> >
> > --
> > Best regards,
> > Koldaev Anton
> >
>
>
>
> --
> Best regards,
> Koldaev Anton

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.balabit.hu/pipermail/syslog-ng/attachments/20121230/307a49d8/attachment.htm