[syslog-ng] Log statistics

[Balazs Scheidler]

On Fri, 2010-09-10 at 10:49 +0200, Stefan Jakobs wrote:
> Hello list,
> 
> I'm using syslog-ng 3.1.1 on a single user system. I'm getting these 
> statistics in my logfile:
> 
> syslog-ng[1653]: Log statistics; processed='destination(newsnotice)=0', 
> processed='destination(firewall)=1', processed='destination(newscrit)=0', 
> processed='src.internal(src#0)=2', stamp='src.internal(src#0)=1284066773', 
> processed='center(queued)=0', processed='destination(cron)=15', 
> processed='destination(acpid)=0', processed='destination(d_freshclam)=13', 
> processed='destination(messages)=175', 
> processed='destination(localmessages)=24', 
> processed='global(payload_reallocs)=0', processed='global(msg_clones)=0', 
> processed='destination(d_clamd)=36', processed='destination(console)=13', 
> processed='destination(mail)=52', processed='destination(netmgm)=0', 
> processed='destination(xconsole)=13', processed='destination(warn)=16', 
> processed='destination(newserr)=0', processed='source(chroots)=10', 
> processed='global(sdata_updates)=0', processed='destination(mailerr)=0', 
> processed='center(received)=0', processed='destination(devnull)=0', 
> processed='destination(mailwarn)=1', processed='source(src)=242', 
> processed='destination(mailinfo)=2'
> 
> I'm wondering why processed='center(received) and processed='center(queued) 
> are zero. The documentation says: "The center(received) entry shows the total 
> number of messages received from every configured sources." Any ideas?

I've got an idea. The reorganization to a tree-like log processing
structure (you know the embedded log statement stuff) basically removed
the "log center" idea, and this didn't get follow up in the statistics
stuff, nor the documentation.

So right now those counters do not exist, and albeit it'd be possible to
reimplement them, it is not that trivial.

A global receiver counter would be still easy, but the other one isn't.

> 
> Is it still true that center(received)=sum(source(*))+sum(src.internal(*))? 

if center(received) was working, then yes, those should be equal. the
"center" used to be the mass dispatching mechanism in syslog-ng, but as
I said earlier it got removed.

but with the formula above, you could recalculate it.

> In 
> this example: center(received)=src.internal(src#0)+source(src).
> And is this also right: center(queued)=sum(destination(*))?
> 
> And what is the meaning of 
> stamp='src.internal(src#0)=1284066773', 

the UNIX timestamp of last time a message was received on this source.

> processed='global(payload_reallocs)=0',
> processed='global(msg_clones)=0' and
> processed='global(sdata_updates)=0'

these are internal counters I was interested in how they would count in
the field. they don't really have too much functionality, but this data
is useful when tuning syslog-ng. Since all those counters are 0, it
means that I've more-or-less figured the default sizing of some data
structures inside syslog-ng allright.

* payload_reallocs: counts the number of times the NVTable holding
name-value pairs gets reallocated because it gets too small
* msg_clones: counts the number of times the LogMessage structure is
cloned because of rewrite operations
* sdata_updates: counts the number of times an SData value is changed
because of a rewrite rule.

> 
> These are a lot of questions, I know. Would be nice if someone can help out.

Hope this helps.

-- 
Bazsi