[syslog-ng] payload_reallocs twice the count number of messages.

[Evan Rempel]

Back in December there was discussion of the payload_reallocs statistic.

https://lists.balabit.hu/pipermail/syslog-ng/2012-December/019842.html


 >
 >> *global;payload_reallocs;;a;processed;760*
 >
 >this counts the number of reallocs of the message payload. syslog-ng sizes the allocated buffer
 >with a simple heuristics in the hope that parsing, rewrite rules will not cause it to grow.
 >in your case syslog-ng had to do a realloc for 760 messages. if this happens to be close to
 >all messages you processed, it's the cause for performance degradation.
 >
 >if it's a minority then you probably don't have to care.
 >
 >if the first one is true, I'd like to know about it.
 >
 >right now the allocated size is twice the length of the incoming message.
 >


Well, You wanted to know if this happens for nearly all of the messages required realloc

Syslog-ng OSE 3.3.7

The d_archive destination receives all of our messages;

global;payload_reallocs;;a;processed;61142004
destination;d_archive;;a;processed;31650382

about 15 seconds later

global;payload_reallocs;;a;processed;61197495
destination;d_archive;;a;processed;31680143

This means that for

# messages = 29761
# reallocs = 55491

or approximately 2 reallocs for each message.

We make heavy use of patternDB to apply meta data to messages,