hi,

in that case it might make sense to add a parameter to control initial allocation size or create a heuristic to automatically adjust that.

did you notice performance issues? how many messages are you processing? did the cpu usage of syslog-ng increase dramatically at one point as you were adding more and more name-value pairs?

thanks for the info.

----- Original message -----
> Back in December there was discussion of the payload_reallocs statistic.
>
> https://lists.balabit.hu/pipermail/syslog-ng/2012-December/019842.html
>
>
>  >
>  >> *global;payload_reallocs;;a;processed;760*
>  >
>  >this counts the number of reallocs of the message payload. syslog-ng
> sizes the allocated buffer >with a simple heuristics in the hope that
> parsing, rewrite rules will not cause it to grow. >in your case
> syslog-ng had to do a realloc for 760 messages. if this happens to be
> close to >all messages you processed, it's the cause for performance
> degradation. > >if it's a minority then you probably don't have to care.
>  >
>  >if the first one is true, I'd like to know about it.
>  >
>  >right now the allocated size is twice the length of the incoming
> message. >
>
>
> Well, You wanted to know if this happens for nearly all of the messages
> required realloc
>
> Syslog-ng OSE 3.3.7
>
> The d_archive destination receives all of our messages;
>
> global;payload_reallocs;;a;processed;61142004
> destination;d_archive;;a;processed;31650382
>
> about 15 seconds later
>
> global;payload_reallocs;;a;processed;61197495
> destination;d_archive;;a;processed;31680143
>
> This means that for
>
> # messages = 29761
> # reallocs = 55491
>
> or approximately 2 reallocs for each message.
>
> We make heavy use of patternDB to apply meta data to messages,
>
>
>