On Sat, 2011-10-29 at 13:38 +0200, Balazs Scheidler wrote:
This is cool stuff, but again doesn't compile with a stock json-c.
Hmm.. this time I've checked, newer json-c versions do install the referenced headers. Then I guess it's it needs an updated version number in configure.in.
I've integrated this, but I may have been a bit too fast handed. There's no locking in json-parser(), which will cause things to go bad if threaded(yes) is enabled in the configuration. And while looking further, $(format-json) is not either, as I've noticed that value-pairs() is not thread safe either, because it uses a similar GString instance that can be used from multiple threads. Can you look into making them thread safe? One way to do that is to allocate a new GString instance at every invocation, which is not nice as that increases malloc() load. The alternative idea I was playing with was to create a new API for managing per-thread scratch buffers that could be reused even between multiple independent portions of syslog-ng. The way it would work: - scratch buffers are a numbered array of GString buffers, allocated for _each_ thread. - as the message is processed various parts of syslog-ng allocate such buffers, do their thing with it. - in order to make it possible for a thread to work with scratch buffers, scratch_buffers_reset() would need to be called, once for each independent task (e.g. log message). GString *scratch_buffer_acquire() Allocate a new scratch buffer, used to hold a temp value, that _will_ automatically be freed in the future, when the current function call returns. The returned GString instance should not be saved to non-local variables. void scratch_buffer_release(GString *scratch) Free the specified scratch buffer to be useful further in the call chain. May not be needed at all, I guesstimate that processing a single message would need 4-8 scratch buffers, which get reset at the end of the current message processing. Probably complicated free-alloc mechanism is not needed. scratch_buffers_reset() Free all per-thread scratch buffers previously allocated. With this API, the required number of GString buffers would be allocated on-demand, but still they would not increase the malloc() load too much, as the scratch buffers subsystem can keep GString instances around. The code using GString buffers become simpler too, as the allocation to store it somewhere vanishes. What do you think? -- Bazsi