[syslog-ng] Per-thread scratch buffers

[Gergely Nagy]

Balazs Scheidler <bazsi at balabit.hu> writes:

>> As a first step in implementing this, I had something like the following
>> in mind:
>> 
>> static GTrashStack *scratch_buffers;
>> 
>> GString *scratch_buffer_acquite (void)
>
> a typo in the function name.

Aye, these were written without any kind of testing. I just looked at
the glib docs, found GTrashStack, and typed in a few functions.

>> void scratch_buffer_release (GString *s)
>> {
>>   g_trash_stack_push (s);
>> }
>> 
>> void scratch_buffers_reset (void)
>> {
>>   GString *s;
>> 
>>   while ((s = g_trash_stack_pop (scratch_buffers)) != NULL)
>>     g_string_free (s, TRUE);
>> }
>
> I wouldn't want to free the GString instances, but reuse them. This is
> the other point: to avoid having to allocate new GString instances all
> the time.

Well, in my implementation, GStrings are allocated on-demand, and free'd
when a thread exits. They're not kept around for use between threads.

I can do that, but that will involve some locking.

> The implementation I had in mind was:
>   - dynamically sized array of GString instances (per-thread)
>   - a stack pointer that contains the index within the array where we
>     are at. Items below the pointer are used, above are not.
>   - acquire: if there are unused items in the array, return it and
>     increase the pointer. if there are no unused items, allocate new and
>     store it in the array.
>   - release: if the item to be freed is the one handed out last,
>     decrement the pointer, else do nothing.
>   - reset: reset the pointer to 0.


> This way: 
>   - the number of GString instances depend on their usage.
>   - GStrings are only allocated rarely, if the parallel number of
>     GString buffers increase for some reason.
>   - no need for locking as scratch buffers are per-thread

I think my implementation achieves these goals too, just a bit
differently. Instead of a dynamically sized array, I use a stack, and
leave the administration stuff to the callers.

The stack-way has the advantage of not having to resize an array, nor
does it have to do any bookkeeping.

The disadvantage is, that if a caller forgets to release a string
instance, that will be lost forever. While in the case of an array-based
implementation, we could free these leaked strings too.

As for reset: I don't really see the use for resetting the pointer to
0. That's basically a give back everything thingy, which might be handy,
but I prefer giving back one by one instead: it makes the flow more
understandable, and involves less magic.

> Of course it only shows its potential if all the similar cases already
> in the code are converted to scratch buffers (log_template_format,
> template functions, value-pairs, etc).

I already converted jsonparser & value-pairs (and thus
format-json). Will have a stab at the rest once we agreed on how scratch
buffers should work :)

-- 
|8]