[syslog-ng] [PATCH 0/3]: Scratch buffers for syslog-ng 3.4

Gergely Nagy algernon at balabit.hu
Mon Oct 31 12:25:24 CET 2011


Following this mail will come three patches:
 * One to implement scratch buffers
 * Another to convert jsonparser to scratch buffers
 * And a third to convert value-pairs

While the implementation works for me (with threaded(yes) and
threaded(no) aswell), and I believe it to be correct, there's one
thing I'm not quite sure about. But before I get there, let me try to
explain how this simple thing is supposed to work!

As Bazsi suggested, there are three methods: scratch_buffer_acquire(),
scratch_buffer_release(), and scratch_buffers_reset().

All of them work with a thread-local GTrashStack, because my idea was
that instead of trying to keep a list of available and used scratch
buffers, we'll just store them in a stack, and leave it up to the
scratch buffer users to return the buffers properly.

This makes the implementation super simple, at the cost of not holding
the hands of the scratch users. I believe that's an acceptable
compromise.

scratch_buffer_acquire() tries to pop an element from the stack, and
if there's none, it creates a new one, and returns that. It also takes
care of truncating the string to zero length.

scratch_buffer_release() simply pushes an element onto the stack.

scratch_buffers_reset() pops all elements from the stack, frees up
both the GString contained within them, and the scratch buffer itself.

Due to the way GTrashStack works, I introduced a new typedef:
ScratchBuffer. The first element is a GTrashStack, which users need
not care about, the second is the GString.

ScratchBuffers are supposed to be used something like this:

void
test_scratchbuffer (void)
{
  ScratchBuffer *sb = scratch_buffer_acquire();

  g_string_printf(sb->s, "Hello %s!", "World");

  /* Do something with sb->s */

  scratch_buffer_release (sb);
}

There's one other trick in the implementation, however, and this is
the one I'm unsure about: in order to avoid leaking madly, the
thread-local scratch buffers MUST be destroyed when a thread finishes.

This must also be done at the lowest level possible, to not litter the
code with scratch_buffers_reset() calls.

I did this with calling scratch_buffers_reset() at the end of
main_loop_io_worker_job_start(), after all the finish callbacks have
finished already.

My reading of the code was that this is the place where threads are
already finished, and they won't be needing scratch buffers anymore,
and they should have released all of them aswell.

All three patches are available on the feature/3.4/scratch-buffers
branch of my github tree[0] too. While this is for 3.4, they should
apply cleanly to 3.3 aswell (except for the jsonparser part, of
course).

 [0]: https://github.com/algernon/syslog-ng/tree/feature/3.4/scratch-buffers




More information about the syslog-ng mailing list