[syslog-ng] [PATCH 0/3] WIP: Convert syslog-ng to use upstream ivykis

[Lennert Buytenhek]

On Mon, May 14, 2012 at 12:03:07PM +0200, Gergely Nagy wrote:

> > Having done that, I indeed see a constructor function in libtest.c
> > being called both before main() and when libplugin.so is dlopen()ed,
> > i.e. it's called twice like what you report above.
> >
> > But then, if I define a static variable in libtest.c, and print its
> > address in that constructor function, I get different addresses for
> > each of the invocations of the constructor.  I.e. the libtest.a linked
> > into tst.c is using a different set of static variables than the
> > libtest.a linked into libplugin.so.
> >
> > If the same happens in your setup, I don't see how we can fix this
> > except building libivykis as a .so instead of as a .a -- but if
> > you're not seeing this, I'd love to know what I'm doing wrong...
> 
> I'm seeing something else, I think. I digged a bit further, and found
> that if I place a break on iv_tls_user_register, I get this when
> syslog-ng starts up:
> 
> (gdb) bt
> #0  iv_tls_user_register (itu=0x7ffff7ddb940) at ../../../../lib/ivykis/lib/iv_tls.c:34
> #1  0x00007ffff7ba0e7c in iv_event_tls_init () at ../../../../lib/ivykis/modules/iv_event.c:98
> #2  0x00007ffff7deacd0 in ?? () from /lib64/ld-linux-x86-64.so.2
> #3  0x00007ffff7deadc7 in ?? () from /lib64/ld-linux-x86-64.so.2
> #4  0x00007ffff7dddb2a in ?? () from /lib64/ld-linux-x86-64.so.2
> (gdb) p &inited
> $3 = (int *) 0x7ffff7ddcaf0
> (gdb) p inited
> $4 = 0
> 
> Hitting continue a few times, so that every ivykis module registers
> their tls stuff, I end up with:
> 
> (gdb) bt
> #0  iv_tls_user_register (itu=0x7ffff59f9b20) at ../../../../lib/ivykis/lib/iv_tls.c:34
> #1  0x00007ffff57f20d0 in iv_event_tls_init () at ../../../../lib/ivykis/modules/iv_event.c:98
> #2  0x00007ffff7deacd0 in ?? () from /lib64/ld-linux-x86-64.so.2
> #3  0x00007ffff7deadc7 in ?? () from /lib64/ld-linux-x86-64.so.2
> #4  0x00007ffff7def0b3 in ?? () from /lib64/ld-linux-x86-64.so.2
> #5  0x00007ffff7dea926 in ?? () from /lib64/ld-linux-x86-64.so.2
> #6  0x00007ffff7dee89a in ?? () from /lib64/ld-linux-x86-64.so.2
> #7  0x00007ffff67c7f66 in dlopen_doit (a=<optimized out>) at dlopen.c:67
> #8  0x00007ffff7dea926 in ?? () from /lib64/ld-linux-x86-64.so.2
> #9  0x00007ffff67c82ec in _dlerror_run (operate=0x7ffff67c7f00 <dlopen_doit>, args=0x7fffffff89f0) at dlerror.c:164
> #10 0x00007ffff67c7ee1 in __dlopen (file=<optimized out>, mode=<optimized out>) at dlopen.c:88
> #11 0x00007ffff7714944 in g_module_open () from /usr/lib/x86_64-linux-gnu/libgmodule-2.0.so.0
> #12 0x00007ffff7b7c440 in plugin_dlopen_module (module_name=0x63df10 "afsocket", module_path=0x617a50 "/home/algernon/install/syslog-ng/ose-3.4/lib/syslog-ng")
>     at ../../lib/plugin.c:305
> #13 0x00007ffff7b7c592 in plugin_load_module (module_name=0x63df10 "afsocket", cfg=0x616930, args=0x617c80) at ../../lib/plugin.c:349
> #14 0x00007ffff7b979c9 in pragma_parse (lexer=0x61c530, result=0x7fffffffaff8, arg=0x0) at pragma-grammar.y:397
> #15 0x00007ffff7b5300b in cfg_parser_parse (self=0x7ffff7dda8a0, lexer=0x61c530, instance=0x7fffffffaff8, arg=0x0) at ../../lib/cfg-parser.h:83
> #16 0x00007ffff7b54bae in cfg_lexer_lex (self=0x61c530, yylval=0x7fffffffd3e0, yylloc=0x7fffffffd3c0) at ../../lib/cfg-lexer.c:733
> #17 0x00007ffff7b55b7b in main_lex (yylval=0x7fffffffd3e0, yylloc=0x7fffffffd3c0, lexer=0x61c530) at ../../lib/cfg-parser.c:164
> #18 0x00007ffff7b8fda3 in main_parse (lexer=0x61c530, dummy=0x7fffffffd598, arg=0x0) at cfg-grammar.c:3153
> #19 0x00007ffff7b51d78 in cfg_parser_parse (self=0x7ffff7dd9a00, lexer=0x61c530, instance=0x7fffffffd598, arg=0x0) at ../../lib/cfg-parser.h:83
> #20 0x00007ffff7b529c1 in cfg_run_parser (self=0x616930, lexer=0x61c530, parser=0x7ffff7dd9a00, result=0x7fffffffd598, arg=0x0) at ../../lib/cfg.c:316
> #21 0x00007ffff7b52add in cfg_read_config (self=0x616930, fname=0x608050 "etc/mongodb.conf", syntax_only=0, preprocess_into=0x0) at ../../lib/cfg.c:347
> #22 0x00007ffff7b75ff3 in main_loop_init () at ../../lib/mainloop.c:677
> #23 0x0000000000401994 in main (argc=1, argv=0x7fffffffd708) at ../../syslog-ng/main.c:239
> (gdb) p &inited
> $13 = (int *) 0x7ffff7ddcaf0
> (gdb) p inited
> $14 = 1
> 
> inited has the same address,

OK, so that's where our test results diverge -- in my test app, a gdb
'p &mylocalvar' also prints different addresses (for a static global
variable).


> but if you notice, the iv_event_tls_init()
> that ends up being called is 0x00007ffff7ba0e7c in one case, and
> 0x00007ffff57f20d0 in the other. On the other hand,
> iv_tls_user_register() ends up being the same (verified that by
> disassembling from the breakpoint). I'm not sure this is relevant,
> though...

(Probably also because iv_event_tls_init is static and
iv_tls_user_register is not..)


> The iv_event_tls_user structures are different aswell in these two
> cases, which is expected, since they're static.
> 
> Truth be told, at this point, I'm leaning towards ignoring the whole
> problem, and not linking dlopen()-able stuff against ivykis if ivykis is
> pulled in in another way. In that case the whole problem goes away, and
> as far as I checked, will work reliably in all cases:
> 
>  - libsyslog-ng.so with libivykis.a statically linked in: the symbols
>    are exported, so dlopen()'d modules are able to use it - OK!
>  - libsyslog-ng.so linked dynamically to libivykis.so: the symbols are
>    available to dlopen()'d modules, so all is well - OK!

That sounds okay to me, then.

In case of a static ivykis, I guess it makes the most sense,
conceptually, to consider the ivykis code to be part of one of
the shared objects, and to provide the ivykis API from that shared
object.  What you argue for is basically that, so I think we're in
agreement.


> And that's the only two cases syslog-ng supports, and what anyone would
> normally support, imo. Statically linking a dlopened module to a library
> the main lib/binary is linked to will almost always result in
> compatibility issues, as that would potentially allow for things like
> syslog-ng linked with ivykis version Y, dlopening a module linked to
> ivykis version X. All hell would break loose.
>
> It's probably better to abort right at the beginning, as a sign that
> someone somewhere did some bad linkage.

(Wouldn't symbol versioning help there, too?)