Hi, I've tried to put a closure on how syslog-ng 3.2 and above is being linked. There were numerous problems as it started to get into Linux distributions. This patch documents the intent with --enable-mixed-linking and how that behaves wrt the executables & shared libraries that syslog-ng installs. commit f2c7f3ea16d81e82b79582d3f2fe3882add4eb23 Author: Balazs Scheidler <bazsi@balabit.hu> Date: Wed Feb 23 13:01:02 2011 +0100 configure.in: reorganize LIBS variables so that unit tests/programs can be linked in mixed mode too Previously if --enable-mixed-linking was specified, unit tests and other tools (like pdbtool) couldn't be linked properly. The situation with --enable-mixed-linking is: * the primary aim here is to have a syslog-ng executable that can be started early in the boot process, and not a set of syslog-ng executables that can be distributed to different hosts that lack the required dependencies. * syslog-ng contains all static libs required by libsyslog-ng.so * therefore libsyslog-ng.so cannot be used outside the syslog-ng binary (except if the executable is linked against the required libs) * tools like pdbtool will be linked dynamically * unit tests will be linked dynamically too This patch cleans up and documents the purpose of various _DEPS_LIBS variables and updates Makefile.am files to use the updated names. A static GLib detection bug was also fixed. Reported-By: Arkadiusz Miśkiewicz <arekm@maven.pl> Signed-off-by: Balazs Scheidler <bazsi@balabit.hu> A related change is that the shared library libsyslog-ng-patterndb.so that was introduced in 3.2.2 will be dropped. Also, the main syslog-ng executable is not linked against libssl/libcrypto on purpose, it is only used by the afsocket module (which implements the tcp/syslog destinations). Could anyone who is in charge of maintaining syslog-ng binaries in various distributions give this a test so 3.2.3 will compile everywhere out-of-the-box? Thanks in advance, -- Bazsi
On Feb 23 13:05, Balazs Scheidler wrote:
Hi,
I've tried to put a closure on how syslog-ng 3.2 and above is being linked. There were numerous problems as it started to get into Linux distributions. [...] Also, the main syslog-ng executable is not linked against libssl/libcrypto on purpose, it is only used by the afsocket module (which implements the tcp/syslog destinations).
Could anyone who is in charge of maintaining syslog-ng binaries in various distributions give this a test so 3.2.3 will compile everywhere out-of-the-box?
Sorry, but this won't work for the Cygwin distro. The changes don't take my patch from https://lists.balabit.hu/pipermail/syslog-ng/2011-February/016007.html into account. Corinna -- Corinna Vinschen Cygwin Project Co-Leader Red Hat
On Wed, 2011-02-23 at 13:57 +0100, Corinna Vinschen wrote:
On Feb 23 13:05, Balazs Scheidler wrote:
Hi,
I've tried to put a closure on how syslog-ng 3.2 and above is being linked. There were numerous problems as it started to get into Linux distributions. [...] Also, the main syslog-ng executable is not linked against libssl/libcrypto on purpose, it is only used by the afsocket module (which implements the tcp/syslog destinations).
Could anyone who is in charge of maintaining syslog-ng binaries in various distributions give this a test so 3.2.3 will compile everywhere out-of-the-box?
Sorry, but this won't work for the Cygwin distro. The changes don't take my patch from https://lists.balabit.hu/pipermail/syslog-ng/2011-February/016007.html into account.
I understand that, but pdbtool doesn't open libdbparser.so anymore and libsyslog-ng-patterndb.so has been removed and I link that into pdbtool statically. Rechecking the patch you referenced, the only important difference is that you mark the convenience library with noinst_ whereas my patch installs that publicly (which indeed is a bug). Am I missing something else? -- Bazsi
On Feb 23 14:11, Balazs Scheidler wrote:
On Wed, 2011-02-23 at 13:57 +0100, Corinna Vinschen wrote:
On Feb 23 13:05, Balazs Scheidler wrote:
Hi,
I've tried to put a closure on how syslog-ng 3.2 and above is being linked. There were numerous problems as it started to get into Linux distributions. [...] Also, the main syslog-ng executable is not linked against libssl/libcrypto on purpose, it is only used by the afsocket module (which implements the tcp/syslog destinations).
Could anyone who is in charge of maintaining syslog-ng binaries in various distributions give this a test so 3.2.3 will compile everywhere out-of-the-box?
Sorry, but this won't work for the Cygwin distro. The changes don't take my patch from https://lists.balabit.hu/pipermail/syslog-ng/2011-February/016007.html into account.
I understand that, but pdbtool doesn't open libdbparser.so anymore and libsyslog-ng-patterndb.so has been removed and I link that into pdbtool statically.
Rechecking the patch you referenced, the only important difference is that you mark the convenience library with noinst_ whereas my patch installs that publicly (which indeed is a bug).
Am I missing something else?
Well, yes. As outlined in https://lists.balabit.hu/pipermail/syslog-ng/2011-February/016006.html there are two problems: - You can't link against a static lib when creating a shared lib. That means, creating libdbparser.so will fail. My patch results in creating a shared libsyslog-ng-patterndb.so for linking libdbparser.so, and a static libsyslog-ng-patterndb.a for linking pdbtool. Marking both libsyslog-ng-patterndb as noinst is just to avoid that both convenience libs are installed. - The link order is important when creating binaries on the Windows platform. Since libsyslog-ng-patterndb.a reference symbols in ../../lib/libsyslog-ng.la, you must reorder them on the command line. Not ../../lib/libsyslog-ng.la libsyslog-ng-patterndb.a but libsyslog-ng-patterndb.a ../../lib/libsyslog-ng.la My patch fixes both problems. The result should work on all platforms. Corinna -- Corinna Vinschen Cygwin Project Co-Leader Red Hat
On Wed, 2011-02-23 at 14:31 +0100, Corinna Vinschen wrote:
On Feb 23 14:11, Balazs Scheidler wrote:
On Wed, 2011-02-23 at 13:57 +0100, Corinna Vinschen wrote:
On Feb 23 13:05, Balazs Scheidler wrote:
Hi,
I've tried to put a closure on how syslog-ng 3.2 and above is being linked. There were numerous problems as it started to get into Linux distributions. [...] Also, the main syslog-ng executable is not linked against libssl/libcrypto on purpose, it is only used by the afsocket module (which implements the tcp/syslog destinations).
Could anyone who is in charge of maintaining syslog-ng binaries in various distributions give this a test so 3.2.3 will compile everywhere out-of-the-box?
Sorry, but this won't work for the Cygwin distro. The changes don't take my patch from https://lists.balabit.hu/pipermail/syslog-ng/2011-February/016007.html into account.
I understand that, but pdbtool doesn't open libdbparser.so anymore and libsyslog-ng-patterndb.so has been removed and I link that into pdbtool statically.
Rechecking the patch you referenced, the only important difference is that you mark the convenience library with noinst_ whereas my patch installs that publicly (which indeed is a bug).
Am I missing something else?
Well, yes. As outlined in https://lists.balabit.hu/pipermail/syslog-ng/2011-February/016006.html there are two problems:
- You can't link against a static lib when creating a shared lib. That means, creating libdbparser.so will fail. My patch results in creating a shared libsyslog-ng-patterndb.so for linking libdbparser.so, and a static libsyslog-ng-patterndb.a for linking pdbtool. Marking both libsyslog-ng-patterndb as noinst is just to avoid that both convenience libs are installed.
hmm.. not even with -fPIC ? If I have to duplicate executable code into pdbtool anyway (because of static linking it), I'd rather not have a separate .so which is only loaded by an .so. If that's the case, I'd include the same source files into both targets (the .la one and pdbtool one)
- The link order is important when creating binaries on the Windows platform. Since libsyslog-ng-patterndb.a reference symbols in ../../lib/libsyslog-ng.la, you must reorder them on the command line. Not
../../lib/libsyslog-ng.la libsyslog-ng-patterndb.a
but
libsyslog-ng-patterndb.a ../../lib/libsyslog-ng.la
Ok, I've missed that, and changed my Makefile.
My patch fixes both problems. The result should work on all platforms.
Thanks for your efforts, and sorry for missing the point. Hopefully we can close this once and for all :) -- Bazsi
Am Mittwoch, 23. Februar 2011, 14:52:18 schrieb Balazs Scheidler:
On Wed, 2011-02-23 at 14:31 +0100, Corinna Vinschen wrote:
On Feb 23 14:11, Balazs Scheidler wrote:
On Wed, 2011-02-23 at 13:57 +0100, Corinna Vinschen wrote:
On Feb 23 13:05, Balazs Scheidler wrote:
Hi,
I've tried to put a closure on how syslog-ng 3.2 and above is being linked. There were numerous problems as it started to get into Linux distributions. [...] Also, the main syslog-ng executable is not linked against libssl/libcrypto on purpose, it is only used by the afsocket module (which implements the tcp/syslog destinations).
Could anyone who is in charge of maintaining syslog-ng binaries in various distributions give this a test so 3.2.3 will compile everywhere out-of-the-box?
Sorry, but this won't work for the Cygwin distro. The changes don't take my patch from https://lists.balabit.hu/pipermail/syslog-ng/2011-February/016007.htm l into account.
I understand that, but pdbtool doesn't open libdbparser.so anymore and libsyslog-ng-patterndb.so has been removed and I link that into pdbtool statically.
Rechecking the patch you referenced, the only important difference is that you mark the convenience library with noinst_ whereas my patch installs that publicly (which indeed is a bug).
Am I missing something else?
Well, yes. As outlined in https://lists.balabit.hu/pipermail/syslog-ng/2011-February/016006.html there are two problems:
- You can't link against a static lib when creating a shared lib. That
means, creating libdbparser.so will fail. My patch results in creating a shared libsyslog-ng-patterndb.so for linking libdbparser.so, and a static libsyslog-ng-patterndb.a for linking pdbtool. Marking both libsyslog-ng-patterndb as noinst is just to avoid that both convenience libs are installed.
hmm.. not even with -fPIC ? If I have to duplicate executable code into pdbtool anyway (because of static linking it), I'd rather not have a separate .so which is only loaded by an .so.
If that's the case, I'd include the same source files into both targets (the .la one and pdbtool one)
This seems to be the easiest way :-) Gruesse / Regards, Marius Tomaschewski <mt@suse.de> <mt@novell.com> -- Server Technologies Team, SUSE LINUX Products GmbH, Nuernberg; GF: Markus Rex; HRB 16746 (AG Nuernberg) GPG/PGP public key fingerprint: DF17 271A AD15 006A 5BB9 6C96 CA2F F3F7 373A 1CC0
On Feb 23 14:52, Balazs Scheidler wrote:
On Wed, 2011-02-23 at 14:31 +0100, Corinna Vinschen wrote:
On Feb 23 14:11, Balazs Scheidler wrote:
Am I missing something else?
Well, yes. As outlined in https://lists.balabit.hu/pipermail/syslog-ng/2011-February/016006.html there are two problems:
- You can't link against a static lib when creating a shared lib. That means, creating libdbparser.so will fail. My patch results in creating a shared libsyslog-ng-patterndb.so for linking libdbparser.so, and a static libsyslog-ng-patterndb.a for linking pdbtool. Marking both libsyslog-ng-patterndb as noinst is just to avoid that both convenience libs are installed.
hmm.. not even with -fPIC ? If I have to duplicate executable code into pdbtool anyway (because of static linking it), I'd rather not have a separate .so which is only loaded by an .so.
-fPIC and -fpic have no effect on Windows. The Windows loader doesn't support position independent code. So, no, it won't work either.
If that's the case, I'd include the same source files into both targets (the .la one and pdbtool one)
- The link order is important when creating binaries on the Windows platform. Since libsyslog-ng-patterndb.a reference symbols in ../../lib/libsyslog-ng.la, you must reorder them on the command line. Not
../../lib/libsyslog-ng.la libsyslog-ng-patterndb.a
but
libsyslog-ng-patterndb.a ../../lib/libsyslog-ng.la
Ok, I've missed that, and changed my Makefile.
My patch fixes both problems. The result should work on all platforms.
Thanks for your efforts, and sorry for missing the point. Hopefully we can close this once and for all :)
No worries. However, something else occured to me a couple of minutes ago. If you apply my patch, you must change noinst_LTLIBRARIES = libsyslog-ng-patterndb.la to lib_LTLIBRARIES = libsyslog-ng-patterndb.la If you keep it in noinst_LTLIBRARIES, the libdbparse.so module will be broken after installtion because it requires libsyslog-ng-patterndb.so to run. Sorry for forgetting that. There are two ways to fix this. - Move libsyslog-ng-patterndb.so to module_LTLIBRARIES and install it. - Don't generate libsyslog-ng-patterndb.so. Instead add the patterndb source files to libdbparser_la_SOURCES, along these lines: diff --git a/modules/dbparser/Makefile.am b/modules/dbparser/Makefile.am index b2ab700..f8a53f4 100644 --- a/modules/dbparser/Makefile.am +++ b/modules/dbparser/Makefile.am @@ -21,7 +21,8 @@ libsyslog_ng_patterndb_a_CFLAGS = $(AM_CFLAGS) -fPIC module_LTLIBRARIES = libdbparser.la libdbparser_la_SOURCES = \ dbparser.c dbparser.h \ - dbparser-grammar.y dbparser-parser.c dbparser-parser.h dbparser-plugin.c + dbparser-grammar.y dbparser-parser.c dbparser-parser.h \ + dbparser-plugin.c $(libsyslog_ng_patterndb_a_SOURCES) libdbparser_la_CPPFLAGS = $(AM_CPPFLAGS) libdbparser_la_LIBADD = ../../lib/libsyslog-ng.la libsyslog-ng-patterndb.a I think this second solution is better. Corinna -- Corinna Vinschen Cygwin Project Co-Leader Red Hat
On Feb 23 15:18, Corinna Vinschen wrote:
On Feb 23 14:52, Balazs Scheidler wrote:
hmm.. not even with -fPIC ? If I have to duplicate executable code into pdbtool anyway (because of static linking it), I'd rather not have a separate .so which is only loaded by an .so.
-fPIC and -fpic have no effect on Windows. The Windows loader doesn't support position independent code. So, no, it won't work either. [...] No worries. However, something else occured to me a couple of minutes ago. If you apply my patch, you must change
noinst_LTLIBRARIES = libsyslog-ng-patterndb.la
to
lib_LTLIBRARIES = libsyslog-ng-patterndb.la
If you keep it in noinst_LTLIBRARIES, the libdbparse.so module will be broken after installtion because it requires libsyslog-ng-patterndb.so to run. Sorry for forgetting that.
...and I'm even more sorry that we have to suffer so much from the restrictions of the underlying Windows system. :-| Corinna -- Corinna Vinschen Cygwin Project Co-Leader Red Hat
On Wed, 2011-02-23 at 15:18 +0100, Corinna Vinschen wrote:
On Feb 23 14:52, Balazs Scheidler wrote:
On Wed, 2011-02-23 at 14:31 +0100, Corinna Vinschen wrote:
On Feb 23 14:11, Balazs Scheidler wrote:
Am I missing something else?
Well, yes. As outlined in https://lists.balabit.hu/pipermail/syslog-ng/2011-February/016006.html there are two problems:
- You can't link against a static lib when creating a shared lib. That means, creating libdbparser.so will fail. My patch results in creating a shared libsyslog-ng-patterndb.so for linking libdbparser.so, and a static libsyslog-ng-patterndb.a for linking pdbtool. Marking both libsyslog-ng-patterndb as noinst is just to avoid that both convenience libs are installed.
hmm.. not even with -fPIC ? If I have to duplicate executable code into pdbtool anyway (because of static linking it), I'd rather not have a separate .so which is only loaded by an .so.
-fPIC and -fpic have no effect on Windows. The Windows loader doesn't support position independent code. So, no, it won't work either.
If that's the case, I'd include the same source files into both targets (the .la one and pdbtool one)
- The link order is important when creating binaries on the Windows platform. Since libsyslog-ng-patterndb.a reference symbols in ../../lib/libsyslog-ng.la, you must reorder them on the command line. Not
../../lib/libsyslog-ng.la libsyslog-ng-patterndb.a
but
libsyslog-ng-patterndb.a ../../lib/libsyslog-ng.la
Ok, I've missed that, and changed my Makefile.
My patch fixes both problems. The result should work on all platforms.
Thanks for your efforts, and sorry for missing the point. Hopefully we can close this once and for all :)
No worries. However, something else occured to me a couple of minutes ago. If you apply my patch, you must change
noinst_LTLIBRARIES = libsyslog-ng-patterndb.la
to
lib_LTLIBRARIES = libsyslog-ng-patterndb.la
If you keep it in noinst_LTLIBRARIES, the libdbparse.so module will be broken after installtion because it requires libsyslog-ng-patterndb.so to run. Sorry for forgetting that.
There are two ways to fix this.
- Move libsyslog-ng-patterndb.so to module_LTLIBRARIES and install it.
- Don't generate libsyslog-ng-patterndb.so. Instead add the patterndb source files to libdbparser_la_SOURCES, along these lines:
diff --git a/modules/dbparser/Makefile.am b/modules/dbparser/Makefile.am index b2ab700..f8a53f4 100644 --- a/modules/dbparser/Makefile.am +++ b/modules/dbparser/Makefile.am @@ -21,7 +21,8 @@ libsyslog_ng_patterndb_a_CFLAGS = $(AM_CFLAGS) -fPIC module_LTLIBRARIES = libdbparser.la libdbparser_la_SOURCES = \ dbparser.c dbparser.h \ - dbparser-grammar.y dbparser-parser.c dbparser-parser.h dbparser-plugin.c + dbparser-grammar.y dbparser-parser.c dbparser-parser.h \ + dbparser-plugin.c $(libsyslog_ng_patterndb_a_SOURCES)
libdbparser_la_CPPFLAGS = $(AM_CPPFLAGS) libdbparser_la_LIBADD = ../../lib/libsyslog-ng.la libsyslog-ng-patterndb.a
I think this second solution is better.
Before making a fool from myself, can you please check if this patch is OK, before commit & push? (against the current HEAD). My intention was: * create a static lib for linking with programs (unit tests & pdbtool) * recompile each source files as a part of the shared library separately This way: * no linking of static lib into shared lib happens Also, a heads up: this will be a problem again in syslog-ng OSE 3.3, where I started to embed dependent libraries into the syslog-ng tree (ivykis, libmongo-client) because those are not yet in widespread use. That's implemented using static libraries and -fPIC, and I don't see an easy solution for that problem, unless of course one obtains and installs those libraries separately. thanks. $ git diff diff --git a/modules/dbparser/Makefile.am b/modules/dbparser/Makefile.am index b2ab700..7e4c366 100644 --- a/modules/dbparser/Makefile.am +++ b/modules/dbparser/Makefile.am @@ -11,26 +11,30 @@ AM_CPPFLAGS = -I$(top_srcdir)/lib -I../../lib AM_CFLAGS = @CFLAGS_NOWARN_POINTER_SIGN@ export top_srcdir -lib_LIBRARIES = libsyslog-ng-patterndb.a +noinst_LIBRARIES = libsyslog-ng-patterndb.a libsyslog_ng_patterndb_a_SOURCES = radix.c radix.h \ patterndb.c patterndb.h patterndb-int.h \ timerwheel.c timerwheel.h \ patternize.c patternize.h -libsyslog_ng_patterndb_a_CFLAGS = $(AM_CFLAGS) -fPIC +# we're not linking against the .a file as this is not supported on Cygwin, +# even if we compile it using -fPIC. Therefore the .a file is only used to +# link programs, and the dbparser.so gets those files as source files, +# thereby recompiling them twice. module_LTLIBRARIES = libdbparser.la libdbparser_la_SOURCES = \ dbparser.c dbparser.h \ - dbparser-grammar.y dbparser-parser.c dbparser-parser.h dbparser-plugin.c + dbparser-grammar.y dbparser-parser.c dbparser-parser.h dbparser-plugin.c \ + $(libsyslog_ng_patterndb_a_SOURCES) libdbparser_la_CPPFLAGS = $(AM_CPPFLAGS) -libdbparser_la_LIBADD = ../../lib/libsyslog-ng.la libsyslog-ng-patterndb.a +libdbparser_la_LIBADD = ../../lib/libsyslog-ng.la libdbparser_la_LDFLAGS = -avoid-version -module -no-undefined bin_PROGRAMS = pdbtool pdbtool_SOURCES = pdbtool.c pdbtool_CPPFLAGS = $(AM_CPPFLAGS) @OPENSSL_CFLAGS@ -pdbtool_LDADD = ../../lib/libsyslog-ng.la libsyslog-ng-patterndb.a @TOOL_DEPS_LIBS@ @OPENSSL_LIBS@ +pdbtool_LDADD = libsyslog-ng-patterndb.a ../../lib/libsyslog-ng.la @TOOL_DEPS_LIBS@ @OPENSSL_LIBS@ BUILT_SOURCES = dbparser-grammar.y dbparser-grammar.c dbparser-grammar.h EXTRA_DIST = $(BUILT_SOURCES) radix-find.c dbparser-grammar.ym diff --git a/modules/dbparser/tests/Makefile.am b/modules/dbparser/tests/Makefile.am index 8d3a970..6726121 100644 --- a/modules/dbparser/tests/Makefile.am +++ b/modules/dbparser/tests/Makefile.am @@ -1,6 +1,6 @@ AM_CFLAGS = -I$(top_srcdir)/lib -I../../../lib -I$(top_srcdir)/modules/dbparser -I.. @CFLAGS_NOWARN_POINTER_SIGN@ AM_LDFLAGS = -dlpreopen ../../syslogformat/libsyslogformat.la -LDADD = $(top_builddir)/lib/libsyslog-ng.la ../libsyslog-ng-patterndb.a ../patternize.lo @TOOL_DEPS_LIBS@ @OPENSSL_LIBS@ +LDADD = $(top_builddir)/lib/libsyslog-ng.la ../libsyslog-ng-patterndb.a @TOOL_DEPS_LIBS@ @OPENSSL_LIBS@ check_PROGRAMS = test_timer_wheel test_patternize test_patterndb test_radix -- Bazsi
Am Mittwoch, 23. Februar 2011, 16:09:42 schrieb Balazs Scheidler: [...]
Before making a fool from myself, can you please check if this patch is OK, before commit & push? (against the current HEAD). [...]
Yes, works for me (11.3 & Factory). Gruesse / Regards, Marius Tomaschewski <mt@suse.de> <mt@novell.com> -- Server Technologies Team, SUSE LINUX Products GmbH, Nuernberg; GF: Markus Rex; HRB 16746 (AG Nuernberg) GPG/PGP public key fingerprint: DF17 271A AD15 006A 5BB9 6C96 CA2F F3F7 373A 1CC0
On Wed, 2011-02-23 at 16:29 +0100, Marius Tomaschewski wrote:
Am Mittwoch, 23. Februar 2011, 16:09:42 schrieb Balazs Scheidler: [...]
Before making a fool from myself, can you please check if this patch is OK, before commit & push? (against the current HEAD). [...]
Yes, works for me (11.3 & Factory).
Committed & pushed. Thanks everyone. -- Bazsi
On Feb 23 16:09, Balazs Scheidler wrote:
Before making a fool from myself, can you please check if this patch is OK, before commit & push? (against the current HEAD).
My intention was: * create a static lib for linking with programs (unit tests & pdbtool) * recompile each source files as a part of the shared library separately
This way: * no linking of static lib into shared lib happens
Also, a heads up: this will be a problem again in syslog-ng OSE 3.3, where I started to embed dependent libraries into the syslog-ng tree (ivykis, libmongo-client) because those are not yet in widespread use. That's implemented using static libraries and -fPIC, and I don't see an easy solution for that problem, unless of course one obtains and installs those libraries separately.
ivykis and libmongo-client are not available for Cygwin. Is there no way around using these libs by using conditional compilation? If that's not feasible for some reason, could you please at least allow to disable affected modules from building so that not the entire build fails just because some modules won't work on some systems?
$ git diff diff --git a/modules/dbparser/Makefile.am b/modules/dbparser/Makefile.am index b2ab700..7e4c366 100644 --- a/modules/dbparser/Makefile.am +++ b/modules/dbparser/Makefile.am @@ -11,26 +11,30 @@ AM_CPPFLAGS = -I$(top_srcdir)/lib -I../../lib AM_CFLAGS = @CFLAGS_NOWARN_POINTER_SIGN@ export top_srcdir
-lib_LIBRARIES = libsyslog-ng-patterndb.a +noinst_LIBRARIES = libsyslog-ng-patterndb.a libsyslog_ng_patterndb_a_SOURCES = radix.c radix.h \ patterndb.c patterndb.h patterndb-int.h \ timerwheel.c timerwheel.h \ patternize.c patternize.h -libsyslog_ng_patterndb_a_CFLAGS = $(AM_CFLAGS) -fPIC
+# we're not linking against the .a file as this is not supported on Cygwin, +# even if we compile it using -fPIC. Therefore the .a file is only used to +# link programs, and the dbparser.so gets those files as source files, +# thereby recompiling them twice. module_LTLIBRARIES = libdbparser.la libdbparser_la_SOURCES = \ dbparser.c dbparser.h \ - dbparser-grammar.y dbparser-parser.c dbparser-parser.h dbparser-plugin.c + dbparser-grammar.y dbparser-parser.c dbparser-parser.h dbparser-plugin.c \ + $(libsyslog_ng_patterndb_a_SOURCES)
libdbparser_la_CPPFLAGS = $(AM_CPPFLAGS) -libdbparser_la_LIBADD = ../../lib/libsyslog-ng.la libsyslog-ng-patterndb.a +libdbparser_la_LIBADD = ../../lib/libsyslog-ng.la
This fails to build on Cygwin because libdbparser_la_LIBADD is missing a @OPENSSL_LIBS@ (needed by patternize.c). Amazing but true: Another restriction of the Windows platform is the inability to create shared libs with undefined symbols. All symbols must be resolvable at link time.
libdbparser_la_LDFLAGS = -avoid-version -module -no-undefined
bin_PROGRAMS = pdbtool pdbtool_SOURCES = pdbtool.c pdbtool_CPPFLAGS = $(AM_CPPFLAGS) @OPENSSL_CFLAGS@ -pdbtool_LDADD = ../../lib/libsyslog-ng.la libsyslog-ng-patterndb.a @TOOL_DEPS_LIBS@ @OPENSSL_LIBS@ +pdbtool_LDADD = libsyslog-ng-patterndb.a ../../lib/libsyslog-ng.la @TOOL_DEPS_LIBS@ @OPENSSL_LIBS@
BUILT_SOURCES = dbparser-grammar.y dbparser-grammar.c dbparser-grammar.h EXTRA_DIST = $(BUILT_SOURCES) radix-find.c dbparser-grammar.ym diff --git a/modules/dbparser/tests/Makefile.am b/modules/dbparser/tests/Makefile.am index 8d3a970..6726121 100644 --- a/modules/dbparser/tests/Makefile.am +++ b/modules/dbparser/tests/Makefile.am @@ -1,6 +1,6 @@ AM_CFLAGS = -I$(top_srcdir)/lib -I../../../lib -I$(top_srcdir)/modules/dbparser -I.. @CFLAGS_NOWARN_POINTER_SIGN@ AM_LDFLAGS = -dlpreopen ../../syslogformat/libsyslogformat.la -LDADD = $(top_builddir)/lib/libsyslog-ng.la ../libsyslog-ng-patterndb.a ../patternize.lo @TOOL_DEPS_LIBS@ @OPENSSL_LIBS@ +LDADD = $(top_builddir)/lib/libsyslog-ng.la ../libsyslog-ng-patterndb.a @TOOL_DEPS_LIBS@ @OPENSSL_LIBS@
I didn't test that, but shouldn't this LDADD be reordered as well, just as in the pdbtool_LDADD case? LDADD = ../libsyslog-ng-patterndb.a $(top_builddir)/lib/libsyslog-ng.la @TOOL_DEPS_LIBS@ @OPENSSL_LIBS@ Btw., I'm seeing a lot of foo.y: warning: X nonterminals useless in grammar foo.y: warning: Y rules useless in grammar messages. Aren't the grep calls in build/lex-rules.am wrong? grep -E -v "warning: (nonterminals|rules) useless in grammar instead of grep -E -v "warning: (nonterminal|rule) useless in grammar ? Or does the message depend on the version of yacc/bison used, perhaps? Thanks, Corinna -- Corinna Vinschen Cygwin Project Co-Leader Red Hat
On Feb 23 17:29, Corinna Vinschen wrote:
On Feb 23 16:09, Balazs Scheidler wrote:
-libdbparser_la_LIBADD = ../../lib/libsyslog-ng.la libsyslog-ng-patterndb.a +libdbparser_la_LIBADD = ../../lib/libsyslog-ng.la
This fails to build on Cygwin because libdbparser_la_LIBADD is missing a @OPENSSL_LIBS@ (needed by patternize.c).
Erm... btw., is it really necessary to link against OpenSSL, just to get the RAND_bytes function? What about testing for /dev/random and using that instead? That wouldn't introduce any external dependencies. Corinna -- Corinna Vinschen Cygwin Project Co-Leader Red Hat
On Wed, 2011-02-23 at 17:35 +0100, Corinna Vinschen wrote:
On Feb 23 17:29, Corinna Vinschen wrote:
On Feb 23 16:09, Balazs Scheidler wrote:
-libdbparser_la_LIBADD = ../../lib/libsyslog-ng.la libsyslog-ng-patterndb.a +libdbparser_la_LIBADD = ../../lib/libsyslog-ng.la
This fails to build on Cygwin because libdbparser_la_LIBADD is missing a @OPENSSL_LIBS@ (needed by patternize.c).
Erm... btw., is it really necessary to link against OpenSSL, just to get the RAND_bytes function? What about testing for /dev/random and using that instead? That wouldn't introduce any external dependencies.
Well, yes. But /dev/random is not available on all platforms that syslog-ng uses (Solaris 8 for example), and libssl nicely wraps that away, not to mention that it supports EGD (entropy gathering daemon) and hand-crafted PRNG seeding via the RANDFILE environment variable. I simply had no time for properly writing/validating an alternative solution and syslog-ng depends on openssl in most cases anyway. (if TLS support is enabled). But IIRC patternize.c will compile without openssl installed (OPENSSL_LIBS will be empty), in that case the UUIDs generated will not be guaranteed to be unique. -- Bazsi
On Feb 24 10:22, Balazs Scheidler wrote:
On Wed, 2011-02-23 at 17:35 +0100, Corinna Vinschen wrote:
On Feb 23 17:29, Corinna Vinschen wrote:
On Feb 23 16:09, Balazs Scheidler wrote:
-libdbparser_la_LIBADD = ../../lib/libsyslog-ng.la libsyslog-ng-patterndb.a +libdbparser_la_LIBADD = ../../lib/libsyslog-ng.la
This fails to build on Cygwin because libdbparser_la_LIBADD is missing a @OPENSSL_LIBS@ (needed by patternize.c).
Erm... btw., is it really necessary to link against OpenSSL, just to get the RAND_bytes function? What about testing for /dev/random and using that instead? That wouldn't introduce any external dependencies.
Well, yes. But /dev/random is not available on all platforms that syslog-ng uses (Solaris 8 for example), and libssl nicely wraps that away, not to mention that it supports EGD (entropy gathering daemon) and hand-crafted PRNG seeding via the RANDFILE environment variable.
I simply had no time for properly writing/validating an alternative solution and syslog-ng depends on openssl in most cases anyway. (if TLS support is enabled).
But IIRC patternize.c will compile without openssl installed (OPENSSL_LIBS will be empty), in that case the UUIDs generated will not be guaranteed to be unique.
You're right. Other parts of syslog-ng will use OpenSSL anyway, so there's no good reason to avoid it's usage if it's available. Corinna -- Corinna Vinschen Cygwin Project Co-Leader Red Hat
Hi, On Wed, 2011-02-23 at 17:29 +0100, Corinna Vinschen wrote:
On Feb 23 16:09, Balazs Scheidler wrote:
Before making a fool from myself, can you please check if this patch is OK, before commit & push? (against the current HEAD).
My intention was: * create a static lib for linking with programs (unit tests & pdbtool) * recompile each source files as a part of the shared library separately
This way: * no linking of static lib into shared lib happens
Also, a heads up: this will be a problem again in syslog-ng OSE 3.3, where I started to embed dependent libraries into the syslog-ng tree (ivykis, libmongo-client) because those are not yet in widespread use. That's implemented using static libraries and -fPIC, and I don't see an easy solution for that problem, unless of course one obtains and installs those libraries separately.
ivykis and libmongo-client are not available for Cygwin. Is there no way around using these libs by using conditional compilation?
There's a way, mongodb can be disabled using --disable-mongodb configure option (but is enabled by default with the bundled libmongo-client library). ivykis is not so simple, syslog-ng core depends on that heavily. ivykis is compiled as a static library (with -fPIC) and then linked into libsyslog-ng.so The staticlib is just a convenience library, however I'd like that to be created by the ivykis configure/automake stuff, and would like to avoid having to rewrite the build system for ivykis. Also, I wouldn't want to install ivykis (as a shared object) from the syslog-ng tree, as I could perhaps break people's setup if ivykis becomes more widely available. (e.g. installed on the system). Do you perhaps have an idea how to solve that differently?
If that's not feasible for some reason, could you please at least allow to disable affected modules from building so that not the entire build fails just because some modules won't work on some systems?
$ git diff diff --git a/modules/dbparser/Makefile.am b/modules/dbparser/Makefile.am index b2ab700..7e4c366 100644 --- a/modules/dbparser/Makefile.am +++ b/modules/dbparser/Makefile.am @@ -11,26 +11,30 @@ AM_CPPFLAGS = -I$(top_srcdir)/lib -I../../lib AM_CFLAGS = @CFLAGS_NOWARN_POINTER_SIGN@ export top_srcdir
-lib_LIBRARIES = libsyslog-ng-patterndb.a +noinst_LIBRARIES = libsyslog-ng-patterndb.a libsyslog_ng_patterndb_a_SOURCES = radix.c radix.h \ patterndb.c patterndb.h patterndb-int.h \ timerwheel.c timerwheel.h \ patternize.c patternize.h -libsyslog_ng_patterndb_a_CFLAGS = $(AM_CFLAGS) -fPIC
+# we're not linking against the .a file as this is not supported on Cygwin, +# even if we compile it using -fPIC. Therefore the .a file is only used to +# link programs, and the dbparser.so gets those files as source files, +# thereby recompiling them twice. module_LTLIBRARIES = libdbparser.la libdbparser_la_SOURCES = \ dbparser.c dbparser.h \ - dbparser-grammar.y dbparser-parser.c dbparser-parser.h dbparser-plugin.c + dbparser-grammar.y dbparser-parser.c dbparser-parser.h dbparser-plugin.c \ + $(libsyslog_ng_patterndb_a_SOURCES)
libdbparser_la_CPPFLAGS = $(AM_CPPFLAGS) -libdbparser_la_LIBADD = ../../lib/libsyslog-ng.la libsyslog-ng-patterndb.a +libdbparser_la_LIBADD = ../../lib/libsyslog-ng.la
This fails to build on Cygwin because libdbparser_la_LIBADD is missing a @OPENSSL_LIBS@ (needed by patternize.c).
Amazing but true: Another restriction of the Windows platform is the inability to create shared libs with undefined symbols. All symbols must be resolvable at link time.
libdbparser_la_LDFLAGS = -avoid-version -module -no-undefined
bin_PROGRAMS = pdbtool pdbtool_SOURCES = pdbtool.c pdbtool_CPPFLAGS = $(AM_CPPFLAGS) @OPENSSL_CFLAGS@ -pdbtool_LDADD = ../../lib/libsyslog-ng.la libsyslog-ng-patterndb.a @TOOL_DEPS_LIBS@ @OPENSSL_LIBS@ +pdbtool_LDADD = libsyslog-ng-patterndb.a ../../lib/libsyslog-ng.la @TOOL_DEPS_LIBS@ @OPENSSL_LIBS@
BUILT_SOURCES = dbparser-grammar.y dbparser-grammar.c dbparser-grammar.h EXTRA_DIST = $(BUILT_SOURCES) radix-find.c dbparser-grammar.ym diff --git a/modules/dbparser/tests/Makefile.am b/modules/dbparser/tests/Makefile.am index 8d3a970..6726121 100644 --- a/modules/dbparser/tests/Makefile.am +++ b/modules/dbparser/tests/Makefile.am @@ -1,6 +1,6 @@ AM_CFLAGS = -I$(top_srcdir)/lib -I../../../lib -I$(top_srcdir)/modules/dbparser -I.. @CFLAGS_NOWARN_POINTER_SIGN@ AM_LDFLAGS = -dlpreopen ../../syslogformat/libsyslogformat.la -LDADD = $(top_builddir)/lib/libsyslog-ng.la ../libsyslog-ng-patterndb.a ../patternize.lo @TOOL_DEPS_LIBS@ @OPENSSL_LIBS@ +LDADD = $(top_builddir)/lib/libsyslog-ng.la ../libsyslog-ng-patterndb.a @TOOL_DEPS_LIBS@ @OPENSSL_LIBS@
I didn't test that, but shouldn't this LDADD be reordered as well, just as in the pdbtool_LDADD case?
LDADD = ../libsyslog-ng-patterndb.a $(top_builddir)/lib/libsyslog-ng.la @TOOL_DEPS_LIBS@ @OPENSSL_LIBS@
Good catch, also applied.
Btw., I'm seeing a lot of
foo.y: warning: X nonterminals useless in grammar foo.y: warning: Y rules useless in grammar
messages. Aren't the grep calls in build/lex-rules.am wrong?
grep -E -v "warning: (nonterminals|rules) useless in grammar
instead of
grep -E -v "warning: (nonterminal|rule) useless in grammar
? Or does the message depend on the version of yacc/bison used, perhaps?
Well, it seems to matter which bison version you are using. 3.3 has a fix for this (where it accepts both the pural and the singular forms). -- Bazsi
On Feb 24 10:31, Balazs Scheidler wrote:
On Wed, 2011-02-23 at 17:29 +0100, Corinna Vinschen wrote:
ivykis and libmongo-client are not available for Cygwin. Is there no way around using these libs by using conditional compilation?
There's a way, mongodb can be disabled using --disable-mongodb configure option (but is enabled by default with the bundled libmongo-client library).
ivykis is not so simple, syslog-ng core depends on that heavily. ivykis is compiled as a static library (with -fPIC) and then linked into libsyslog-ng.so
The staticlib is just a convenience library, however I'd like that to be created by the ivykis configure/automake stuff, and would like to avoid having to rewrite the build system for ivykis. Also, I wouldn't want to install ivykis (as a shared object) from the syslog-ng tree, as I could perhaps break people's setup if ivykis becomes more widely available. (e.g. installed on the system).
Do you perhaps have an idea how to solve that differently?
If you don't provide a shared ivykis lib, building libsyslog-ng.so on Cygwin will break. Assuming ivykis builds on Cygwin, which nobody tested yet either... Off the top my head I can see two ways around the shared/static problem. - Add the ivykis source files to libsyslog_ng_la_SOURCES. - Don't link libsyslog-ng.so against ivykis at all. Instead, build ivykis as a module which gets installed into /usr/lib/syslog-ng and use dlopen/dlsym to call its functions.
Btw., I'm seeing a lot of
foo.y: warning: X nonterminals useless in grammar foo.y: warning: Y rules useless in grammar
messages. Aren't the grep calls in build/lex-rules.am wrong?
grep -E -v "warning: (nonterminals|rules) useless in grammar
instead of
grep -E -v "warning: (nonterminal|rule) useless in grammar
? Or does the message depend on the version of yacc/bison used, perhaps?
Well, it seems to matter which bison version you are using. 3.3 has a fix for this (where it accepts both the pural and the singular forms).
Any chance to fix this in 3.2, too? It's just adding a "s?" to the regex: grep -E -v "warning: (nonterminal|rule)s? useless in grammar" Thanks, Corinna -- Corinna Vinschen Cygwin Project Co-Leader Red Hat
On Thu, 2011-02-24 at 10:59 +0100, Corinna Vinschen wrote:
On Feb 24 10:31, Balazs Scheidler wrote:
On Wed, 2011-02-23 at 17:29 +0100, Corinna Vinschen wrote:
ivykis and libmongo-client are not available for Cygwin. Is there no way around using these libs by using conditional compilation?
There's a way, mongodb can be disabled using --disable-mongodb configure option (but is enabled by default with the bundled libmongo-client library).
ivykis is not so simple, syslog-ng core depends on that heavily. ivykis is compiled as a static library (with -fPIC) and then linked into libsyslog-ng.so
The staticlib is just a convenience library, however I'd like that to be created by the ivykis configure/automake stuff, and would like to avoid having to rewrite the build system for ivykis. Also, I wouldn't want to install ivykis (as a shared object) from the syslog-ng tree, as I could perhaps break people's setup if ivykis becomes more widely available. (e.g. installed on the system).
Do you perhaps have an idea how to solve that differently?
If you don't provide a shared ivykis lib, building libsyslog-ng.so on Cygwin will break. Assuming ivykis builds on Cygwin, which nobody tested yet either...
Off the top my head I can see two ways around the shared/static problem.
- Add the ivykis source files to libsyslog_ng_la_SOURCES.
- Don't link libsyslog-ng.so against ivykis at all. Instead, build ivykis as a module which gets installed into /usr/lib/syslog-ng and use dlopen/dlsym to call its functions.
Hmm... I've attempted to create a completely static syslog-ng binary today, and although I've some doubts that I could do that for all the platforms that syslog-ng supports, it might work for a couple of them. So the general idea is: * everything gets linked into the main syslog-ng binary and libsyslog-ng.so/libivykis.so is not used. * even some of the selected modules can be linked in, which is a performance win The issue with that is that the final link probably can't be performed by libtool, I've just manually edited the command line that it emitted. I guess this can be a good solution for cygwin too, and will probably not be available for all platforms. But we'll see as we get there, I'm focusing on other stuff right now. :) -- Bazsi
On Mar 1 19:23, Balazs Scheidler wrote:
On Thu, 2011-02-24 at 10:59 +0100, Corinna Vinschen wrote:
On Feb 24 10:31, Balazs Scheidler wrote:
ivykis is not so simple, syslog-ng core depends on that heavily. ivykis is compiled as a static library (with -fPIC) and then linked into libsyslog-ng.so
The staticlib is just a convenience library, however I'd like that to be created by the ivykis configure/automake stuff, and would like to avoid having to rewrite the build system for ivykis. Also, I wouldn't want to install ivykis (as a shared object) from the syslog-ng tree, as I could perhaps break people's setup if ivykis becomes more widely available. (e.g. installed on the system).
Do you perhaps have an idea how to solve that differently?
If you don't provide a shared ivykis lib, building libsyslog-ng.so on Cygwin will break. Assuming ivykis builds on Cygwin, which nobody tested yet either...
Off the top my head I can see two ways around the shared/static problem.
- Add the ivykis source files to libsyslog_ng_la_SOURCES.
- Don't link libsyslog-ng.so against ivykis at all. Instead, build ivykis as a module which gets installed into /usr/lib/syslog-ng and use dlopen/dlsym to call its functions.
Hmm... I've attempted to create a completely static syslog-ng binary today, and although I've some doubts that I could do that for all the platforms that syslog-ng supports, it might work for a couple of them.
So the general idea is: * everything gets linked into the main syslog-ng binary and libsyslog-ng.so/libivykis.so is not used. * even some of the selected modules can be linked in, which is a performance win
The issue with that is that the final link probably can't be performed by libtool, I've just manually edited the command line that it emitted.
I guess this can be a good solution for cygwin too, and will probably not be available for all platforms.
Thanks for the info. I'm curious how this will work. Corinna -- Corinna Vinschen Cygwin Project Co-Leader Red Hat
On Wed, 2011-03-02 at 09:57 +0100, Corinna Vinschen wrote:
On Mar 1 19:23, Balazs Scheidler wrote:
On Thu, 2011-02-24 at 10:59 +0100, Corinna Vinschen wrote:
On Feb 24 10:31, Balazs Scheidler wrote:
ivykis is not so simple, syslog-ng core depends on that heavily. ivykis is compiled as a static library (with -fPIC) and then linked into libsyslog-ng.so
The staticlib is just a convenience library, however I'd like that to be created by the ivykis configure/automake stuff, and would like to avoid having to rewrite the build system for ivykis. Also, I wouldn't want to install ivykis (as a shared object) from the syslog-ng tree, as I could perhaps break people's setup if ivykis becomes more widely available. (e.g. installed on the system).
Do you perhaps have an idea how to solve that differently?
If you don't provide a shared ivykis lib, building libsyslog-ng.so on Cygwin will break. Assuming ivykis builds on Cygwin, which nobody tested yet either...
Off the top my head I can see two ways around the shared/static problem.
- Add the ivykis source files to libsyslog_ng_la_SOURCES.
- Don't link libsyslog-ng.so against ivykis at all. Instead, build ivykis as a module which gets installed into /usr/lib/syslog-ng and use dlopen/dlsym to call its functions.
Hmm... I've attempted to create a completely static syslog-ng binary today, and although I've some doubts that I could do that for all the platforms that syslog-ng supports, it might work for a couple of them.
So the general idea is: * everything gets linked into the main syslog-ng binary and libsyslog-ng.so/libivykis.so is not used. * even some of the selected modules can be linked in, which is a performance win
The issue with that is that the final link probably can't be performed by libtool, I've just manually edited the command line that it emitted.
I guess this can be a good solution for cygwin too, and will probably not be available for all platforms.
Thanks for the info. I'm curious how this will work.
well, something along the lines of --enable-static --disable-shared and some libtool magic to get all .a files linked into the syslog-ng binary with -Wl,--whole-archive This doesn't need PIC compilation (as the .a files will be linked into the main executable) and can be done even for embedded libraries. plugin loading would work on platforms where the .so files can use symbols from the executable: * this is the case on Linux * this is definitely not the case on AIX * on cygwin, if my guess is right, this will not work, so there all syslog-ng plugins must be linked in statically too. The alternative is to introduce ivykis as a separate package on cygwin. Would that be a doable alternative? -- Bazsi
On Mar 2 10:47, Balazs Scheidler wrote:
On Wed, 2011-03-02 at 09:57 +0100, Corinna Vinschen wrote:
On Mar 1 19:23, Balazs Scheidler wrote:
So the general idea is: * everything gets linked into the main syslog-ng binary and libsyslog-ng.so/libivykis.so is not used. * even some of the selected modules can be linked in, which is a performance win
The issue with that is that the final link probably can't be performed by libtool, I've just manually edited the command line that it emitted.
I guess this can be a good solution for cygwin too, and will probably not be available for all platforms.
Thanks for the info. I'm curious how this will work.
well, something along the lines of
--enable-static --disable-shared
and some libtool magic to get all .a files linked into the syslog-ng binary with -Wl,--whole-archive
This doesn't need PIC compilation (as the .a files will be linked into the main executable) and can be done even for embedded libraries.
plugin loading would work on platforms where the .so files can use symbols from the executable: * this is the case on Linux * this is definitely not the case on AIX * on cygwin, if my guess is right, this will not work, so there all syslog-ng plugins must be linked in statically too.
The alternative is to introduce ivykis as a separate package on cygwin. Would that be a doable alternative?
We only allow packages in the distro for which we have a an active maintainer. If you're willing to package and maintain ivykis for Cygwin, or if you have somebody who's willing to do so, then that would be a doable solution, yes. I gave ivykis a quick whirl, but it doesn't build on Cygwin because Cygwin has no support for pthread_spinlocks so far. Corinna -- Corinna Vinschen Cygwin Project Co-Leader Red Hat
On Mar 2 12:52, Corinna Vinschen wrote:
On Mar 2 10:47, Balazs Scheidler wrote:
well, something along the lines of
--enable-static --disable-shared
and some libtool magic to get all .a files linked into the syslog-ng binary with -Wl,--whole-archive
This doesn't need PIC compilation (as the .a files will be linked into the main executable) and can be done even for embedded libraries.
plugin loading would work on platforms where the .so files can use symbols from the executable: * this is the case on Linux * this is definitely not the case on AIX * on cygwin, if my guess is right, this will not work, so there all syslog-ng plugins must be linked in statically too.
The alternative is to introduce ivykis as a separate package on cygwin. Would that be a doable alternative?
We only allow packages in the distro for which we have a an active maintainer. If you're willing to package and maintain ivykis for Cygwin, or if you have somebody who's willing to do so, then that would be a doable solution, yes.
I gave ivykis a quick whirl, but it doesn't build on Cygwin because Cygwin has no support for pthread_spinlocks so far.
Unfortunately ivykis uses pthread_spinlocks unconditionally, even though they are marked as optional in POSIX-1.2004 and only moved to the Base in POSIX-1.2008. Corinna -- Corinna Vinschen Cygwin Project Co-Leader Red Hat
On Wed, 2011-03-02 at 13:07 +0100, Corinna Vinschen wrote:
On Mar 2 12:52, Corinna Vinschen wrote:
On Mar 2 10:47, Balazs Scheidler wrote:
well, something along the lines of
--enable-static --disable-shared
and some libtool magic to get all .a files linked into the syslog-ng binary with -Wl,--whole-archive
This doesn't need PIC compilation (as the .a files will be linked into the main executable) and can be done even for embedded libraries.
plugin loading would work on platforms where the .so files can use symbols from the executable: * this is the case on Linux * this is definitely not the case on AIX * on cygwin, if my guess is right, this will not work, so there all syslog-ng plugins must be linked in statically too.
The alternative is to introduce ivykis as a separate package on cygwin. Would that be a doable alternative?
We only allow packages in the distro for which we have a an active maintainer. If you're willing to package and maintain ivykis for Cygwin, or if you have somebody who's willing to do so, then that would be a doable solution, yes.
I gave ivykis a quick whirl, but it doesn't build on Cygwin because Cygwin has no support for pthread_spinlocks so far.
Unfortunately ivykis uses pthread_spinlocks unconditionally, even though they are marked as optional in POSIX-1.2004 and only moved to the Base in POSIX-1.2008.
Yes, they are needed because pthread_mutex is not async signal safe. We've also ran into this issue (on our set of ancient UNIX systems), but because syslog-ng is only using statically registered signal handlers (e.g. once at program startup), it's not a real issue, so we simply removed the locking that protected against parallel signal handler execution & registration. -- Bazsi
On Mar 2 13:55, Balazs Scheidler wrote:
On Wed, 2011-03-02 at 13:07 +0100, Corinna Vinschen wrote:
On Mar 2 12:52, Corinna Vinschen wrote:
On Mar 2 10:47, Balazs Scheidler wrote:
well, something along the lines of
--enable-static --disable-shared
and some libtool magic to get all .a files linked into the syslog-ng binary with -Wl,--whole-archive
This doesn't need PIC compilation (as the .a files will be linked into the main executable) and can be done even for embedded libraries.
plugin loading would work on platforms where the .so files can use symbols from the executable: * this is the case on Linux * this is definitely not the case on AIX * on cygwin, if my guess is right, this will not work, so there all syslog-ng plugins must be linked in statically too.
The alternative is to introduce ivykis as a separate package on cygwin. Would that be a doable alternative?
We only allow packages in the distro for which we have a an active maintainer. If you're willing to package and maintain ivykis for Cygwin, or if you have somebody who's willing to do so, then that would be a doable solution, yes.
I gave ivykis a quick whirl, but it doesn't build on Cygwin because Cygwin has no support for pthread_spinlocks so far.
Unfortunately ivykis uses pthread_spinlocks unconditionally, even though they are marked as optional in POSIX-1.2004 and only moved to the Base in POSIX-1.2008.
Yes, they are needed because pthread_mutex is not async signal safe. We've also ran into this issue (on our set of ancient UNIX systems), but because syslog-ng is only using statically registered signal handlers (e.g. once at program startup), it's not a real issue, so we simply removed the locking that protected against parallel signal handler execution & registration.
I'll look into implementing pthread_spinlocks for the next Cygwin release. But that will take some time, right now we're rather busy with two problems in the current release. Corinna -- Corinna Vinschen Cygwin Project Co-Leader Red Hat
On Wed, 2011-03-02 at 14:14 +0100, Corinna Vinschen wrote:
On Mar 2 13:55, Balazs Scheidler wrote:
On Wed, 2011-03-02 at 13:07 +0100, Corinna Vinschen wrote:
On Mar 2 12:52, Corinna Vinschen wrote:
On Mar 2 10:47, Balazs Scheidler wrote:
well, something along the lines of
--enable-static --disable-shared
and some libtool magic to get all .a files linked into the syslog-ng binary with -Wl,--whole-archive
This doesn't need PIC compilation (as the .a files will be linked into the main executable) and can be done even for embedded libraries.
plugin loading would work on platforms where the .so files can use symbols from the executable: * this is the case on Linux * this is definitely not the case on AIX * on cygwin, if my guess is right, this will not work, so there all syslog-ng plugins must be linked in statically too.
The alternative is to introduce ivykis as a separate package on cygwin. Would that be a doable alternative?
We only allow packages in the distro for which we have a an active maintainer. If you're willing to package and maintain ivykis for Cygwin, or if you have somebody who's willing to do so, then that would be a doable solution, yes.
I gave ivykis a quick whirl, but it doesn't build on Cygwin because Cygwin has no support for pthread_spinlocks so far.
Unfortunately ivykis uses pthread_spinlocks unconditionally, even though they are marked as optional in POSIX-1.2004 and only moved to the Base in POSIX-1.2008.
Yes, they are needed because pthread_mutex is not async signal safe. We've also ran into this issue (on our set of ancient UNIX systems), but because syslog-ng is only using statically registered signal handlers (e.g. once at program startup), it's not a real issue, so we simply removed the locking that protected against parallel signal handler execution & registration.
I'll look into implementing pthread_spinlocks for the next Cygwin release.
It'd be really cool. I guess, then it means something that you are "Cygwin Project Co-Leader". :)
But that will take some time, right now we're rather busy with two problems in the current release.
Yes, sure. btw: I once pondered about purchasing a cygwin license, I even went as far as submitted a form on redhat.com, that we'd be interested. But I got nothing back. I've since dropped that project, but knowing more details about cygwin options would be useful. Can you contact me to someone who can help me? Thanks in advance. -- Bazsi
On Mar 2 17:51, Balazs Scheidler wrote:
On Wed, 2011-03-02 at 14:14 +0100, Corinna Vinschen wrote:
I'll look into implementing pthread_spinlocks for the next Cygwin release.
It'd be really cool. I guess, then it means something that you are "Cygwin Project Co-Leader". :)
But that will take some time, right now we're rather busy with two problems in the current release.
Yes, sure.
btw: I once pondered about purchasing a cygwin license, I even went as far as submitted a form on redhat.com, that we'd be interested. But I got nothing back. I've since dropped that project, but knowing more details about cygwin options would be useful. Can you contact me to someone who can help me? Thanks in advance.
I'm a bit surprised. Cygwin is an Open Source project just like syslog-ng. You don't need a license, unless you try to link a proprietary project to the Cygwin DLL. See http://cygwin.com/licensing.html Just start here: http://cygwin.com/ and run the setup.exe tool from the homepage to install the Cygwin net distro with all the latest tools available. For the User's Guide, see http://cygwin.com/cygwin-ug-net/cygwin-ug-net.html There's also a bunch of mailing lists, see http://cygwi.com/lists.html. The cygwin AT cygwin DOT com mailing list is the central point of contact. If you really need a commercial license for a proprietary project, just contact me off-list. Corinna -- Corinna Vinschen Cygwin Project Co-Leader Red Hat
On Wed, 2011-03-02 at 18:13 +0100, Corinna Vinschen wrote:
On Mar 2 17:51, Balazs Scheidler wrote:
On Wed, 2011-03-02 at 14:14 +0100, Corinna Vinschen wrote:
I'll look into implementing pthread_spinlocks for the next Cygwin release.
It'd be really cool. I guess, then it means something that you are "Cygwin Project Co-Leader". :)
But that will take some time, right now we're rather busy with two problems in the current release.
Yes, sure.
btw: I once pondered about purchasing a cygwin license, I even went as far as submitted a form on redhat.com, that we'd be interested. But I got nothing back. I've since dropped that project, but knowing more details about cygwin options would be useful. Can you contact me to someone who can help me? Thanks in advance.
I'm a bit surprised. Cygwin is an Open Source project just like syslog-ng. You don't need a license, unless you try to link a proprietary project to the Cygwin DLL. See http://cygwin.com/licensing.html
Just start here: http://cygwin.com/ and run the setup.exe tool from the homepage to install the Cygwin net distro with all the latest tools available. For the User's Guide, see http://cygwin.com/cygwin-ug-net/cygwin-ug-net.html
There's also a bunch of mailing lists, see http://cygwi.com/lists.html. The cygwin AT cygwin DOT com mailing list is the central point of contact.
I know cygwin this much :)
If you really need a commercial license for a proprietary project, just contact me off-list.
Yes, that's what I meant, but sending you an email as you suggest. -- Bazsi
On Mar 2 20:38, Balazs Scheidler wrote:
On Wed, 2011-03-02 at 18:13 +0100, Corinna Vinschen wrote:
On Mar 2 17:51, Balazs Scheidler wrote:
btw: I once pondered about purchasing a cygwin license, I even went as far as submitted a form on redhat.com, that we'd be interested. But I got nothing back. I've since dropped that project, but knowing more details about cygwin options would be useful. Can you contact me to someone who can help me? Thanks in advance.
I'm a bit surprised. [blah]
I know cygwin this much :)
Sorry, I misunderstood your question. Corinna -- Corinna Vinschen Cygwin Project Co-Leader Red Hat
On Feb 23 15:18, Corinna Vinschen wrote:
diff --git a/modules/dbparser/Makefile.am b/modules/dbparser/Makefile.am index b2ab700..f8a53f4 100644 --- a/modules/dbparser/Makefile.am +++ b/modules/dbparser/Makefile.am @@ -21,7 +21,8 @@ libsyslog_ng_patterndb_a_CFLAGS = $(AM_CFLAGS) -fPIC module_LTLIBRARIES = libdbparser.la libdbparser_la_SOURCES = \ dbparser.c dbparser.h \ - dbparser-grammar.y dbparser-parser.c dbparser-parser.h dbparser-plugin.c + dbparser-grammar.y dbparser-parser.c dbparser-parser.h \ + dbparser-plugin.c $(libsyslog_ng_patterndb_a_SOURCES)
libdbparser_la_CPPFLAGS = $(AM_CPPFLAGS) libdbparser_la_LIBADD = ../../lib/libsyslog-ng.la libsyslog-ng-patterndb.a
Of course, libsyslog-ng-patterndb.a has to be removed from libdbparser_la_LIBADD, and the dependencies have to be added, so the full patch against current git HEAD would look like this: Signed-off-by: Corinna Vinschen <vinschen@redhat.com> diff --git a/modules/dbparser/Makefile.am b/modules/dbparser/Makefile.am index b2ab700..417c8b7 100644 --- a/modules/dbparser/Makefile.am +++ b/modules/dbparser/Makefile.am @@ -11,7 +11,7 @@ AM_CPPFLAGS = -I$(top_srcdir)/lib -I../../lib AM_CFLAGS = @CFLAGS_NOWARN_POINTER_SIGN@ export top_srcdir -lib_LIBRARIES = libsyslog-ng-patterndb.a +noinst_LIBRARIES = libsyslog-ng-patterndb.a libsyslog_ng_patterndb_a_SOURCES = radix.c radix.h \ patterndb.c patterndb.h patterndb-int.h \ timerwheel.c timerwheel.h \ @@ -21,16 +21,17 @@ libsyslog_ng_patterndb_a_CFLAGS = $(AM_CFLAGS) -fPIC module_LTLIBRARIES = libdbparser.la libdbparser_la_SOURCES = \ dbparser.c dbparser.h \ - dbparser-grammar.y dbparser-parser.c dbparser-parser.h dbparser-plugin.c + dbparser-grammar.y dbparser-parser.c dbparser-parser.h \ + dbparser-plugin.c $(libsyslog_ng_patterndb_a_SOURCES) libdbparser_la_CPPFLAGS = $(AM_CPPFLAGS) -libdbparser_la_LIBADD = ../../lib/libsyslog-ng.la libsyslog-ng-patterndb.a +libdbparser_la_LIBADD = ../../lib/libsyslog-ng.la @OPENSSL_LIBS@ libdbparser_la_LDFLAGS = -avoid-version -module -no-undefined bin_PROGRAMS = pdbtool pdbtool_SOURCES = pdbtool.c pdbtool_CPPFLAGS = $(AM_CPPFLAGS) @OPENSSL_CFLAGS@ -pdbtool_LDADD = ../../lib/libsyslog-ng.la libsyslog-ng-patterndb.a @TOOL_DEPS_LIBS@ @OPENSSL_LIBS@ +pdbtool_LDADD = libsyslog-ng-patterndb.a ../../lib/libsyslog-ng.la @TOOL_DEPS_LIBS@ @OPENSSL_LIBS@ BUILT_SOURCES = dbparser-grammar.y dbparser-grammar.c dbparser-grammar.h EXTRA_DIST = $(BUILT_SOURCES) radix-find.c dbparser-grammar.ym I'm just testing this on Cygwin. Hang on... [...time passes...] Yes, that works. I hope that's ok to check in. Corinna -- Corinna Vinschen Cygwin Project Co-Leader Red Hat
Hello, On 02/23/2011 01:05 PM, Balazs Scheidler wrote:
Could anyone who is in charge of maintaining syslog-ng binaries in various distributions give this a test so 3.2.3 will compile everywhere out-of-the-box?
Testing it on openSUSE. Compiled fine locally on 11.3 and Factory i586, now waiting for the build service for the rest: https://build.opensuse.org/project/show?project=home%3Aczanik%3Abranches%3AB... Bye, -- Peter Czanik (CzP) <czanik@balabit.hu> BalaBit IT Security / syslog-ng upstream http://czanik.blogs.balabit.com/
participants (4)
-
Balazs Scheidler
-
Corinna Vinschen
-
Marius Tomaschewski
-
Peter Czanik