[syslog-ng] Solaris 10 support(?) in latest versions (e.g. 3.3.2)
Pal Tamas
folti at balabit.hu
Wed Nov 16 10:29:42 CET 2011
On Tue, Nov 15, 2011 at 08:11:34PM +0100, Balazs Scheidler wrote:
> On Tue, 2011-11-15 at 17:36 +0100, Pal Tamas wrote:
> > Hi,
> >
> > On Tue, Nov 15, 2011 at 10:09:50AM -0600, Marvin Nipper wrote:
> > > Apologies upfront for the length, but this addresses two different problems.
> > >
> > >
> > > Environment: Solaris 10 x64 U10, fully patched (i.e. pretty much "new").
> > > Currently operating (successfully) with 3.0.5, compiled on that platform, and
> > > distributed to a number of other Solaris 10 x64 targets.
> > >
> > > Problem: Trying to "make the leap" to 3.3.2, and get current, but that version
> > > seems to have numerous Solaris compatibility issues. Just trying to understand
> > > the status of Solaris support.
> > >
> > >
> > >
> > > Problem #1
> > > Latest packaging no longer works with Solaris-based "make".
> > >
> > > After a clean configure (output here):
> > > syslog-ng Open Source Edition 3.3.2 configured
> > > Compiler options:
> > > compiler : gcc -std=gnu99
> > > compiler options : -g -O2 -Wall -pthread -D_REENTRANT -D_PTHREADS
> > > -I/usr/local/include/glib-2.0 -I/usr/local/lib/glib-2.0/include -I/usr/local/
> > > include/eventlog -I/usr/local/ssl/include -DLIBNET_LIL_ENDIAN -I$
> > > (top_srcdir)/lib/ivykis/lib/include -I$(top_builddir)/lib/ivykis/lib/include
> > > -I$(top_srcdir)/lib/ivykis/modules/include -D_GNU_SOURCE -D_LARGEFILE_SOURCE
> > > -D_FILE_OFFSET_BITS=64
> > > linker flags : -R/usr/local/lib -lpthread
> > > prefix : /usr/local
> > > linking mode : dynamic
> > > __thread keyword : yes
> > > Submodules:
> > > ivykis : internal
> > > libmongo-client : internal
> > > Features:
> > > Debug symbols : no
> > > GCC profiling : no
> > > Memtrace : no
> > > IPV6 support : yes
> > > spoof-source support : yes
> > > tcp-wrapper support : no
> > > Linux capability support : no
> > > PCRE support : no
> > > Env wrapper support : no
> > > systemd support : no (unit dir: none)
> > > Modules:
> > > Module search path : /usr/local/lib/syslog-ng
> > > Default module list :
> > > affile,afprog,afsocket,afuser,basicfuncs,csvparser,dbparser,syslogformat,afstreams
> > > Sun STREAMS support (module): yes
> > > SSL support (module) : yes
> > > SQL support (module) : no
> > > PACCT module (EXPERIMENTAL) : no
> > > MongoDB destination (module): yes
> > > JSON support (module) : no (using no)
> > >
> > >
> > > ..... a simple "make" attempt now immediately results in this:
> > > # make
> > > make all-recursive
> > > Making all in lib
> > > make: Fatal error in reader: Makefile, line 991: Unexpected end of line seen
> > > Current working directory /var/opt/packages/syslog-ng-3.3.2/lib
> > > *** Error code 1
> > > The following command caused the error:
> > > fail= failcom='exit 1'; \
> > > for f in x $MAKEFLAGS; do \
> > > case $f in \
> > > *=* | --[!k]*);; \
> > > *k*) failcom='fail=yes';; \
> > > esac; \
> > > done; \
> > > dot_seen=no; \
> > > target=`echo all-recursive | sed s/-recursive//`; \
> > > list='lib modules syslog-ng scripts tests doc contrib scl debian tgz2build
> > > build'; for subdir in $list; do \
> > > echo "Making $target in $subdir"; \
> > > if test "$subdir" = "."; then \
> > > dot_seen=yes; \
> > > local_target="$target-am"; \
> > > else \
> > > local_target="$target"; \
> > > fi; \
> > > (CDPATH="${ZSH_VERSION+.}:" && cd $subdir && make $local_target) \
> > > || eval $failcom; \
> > > done; \
> > > if test "$dot_seen" = "no"; then \
> > > make "$target-am" || exit 1; \
> > > fi; test -z "$fail"
> > > make: Fatal error: Command failed for target `all-recursive'
> > > Current working directory /var/opt/packages/syslog-ng-3.3.2
> > > *** Error code 1
> > > make: Fatal error: Command failed for target `all'
> > >
> > >
> > > Not being a developer, per se, I spent time in Google-land, the general result
> > > that I find is:
> > > "These are caused because Solaris "make" doesn't accept the ‘GNU multi-line
> > > Makefile format’. Use gmake instead."
> > > But there are obviously folks who also say that this can be fixed by
> > > "correcting" the problems in the Makefiles (so that would seem to be a
> > > preferable alternative).
> > The syslog-ng uses the "autoconfuse" (automake and autoconf tools)
> > system to generate the makefiles needed for build. Since both autoconf
> > and automake are part of the GNU project, the Makefiles generated by
> > them are GNU make specific. Sorry there's no way around it.
> >
> > If you hadn't done it, please install the SUNWgcmn (Common GNU package)
> > and the SUNWgmake from your Solaris install media. It'll put a gmake
> > into /usr/sfw/bin.
> > >
> > >
> > > Problem #2
> > > Can no longer perform successful compilations, due to this error:
> > > ld: fatal: relocation error: R_386_GOTOFF
> > >
> > > Same configure output as previously shown, followed by gmake (to get around the
> > > first problem).
> > >
> > > Numerous modules compile successfully, but then this happens:
> > > gcc -DHAVE_CONFIG_H -I. -I../.. -D_GNU_SOURCE -I../../lib/include -I../../lib/
> > > include -I../../modules/include -Wall -g -O2 -D_REENTRANT -D_POSIX_C_SOURCE=
> > > 199506L -D_POSIX_PTHREAD_SEMANTICS -D_XPG4_2 -MT iv_event_test.o -MD -MP -MF
> > > .deps/iv_event_test.Tpo -c -o iv_event_test.o iv_event_test.c
> > > mv -f .deps/iv_event_test.Tpo .deps/iv_event_test.Po
> > > /bin/bash ../../libtool --tag=CC --mode=link gcc -Wall -g -O2 -D_REENTRANT
> > > -D_POSIX_C_SOURCE=199506L -D_POSIX_PTHREAD_SEMANTICS -D_XPG4_2 -R/usr/local/
> > > lib -o iv_event_test iv_event_test.o ../../lib/libivykis.la ../
> > > libivykis-modules.la -lrt -lpthread -lsocket
> > > libtool: link: gcc -Wall -g -O2 -D_REENTRANT -D_POSIX_C_SOURCE=199506L
> > > -D_POSIX_PTHREAD_SEMANTICS -D_XPG4_2 -o iv_event_test iv_event_test.o ../../
> > > lib/.libs/libivykis.a ../.libs/libivykis-modules.a /var/opt/packages/
> > > syslog-ng-3.3.2/lib/ivykis/lib/.libs/libivykis.a -lrt -lpthread -lsocket -R/usr
> > > /local/lib
> > > ld: fatal: relocation error: R_386_GOTOFF: file ../.libs/libivykis-modules.a
> > > (iv_event.o): symbol __tls: relocation illegal for TLS symbol
> > > ld: fatal: relocation error: R_386_GOTOFF: file ../.libs/libivykis-modules.a
> > > (iv_event.o): symbol __tls: relocation illegal for TLS symbol
> > > ld: fatal: relocation error: R_386_GOTOFF: file ../.libs/libivykis-modules.a
> > > (iv_event.o): symbol __tls: relocation illegal for TLS symbol
> > > ld: fatal: relocation error: R_386_GOTOFF: file ../.libs/libivykis-modules.a
> > > (iv_event.o): symbol __tls: relocation illegal for TLS symbol
> > > collect2: ld returned 1 exit status
> > > gmake[7]: *** [iv_event_test] Error 1
> > > gmake[7]: Leaving directory `/var/opt/packages/syslog-ng-3.3.2/lib/ivykis/
> > > modules/test'
> > > gmake[6]: *** [all-recursive] Error 1
> > > gmake[6]: Leaving directory `/var/opt/packages/syslog-ng-3.3.2/lib/ivykis/
> > > modules'
> > > gmake[5]: *** [all-recursive] Error 1
> > > gmake[5]: Leaving directory `/var/opt/packages/syslog-ng-3.3.2/lib/ivykis'
> > > gmake[4]: *** [all] Error 2
> > > gmake[4]: Leaving directory `/var/opt/packages/syslog-ng-3.3.2/lib/ivykis'
> > > gmake[3]: *** [all-recursive] Error 1
> > > gmake[3]: Leaving directory `/var/opt/packages/syslog-ng-3.3.2/lib'
> > > gmake[2]: *** [all] Error 2
> > > gmake[2]: Leaving directory `/var/opt/packages/syslog-ng-3.3.2/lib'
> > > gmake[1]: *** [all-recursive] Error 1
> > > gmake[1]: Leaving directory `/var/opt/packages/syslog-ng-3.3.2'
> > > gmake: *** [all] Error 2
> > >
> > >
> > > Again, adopting the "Google is my friend" approach, this appears to be an issue
> > > with "Solaris not supporting Visibility". (http://www.ilkda.com/compile/
> > > Build_Problems.htm)
> > > Most of the sites where I see this being discussed, with other (various) source
> > > components, I also see the developers subsequently modifying their environment
> > > to "adjust for" the Visibility issue in Solaris.
> > >
> > > So, again, not being a C developer myself, I guess I'm wondering whether I'm
> > > just doing something wrong with 3.3.2 (i.e. something that I need to do
> > > different from my 3.0.5 compilation), or whether this is truly just a "Solaris
> > > thing" that has not been accounted for in the current code base?
> > Looks like your gcc doesn't support TLS (Thread Local Storage) properly.
> > The only way we were able to solve it by building a new (4.6.0) gcc
> > following the steps in this blog post:
> > http://jblopen.com/node/16
>
> There's a bundled library in syslog-ng code, ivykis. The error happens
> when linking one of the test programs for ivykis. I'm not sure why that
> happens, it does link for us properly, but maybe because we use a
> different compiler (as described by Folti).
>
> However TLS (=Thread Local Storage) is not a mandatory compiler feature,
> both syslog-ng and ivykis can operate without it, the issue is that the
> configure script detects that your compiler/linker is supposed to
> support it.
>
> Check out syslog-ng-3.3.2/config.log and
> syslog-ng-3.3.2/lib/ivykis/config.log, and look for "__thread" in the
> logs, alternatively check config.h for both directories and check if
> * syslog-ng/config.h: HAVE_THREAD_KEYWORD is defined
> * syslog-ng/lib/ivykis/config.h: HAVE_TLS is defined.
>
> It'll probably be defined. You can retry compilation by changing the
> config.h files and undef those and recompile.
>
> The only question is why it gets detected and not work at the end. This
> test is used for both:
>
>
> AC_LINK_IFELSE([AC_LANG_PROGRAM(
> [[#include <pthread.h>
> __thread int a;
> ]],
> [a=0;])],
> [ac_cv_have_tls=yes; AC_DEFINE_UNQUOTED(HAVE_THREAD_KEYWORD, 1, "Whether Transport Layer Security is supported by the system")])
>
> I'm not sure if we can actually reliably detect if TLS is available. It
> could make sense to make it a configure parameter after all. (which I'm
> usually against...).
Because on a vanilla Solaris __thread works as long as you use it in an
executable. If you use it in a library then try to link said library to
an executeble, you'll get the error. Probably we would be better off in
the long run, if we provide a configure parameter to disable TLS
on systems where it's broken by default.
>
> syslog-ng does support Solaris (the PE staff regularly compiles & tests
> it under Solaris 8, 9, 10; sparc, i386, amd64), however it is not just
> Solaris, gcc/binutils/etc are all part of the picture.
Also patches. Our u4 builderbox has been patches applied which modify
the linker too. These are:
120830-05
118833-36
120011-14
125095-15
--
Pal Tamas/Folti
folti at balabit.hu
More information about the syslog-ng
mailing list