On Tue, Nov 15, 2011 at 08:11:34PM +0100, Balazs Scheidler wrote:
On Tue, 2011-11-15 at 17:36 +0100, Pal Tamas wrote:
Hi,
On Tue, Nov 15, 2011 at 10:09:50AM -0600, Marvin Nipper wrote:
Apologies upfront for the length, but this addresses two different problems.
Environment: Solaris 10 x64 U10, fully patched (i.e. pretty much "new"). Currently operating (successfully) with 3.0.5, compiled on that platform, and distributed to a number of other Solaris 10 x64 targets.
Problem: Trying to "make the leap" to 3.3.2, and get current, but that version seems to have numerous Solaris compatibility issues. Just trying to understand the status of Solaris support.
Problem #1 Latest packaging no longer works with Solaris-based "make".
After a clean configure (output here): syslog-ng Open Source Edition 3.3.2 configured Compiler options: compiler : gcc -std=gnu99 compiler options : -g -O2 -Wall -pthread -D_REENTRANT -D_PTHREADS -I/usr/local/include/glib-2.0 -I/usr/local/lib/glib-2.0/include -I/usr/local/ include/eventlog -I/usr/local/ssl/include -DLIBNET_LIL_ENDIAN -I$ (top_srcdir)/lib/ivykis/lib/include -I$(top_builddir)/lib/ivykis/lib/include -I$(top_srcdir)/lib/ivykis/modules/include -D_GNU_SOURCE -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 linker flags : -R/usr/local/lib -lpthread prefix : /usr/local linking mode : dynamic __thread keyword : yes Submodules: ivykis : internal libmongo-client : internal Features: Debug symbols : no GCC profiling : no Memtrace : no IPV6 support : yes spoof-source support : yes tcp-wrapper support : no Linux capability support : no PCRE support : no Env wrapper support : no systemd support : no (unit dir: none) Modules: Module search path : /usr/local/lib/syslog-ng Default module list : affile,afprog,afsocket,afuser,basicfuncs,csvparser,dbparser,syslogformat,afstreams Sun STREAMS support (module): yes SSL support (module) : yes SQL support (module) : no PACCT module (EXPERIMENTAL) : no MongoDB destination (module): yes JSON support (module) : no (using no)
..... a simple "make" attempt now immediately results in this: # make make all-recursive Making all in lib make: Fatal error in reader: Makefile, line 991: Unexpected end of line seen Current working directory /var/opt/packages/syslog-ng-3.3.2/lib *** Error code 1 The following command caused the error: fail= failcom='exit 1'; \ for f in x $MAKEFLAGS; do \ case $f in \ *=* | --[!k]*);; \ *k*) failcom='fail=yes';; \ esac; \ done; \ dot_seen=no; \ target=`echo all-recursive | sed s/-recursive//`; \ list='lib modules syslog-ng scripts tests doc contrib scl debian tgz2build build'; for subdir in $list; do \ echo "Making $target in $subdir"; \ if test "$subdir" = "."; then \ dot_seen=yes; \ local_target="$target-am"; \ else \ local_target="$target"; \ fi; \ (CDPATH="${ZSH_VERSION+.}:" && cd $subdir && make $local_target) \ || eval $failcom; \ done; \ if test "$dot_seen" = "no"; then \ make "$target-am" || exit 1; \ fi; test -z "$fail" make: Fatal error: Command failed for target `all-recursive' Current working directory /var/opt/packages/syslog-ng-3.3.2 *** Error code 1 make: Fatal error: Command failed for target `all'
Not being a developer, per se, I spent time in Google-land, the general result that I find is: "These are caused because Solaris "make" doesn't accept the ‘GNU multi-line Makefile format’. Use gmake instead." But there are obviously folks who also say that this can be fixed by "correcting" the problems in the Makefiles (so that would seem to be a preferable alternative). The syslog-ng uses the "autoconfuse" (automake and autoconf tools) system to generate the makefiles needed for build. Since both autoconf and automake are part of the GNU project, the Makefiles generated by them are GNU make specific. Sorry there's no way around it.
If you hadn't done it, please install the SUNWgcmn (Common GNU package) and the SUNWgmake from your Solaris install media. It'll put a gmake into /usr/sfw/bin.
Problem #2 Can no longer perform successful compilations, due to this error: ld: fatal: relocation error: R_386_GOTOFF
Same configure output as previously shown, followed by gmake (to get around the first problem).
Numerous modules compile successfully, but then this happens: gcc -DHAVE_CONFIG_H -I. -I../.. -D_GNU_SOURCE -I../../lib/include -I../../lib/ include -I../../modules/include -Wall -g -O2 -D_REENTRANT -D_POSIX_C_SOURCE= 199506L -D_POSIX_PTHREAD_SEMANTICS -D_XPG4_2 -MT iv_event_test.o -MD -MP -MF .deps/iv_event_test.Tpo -c -o iv_event_test.o iv_event_test.c mv -f .deps/iv_event_test.Tpo .deps/iv_event_test.Po /bin/bash ../../libtool --tag=CC --mode=link gcc -Wall -g -O2 -D_REENTRANT -D_POSIX_C_SOURCE=199506L -D_POSIX_PTHREAD_SEMANTICS -D_XPG4_2 -R/usr/local/ lib -o iv_event_test iv_event_test.o ../../lib/libivykis.la ../ libivykis-modules.la -lrt -lpthread -lsocket libtool: link: gcc -Wall -g -O2 -D_REENTRANT -D_POSIX_C_SOURCE=199506L -D_POSIX_PTHREAD_SEMANTICS -D_XPG4_2 -o iv_event_test iv_event_test.o ../../ lib/.libs/libivykis.a ../.libs/libivykis-modules.a /var/opt/packages/ syslog-ng-3.3.2/lib/ivykis/lib/.libs/libivykis.a -lrt -lpthread -lsocket -R/usr /local/lib ld: fatal: relocation error: R_386_GOTOFF: file ../.libs/libivykis-modules.a (iv_event.o): symbol __tls: relocation illegal for TLS symbol ld: fatal: relocation error: R_386_GOTOFF: file ../.libs/libivykis-modules.a (iv_event.o): symbol __tls: relocation illegal for TLS symbol ld: fatal: relocation error: R_386_GOTOFF: file ../.libs/libivykis-modules.a (iv_event.o): symbol __tls: relocation illegal for TLS symbol ld: fatal: relocation error: R_386_GOTOFF: file ../.libs/libivykis-modules.a (iv_event.o): symbol __tls: relocation illegal for TLS symbol collect2: ld returned 1 exit status gmake[7]: *** [iv_event_test] Error 1 gmake[7]: Leaving directory `/var/opt/packages/syslog-ng-3.3.2/lib/ivykis/ modules/test' gmake[6]: *** [all-recursive] Error 1 gmake[6]: Leaving directory `/var/opt/packages/syslog-ng-3.3.2/lib/ivykis/ modules' gmake[5]: *** [all-recursive] Error 1 gmake[5]: Leaving directory `/var/opt/packages/syslog-ng-3.3.2/lib/ivykis' gmake[4]: *** [all] Error 2 gmake[4]: Leaving directory `/var/opt/packages/syslog-ng-3.3.2/lib/ivykis' gmake[3]: *** [all-recursive] Error 1 gmake[3]: Leaving directory `/var/opt/packages/syslog-ng-3.3.2/lib' gmake[2]: *** [all] Error 2 gmake[2]: Leaving directory `/var/opt/packages/syslog-ng-3.3.2/lib' gmake[1]: *** [all-recursive] Error 1 gmake[1]: Leaving directory `/var/opt/packages/syslog-ng-3.3.2' gmake: *** [all] Error 2
Again, adopting the "Google is my friend" approach, this appears to be an issue with "Solaris not supporting Visibility". (http://www.ilkda.com/compile/ Build_Problems.htm) Most of the sites where I see this being discussed, with other (various) source components, I also see the developers subsequently modifying their environment to "adjust for" the Visibility issue in Solaris.
So, again, not being a C developer myself, I guess I'm wondering whether I'm just doing something wrong with 3.3.2 (i.e. something that I need to do different from my 3.0.5 compilation), or whether this is truly just a "Solaris thing" that has not been accounted for in the current code base?
Looks like your gcc doesn't support TLS (Thread Local Storage) properly. The only way we were able to solve it by building a new (4.6.0) gcc following the steps in this blog post: http://jblopen.com/node/16
There's a bundled library in syslog-ng code, ivykis. The error happens when linking one of the test programs for ivykis. I'm not sure why that happens, it does link for us properly, but maybe because we use a different compiler (as described by Folti).
However TLS (=Thread Local Storage) is not a mandatory compiler feature, both syslog-ng and ivykis can operate without it, the issue is that the configure script detects that your compiler/linker is supposed to support it.
Check out syslog-ng-3.3.2/config.log and syslog-ng-3.3.2/lib/ivykis/config.log, and look for "__thread" in the logs, alternatively check config.h for both directories and check if * syslog-ng/config.h: HAVE_THREAD_KEYWORD is defined * syslog-ng/lib/ivykis/config.h: HAVE_TLS is defined.
It'll probably be defined. You can retry compilation by changing the config.h files and undef those and recompile.
The only question is why it gets detected and not work at the end. This test is used for both:
AC_LINK_IFELSE([AC_LANG_PROGRAM( [[#include <pthread.h> __thread int a; ]], [a=0;])], [ac_cv_have_tls=yes; AC_DEFINE_UNQUOTED(HAVE_THREAD_KEYWORD, 1, "Whether Transport Layer Security is supported by the system")])
I'm not sure if we can actually reliably detect if TLS is available. It could make sense to make it a configure parameter after all. (which I'm usually against...). Because on a vanilla Solaris __thread works as long as you use it in an executable. If you use it in a library then try to link said library to an executeble, you'll get the error. Probably we would be better off in the long run, if we provide a configure parameter to disable TLS on systems where it's broken by default.
syslog-ng does support Solaris (the PE staff regularly compiles & tests it under Solaris 8, 9, 10; sparc, i386, amd64), however it is not just Solaris, gcc/binutils/etc are all part of the picture. Also patches. Our u4 builderbox has been patches applied which modify the linker too. These are: 120830-05 118833-36 120011-14 125095-15
-- Pal Tamas/Folti folti@balabit.hu