[syslog-ng] Solaris 10 support(?) in latest versions (e.g. 3.3.2)
Marvin Nipper
Marvin.Nipper at stream.com
Thu Nov 17 20:27:31 CET 2011
OK. After a "day and a half" of "hoop jumping", I managed to build a gcc 4.4.2 compiler on Solaris 10. That's definitely _not_ a straightforward effort, and I had to "pick and choose" things from a number of different sites, to get that to work correctly.
So... I have successfully used that new compiler to rebuild my working syslog-ng 3.0.5 executable, so I know that it works. (And I completely recompiled all of the underlying components, including pkg-config, libnet, glib, eventlog, and, openssl, so that none of the original gcc 3.4.3 "stuff" would be used.) I even validated (with ldd) that the new executable had correctly linked the new gcc components (and nothing from the old gcc 3.4.3 world).
So THEN, I tried, again, to compile the syslog-ng 3.3.2 components.
The good news is that it did not abend on the previously reported TLS-related portion of the code! (a "small success"!).
The bad news is that it now gets to this point in the process (obviously a really long line):
/bin/bash ../libtool --tag=CC --mode=link gcc -std=gnu99 -g -O2 -Wall -pthrea
d -no-undefined -release 3.3.2 -R/usr/local/lib -o libsyslog-ng.la -rpath /usr/l
ocal/lib afinter.lo alarms.lo apphook.lo block-ref-parser.lo center.lo cfg.lo cf
g-lexer.lo cfg-parser.lo children.lo compat.lo control.lo dgroup.lo dnscache.lo
driver.lo filter.lo filter-expr-parser.lo globals.lo gprocess.lo gsockaddr.lo gs
ocket.lo logmatcher.lo logmpx.lo logmsg.lo logparser.lo logpipe.lo logprocess.lo
logproto.lo logqueue.lo logqueue-fifo.lo logreader.lo logrewrite.lo logsource.l
o logstamp.lo logtransport.lo logwriter.lo mainloop.lo memtrace.lo messages.lo m
isc.lo msg-format.lo nvtable.lo parser-expr-parser.lo persist-state.lo plugin.lo
pragma-parser.lo rewrite-expr-parser.lo scratch-buffers.lo serialize.lo sgroup.
lo stats.lo str-format.lo syslog-names.lo tags.lo templates.lo timeutils.lo util
s.lo value-pairs.lo cfg-lex.lo cfg-grammar.lo filter-expr-grammar.lo block-ref-g
rammar.lo pragma-grammar.lo parser-expr-grammar.lo rewrite-expr-grammar.lo -lpth
read -ldoor -lsocket -lrt -lnsl -L/usr/local/lib -lgmodule-2.0 -lgthread-2.0 -
lpthread -lthread -lrt -lglib-2.0 -L/usr/local/lib -levtlog -lresolv -ldl
-Wl,--whole-archive -L../lib/ivykis/lib -livykis -L../lib/ivykis/modules -livyk
is-modules -Wl,--no-whole-archive -lpthread
And then starts throwing out these errors:
ld: fatal: symbol 'g_trash_stack_push' is multiply-defined:
(file .libs/afinter.o type=FUNC; file .libs/alarms.o type=FUNC);
ld: fatal: symbol 'g_trash_stack_pop' is multiply-defined:
(file .libs/afinter.o type=FUNC; file .libs/alarms.o type=FUNC);
ld: fatal: symbol 'g_trash_stack_peek' is multiply-defined:
(file .libs/afinter.o type=FUNC; file .libs/alarms.o type=FUNC);
ld: fatal: symbol 'g_trash_stack_height' is multiply-defined:
(file .libs/afinter.o type=FUNC; file .libs/alarms.o type=FUNC);
ld: fatal: symbol 'g_bit_nth_msf' is multiply-defined:
(file .libs/afinter.o type=FUNC; file .libs/alarms.o type=FUNC);
ld: fatal: symbol 'g_bit_nth_lsf' is multiply-defined:
(file .libs/afinter.o type=FUNC; file .libs/alarms.o type=FUNC);
ld: fatal: symbol 'g_bit_storage' is multiply-defined:
(file .libs/afinter.o type=FUNC; file .libs/alarms.o type=FUNC);
ld: fatal: symbol 'g_trash_stack_push' is multiply-defined:
(file .libs/afinter.o type=FUNC; file .libs/apphook.o type=FUNC);
ld: fatal: symbol 'g_trash_stack_pop' is multiply-defined:
(file .libs/afinter.o type=FUNC; file .libs/apphook.o type=FUNC);
ld: fatal: symbol 'g_trash_stack_peek' is multiply-defined:
(file .libs/afinter.o type=FUNC; file .libs/apphook.o type=FUNC);
ld: fatal: symbol 'g_trash_stack_height' is multiply-defined:
(file .libs/afinter.o type=FUNC; file .libs/apphook.o type=FUNC);
ld: fatal: symbol 'g_bit_nth_msf' is multiply-defined:
It issues 427 "multiply-defined" error lines, similar to the ones above, before eventually ending with this final block of output:
ld: fatal: symbol 'g_bit_nth_lsf' is multiply-defined:
(file .libs/afinter.o type=FUNC; file .libs/rewrite-expr-grammar.o type=FUNC);
ld: fatal: symbol 'g_bit_storage' is multiply-defined:
(file .libs/afinter.o type=FUNC; file .libs/rewrite-expr-grammar.o type=FUNC);
ld: fatal: file processing errors. No output written to .libs/libsyslog-ng-3.3.2.so
collect2: ld returned 1 exit status
gmake[4]: *** [libsyslog-ng.la] Error 1
gmake[4]: Leaving directory `/var/opt/packages/syslog-ng-3.3.2/lib'
gmake[3]: *** [all-recursive] Error 1
gmake[3]: Leaving directory `/var/opt/packages/syslog-ng-3.3.2/lib'
gmake[2]: *** [all] Error 2
gmake[2]: Leaving directory `/var/opt/packages/syslog-ng-3.3.2/lib'
gmake[1]: *** [all-recursive] Error 1
gmake[1]: Leaving directory `/var/opt/packages/syslog-ng-3.3.2'
gmake: *** [all] Error 2
So, once again, I'm looking for some guidance. My system environment is mentioned in the original thread (below). This is a "really new" Solaris 10 setup (i.e. not some "ancient" version), and now with an operational 4.x gcc compiler, and I still can't seem to make this work.
Sorry to be a pest. I really do appreciate ALL of the assistance that has been given.
-----Original Message-----
From: syslog-ng-bounces at lists.balabit.hu [mailto:syslog-ng-bounces at lists.balabit.hu] On Behalf Of Balazs Scheidler
Sent: Tuesday, November 15, 2011 12:12 PM
To: Syslog-ng users' and developers' mailing list
Subject: Re: [syslog-ng] Solaris 10 support(?) in latest versions (e.g. 3.3.2)
On Tue, 2011-11-15 at 17:36 +0100, Pal Tamas wrote:
> Hi,
>
> On Tue, Nov 15, 2011 at 10:09:50AM -0600, Marvin Nipper wrote:
> > Apologies upfront for the length, but this addresses two different problems.
> >
> >
> > Environment: Solaris 10 x64 U10, fully patched (i.e. pretty much "new").
> > Currently operating (successfully) with 3.0.5, compiled on that platform, and
> > distributed to a number of other Solaris 10 x64 targets.
> >
> > Problem: Trying to "make the leap" to 3.3.2, and get current, but that version
> > seems to have numerous Solaris compatibility issues. Just trying to understand
> > the status of Solaris support.
> >
> >
> >
> > Problem #1
> > Latest packaging no longer works with Solaris-based "make".
> >
> > After a clean configure (output here):
> > syslog-ng Open Source Edition 3.3.2 configured
> > Compiler options:
> > compiler : gcc -std=gnu99
> > compiler options : -g -O2 -Wall -pthread -D_REENTRANT -D_PTHREADS
> > -I/usr/local/include/glib-2.0 -I/usr/local/lib/glib-2.0/include -I/usr/local/
> > include/eventlog -I/usr/local/ssl/include -DLIBNET_LIL_ENDIAN -I$
> > (top_srcdir)/lib/ivykis/lib/include -I$(top_builddir)/lib/ivykis/lib/include
> > -I$(top_srcdir)/lib/ivykis/modules/include -D_GNU_SOURCE -D_LARGEFILE_SOURCE
> > -D_FILE_OFFSET_BITS=64
> > linker flags : -R/usr/local/lib -lpthread
> > prefix : /usr/local
> > linking mode : dynamic
> > __thread keyword : yes
> > Submodules:
> > ivykis : internal
> > libmongo-client : internal
> > Features:
> > Debug symbols : no
> > GCC profiling : no
> > Memtrace : no
> > IPV6 support : yes
> > spoof-source support : yes
> > tcp-wrapper support : no
> > Linux capability support : no
> > PCRE support : no
> > Env wrapper support : no
> > systemd support : no (unit dir: none)
> > Modules:
> > Module search path : /usr/local/lib/syslog-ng
> > Default module list :
> > affile,afprog,afsocket,afuser,basicfuncs,csvparser,dbparser,syslogformat,afstreams
> > Sun STREAMS support (module): yes
> > SSL support (module) : yes
> > SQL support (module) : no
> > PACCT module (EXPERIMENTAL) : no
> > MongoDB destination (module): yes
> > JSON support (module) : no (using no)
> >
> >
> > ..... a simple "make" attempt now immediately results in this:
> > # make
> > make all-recursive
> > Making all in lib
> > make: Fatal error in reader: Makefile, line 991: Unexpected end of line seen
> > Current working directory /var/opt/packages/syslog-ng-3.3.2/lib
> > *** Error code 1
> > The following command caused the error:
> > fail= failcom='exit 1'; \
> > for f in x $MAKEFLAGS; do \
> > case $f in \
> > *=* | --[!k]*);; \
> > *k*) failcom='fail=yes';; \
> > esac; \
> > done; \
> > dot_seen=no; \
> > target=`echo all-recursive | sed s/-recursive//`; \
> > list='lib modules syslog-ng scripts tests doc contrib scl debian tgz2build
> > build'; for subdir in $list; do \
> > echo "Making $target in $subdir"; \
> > if test "$subdir" = "."; then \
> > dot_seen=yes; \
> > local_target="$target-am"; \
> > else \
> > local_target="$target"; \
> > fi; \
> > (CDPATH="${ZSH_VERSION+.}:" && cd $subdir && make $local_target) \
> > || eval $failcom; \
> > done; \
> > if test "$dot_seen" = "no"; then \
> > make "$target-am" || exit 1; \
> > fi; test -z "$fail"
> > make: Fatal error: Command failed for target `all-recursive'
> > Current working directory /var/opt/packages/syslog-ng-3.3.2
> > *** Error code 1
> > make: Fatal error: Command failed for target `all'
> >
> >
> > Not being a developer, per se, I spent time in Google-land, the general result
> > that I find is:
> > "These are caused because Solaris "make" doesn't accept the ‘GNU multi-line
> > Makefile format’. Use gmake instead."
> > But there are obviously folks who also say that this can be fixed by
> > "correcting" the problems in the Makefiles (so that would seem to be a
> > preferable alternative).
> The syslog-ng uses the "autoconfuse" (automake and autoconf tools)
> system to generate the makefiles needed for build. Since both autoconf
> and automake are part of the GNU project, the Makefiles generated by
> them are GNU make specific. Sorry there's no way around it.
>
> If you hadn't done it, please install the SUNWgcmn (Common GNU package)
> and the SUNWgmake from your Solaris install media. It'll put a gmake
> into /usr/sfw/bin.
> >
> >
> > Problem #2
> > Can no longer perform successful compilations, due to this error:
> > ld: fatal: relocation error: R_386_GOTOFF
> >
> > Same configure output as previously shown, followed by gmake (to get around the
> > first problem).
> >
> > Numerous modules compile successfully, but then this happens:
> > gcc -DHAVE_CONFIG_H -I. -I../.. -D_GNU_SOURCE -I../../lib/include -I../../lib/
> > include -I../../modules/include -Wall -g -O2 -D_REENTRANT -D_POSIX_C_SOURCE=
> > 199506L -D_POSIX_PTHREAD_SEMANTICS -D_XPG4_2 -MT iv_event_test.o -MD -MP -MF
> > .deps/iv_event_test.Tpo -c -o iv_event_test.o iv_event_test.c
> > mv -f .deps/iv_event_test.Tpo .deps/iv_event_test.Po
> > /bin/bash ../../libtool --tag=CC --mode=link gcc -Wall -g -O2 -D_REENTRANT
> > -D_POSIX_C_SOURCE=199506L -D_POSIX_PTHREAD_SEMANTICS -D_XPG4_2 -R/usr/local/
> > lib -o iv_event_test iv_event_test.o ../../lib/libivykis.la ../
> > libivykis-modules.la -lrt -lpthread -lsocket
> > libtool: link: gcc -Wall -g -O2 -D_REENTRANT -D_POSIX_C_SOURCE=199506L
> > -D_POSIX_PTHREAD_SEMANTICS -D_XPG4_2 -o iv_event_test iv_event_test.o ../../
> > lib/.libs/libivykis.a ../.libs/libivykis-modules.a /var/opt/packages/
> > syslog-ng-3.3.2/lib/ivykis/lib/.libs/libivykis.a -lrt -lpthread -lsocket -R/usr
> > /local/lib
> > ld: fatal: relocation error: R_386_GOTOFF: file ../.libs/libivykis-modules.a
> > (iv_event.o): symbol __tls: relocation illegal for TLS symbol
> > ld: fatal: relocation error: R_386_GOTOFF: file ../.libs/libivykis-modules.a
> > (iv_event.o): symbol __tls: relocation illegal for TLS symbol
> > ld: fatal: relocation error: R_386_GOTOFF: file ../.libs/libivykis-modules.a
> > (iv_event.o): symbol __tls: relocation illegal for TLS symbol
> > ld: fatal: relocation error: R_386_GOTOFF: file ../.libs/libivykis-modules.a
> > (iv_event.o): symbol __tls: relocation illegal for TLS symbol
> > collect2: ld returned 1 exit status
> > gmake[7]: *** [iv_event_test] Error 1
> > gmake[7]: Leaving directory `/var/opt/packages/syslog-ng-3.3.2/lib/ivykis/
> > modules/test'
> > gmake[6]: *** [all-recursive] Error 1
> > gmake[6]: Leaving directory `/var/opt/packages/syslog-ng-3.3.2/lib/ivykis/
> > modules'
> > gmake[5]: *** [all-recursive] Error 1
> > gmake[5]: Leaving directory `/var/opt/packages/syslog-ng-3.3.2/lib/ivykis'
> > gmake[4]: *** [all] Error 2
> > gmake[4]: Leaving directory `/var/opt/packages/syslog-ng-3.3.2/lib/ivykis'
> > gmake[3]: *** [all-recursive] Error 1
> > gmake[3]: Leaving directory `/var/opt/packages/syslog-ng-3.3.2/lib'
> > gmake[2]: *** [all] Error 2
> > gmake[2]: Leaving directory `/var/opt/packages/syslog-ng-3.3.2/lib'
> > gmake[1]: *** [all-recursive] Error 1
> > gmake[1]: Leaving directory `/var/opt/packages/syslog-ng-3.3.2'
> > gmake: *** [all] Error 2
> >
> >
> > Again, adopting the "Google is my friend" approach, this appears to be an issue
> > with "Solaris not supporting Visibility". (http://www.ilkda.com/compile/
> > Build_Problems.htm)
> > Most of the sites where I see this being discussed, with other (various) source
> > components, I also see the developers subsequently modifying their environment
> > to "adjust for" the Visibility issue in Solaris.
> >
> > So, again, not being a C developer myself, I guess I'm wondering whether I'm
> > just doing something wrong with 3.3.2 (i.e. something that I need to do
> > different from my 3.0.5 compilation), or whether this is truly just a "Solaris
> > thing" that has not been accounted for in the current code base?
> Looks like your gcc doesn't support TLS (Thread Local Storage) properly.
> The only way we were able to solve it by building a new (4.6.0) gcc
> following the steps in this blog post:
> http://jblopen.com/node/16
There's a bundled library in syslog-ng code, ivykis. The error happens
when linking one of the test programs for ivykis. I'm not sure why that
happens, it does link for us properly, but maybe because we use a
different compiler (as described by Folti).
However TLS (=Thread Local Storage) is not a mandatory compiler feature,
both syslog-ng and ivykis can operate without it, the issue is that the
configure script detects that your compiler/linker is supposed to
support it.
Check out syslog-ng-3.3.2/config.log and
syslog-ng-3.3.2/lib/ivykis/config.log, and look for "__thread" in the
logs, alternatively check config.h for both directories and check if
* syslog-ng/config.h: HAVE_THREAD_KEYWORD is defined
* syslog-ng/lib/ivykis/config.h: HAVE_TLS is defined.
It'll probably be defined. You can retry compilation by changing the
config.h files and undef those and recompile.
The only question is why it gets detected and not work at the end. This
test is used for both:
AC_LINK_IFELSE([AC_LANG_PROGRAM(
[[#include <pthread.h>
__thread int a;
]],
[a=0;])],
[ac_cv_have_tls=yes; AC_DEFINE_UNQUOTED(HAVE_THREAD_KEYWORD, 1, "Whether Transport Layer Security is supported by the system")])
I'm not sure if we can actually reliably detect if TLS is available. It
could make sense to make it a configure parameter after all. (which I'm
usually against...).
syslog-ng does support Solaris (the PE staff regularly compiles & tests
it under Solaris 8, 9, 10; sparc, i386, amd64), however it is not just
Solaris, gcc/binutils/etc are all part of the picture.
Keeping a "good" build environment on these platforms is not always
easy.
The use of __thread is completely optional in the codebase itself.
--
Bazsi
______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq
This e-mail may contain confidential and/or privileged information. If you are
not the intended recipient (or have received this e-mail in error) please
notify the sender immediately and destroy this e-mail. Any unauthorized
copying, disclosure or distribution of the material in this e-mail is strictly
forbidden.
More information about the syslog-ng
mailing list