[syslog-ng] Solaris 10 support(?) in latest versions (e.g. 3.3.2)
Balazs Scheidler
bazsi at balabit.hu
Fri Nov 25 12:56:21 CET 2011
On Thu, 2011-11-17 at 21:17 -0600, Marvin Nipper wrote:
>
> -----Original Message-----
> From: syslog-ng-bounces at lists.balabit.hu [mailto:syslog-ng-bounces at lists.balabit.hu] On Behalf Of Balazs Scheidler
> Sent: Thursday, November 17, 2011 1:47 PM
> To: Syslog-ng users' and developers' mailing list
> Subject: Re: [syslog-ng] Solaris 10 support(?) in latest versions (e.g. 3.3.2)
>
> On Thu, 2011-11-17 at 13:27 -0600, Marvin Nipper wrote:
> > OK. After a "day and a half" of "hoop jumping", I managed to build a gcc 4.4.2 compiler on Solaris 10. That's definitely _not_ a straightforward effort, and I had to "pick and choose" things from a number of different sites, to get that to work correctly.
> >
> > So... I have successfully used that new compiler to rebuild my working syslog-ng 3.0.5 executable, so I know that it works. (And I completely recompiled all of the underlying components, including pkg-config, libnet, glib, eventlog, and, openssl, so that none of the original gcc 3.4.3 "stuff" would be used.) I even validated (with ldd) that the new executable had correctly linked the new gcc components (and nothing from the old gcc 3.4.3 world).
> >
> >
> > So THEN, I tried, again, to compile the syslog-ng 3.3.2 components.
> >
> > The good news is that it did not abend on the previously reported TLS-related portion of the code! (a "small success"!).
> >
> >
> > The bad news is that it now gets to this point in the process (obviously a really long line):
> > /bin/bash ../libtool --tag=CC --mode=link gcc -std=gnu99 -g -O2 -Wall -pthrea
> > d -no-undefined -release 3.3.2 -R/usr/local/lib -o libsyslog-ng.la -rpath /usr/l
> > ocal/lib afinter.lo alarms.lo apphook.lo block-ref-parser.lo center.lo cfg.lo cf
> > g-lexer.lo cfg-parser.lo children.lo compat.lo control.lo dgroup.lo dnscache.lo
> > driver.lo filter.lo filter-expr-parser.lo globals.lo gprocess.lo gsockaddr.lo gs
> > ocket.lo logmatcher.lo logmpx.lo logmsg.lo logparser.lo logpipe.lo logprocess.lo
> > logproto.lo logqueue.lo logqueue-fifo.lo logreader.lo logrewrite.lo logsource.l
> > o logstamp.lo logtransport.lo logwriter.lo mainloop.lo memtrace.lo messages.lo m
> > isc.lo msg-format.lo nvtable.lo parser-expr-parser.lo persist-state.lo plugin.lo
> > pragma-parser.lo rewrite-expr-parser.lo scratch-buffers.lo serialize.lo sgroup.
> > lo stats.lo str-format.lo syslog-names.lo tags.lo templates.lo timeutils.lo util
> > s.lo value-pairs.lo cfg-lex.lo cfg-grammar.lo filter-expr-grammar.lo block-ref-g
> > rammar.lo pragma-grammar.lo parser-expr-grammar.lo rewrite-expr-grammar.lo -lpth
> > read -ldoor -lsocket -lrt -lnsl -L/usr/local/lib -lgmodule-2.0 -lgthread-2.0 -
> > lpthread -lthread -lrt -lglib-2.0 -L/usr/local/lib -levtlog -lresolv -ldl
> > -Wl,--whole-archive -L../lib/ivykis/lib -livykis -L../lib/ivykis/modules -livyk
> > is-modules -Wl,--no-whole-archive -lpthread
> >
> >
> > And then starts throwing out these errors:
> > ld: fatal: symbol 'g_trash_stack_push' is multiply-defined:
> > (file .libs/afinter.o type=FUNC; file .libs/alarms.o type=FUNC);
> > ld: fatal: symbol 'g_trash_stack_pop' is multiply-defined:
> > (file .libs/afinter.o type=FUNC; file .libs/alarms.o type=FUNC);
> > ld: fatal: symbol 'g_trash_stack_peek' is multiply-defined:
> > (file .libs/afinter.o type=FUNC; file .libs/alarms.o type=FUNC);
> > ld: fatal: symbol 'g_trash_stack_height' is multiply-defined:
> > (file .libs/afinter.o type=FUNC; file .libs/alarms.o type=FUNC);
> > ld: fatal: symbol 'g_bit_nth_msf' is multiply-defined:
> > (file .libs/afinter.o type=FUNC; file .libs/alarms.o type=FUNC);
> > ld: fatal: symbol 'g_bit_nth_lsf' is multiply-defined:
> > (file .libs/afinter.o type=FUNC; file .libs/alarms.o type=FUNC);
> > ld: fatal: symbol 'g_bit_storage' is multiply-defined:
> > (file .libs/afinter.o type=FUNC; file .libs/alarms.o type=FUNC);
> > ld: fatal: symbol 'g_trash_stack_push' is multiply-defined:
> > (file .libs/afinter.o type=FUNC; file .libs/apphook.o type=FUNC);
> > ld: fatal: symbol 'g_trash_stack_pop' is multiply-defined:
> > (file .libs/afinter.o type=FUNC; file .libs/apphook.o type=FUNC);
> > ld: fatal: symbol 'g_trash_stack_peek' is multiply-defined:
> > (file .libs/afinter.o type=FUNC; file .libs/apphook.o type=FUNC);
> > ld: fatal: symbol 'g_trash_stack_height' is multiply-defined:
> > (file .libs/afinter.o type=FUNC; file .libs/apphook.o type=FUNC);
> > ld: fatal: symbol 'g_bit_nth_msf' is multiply-defined:
> >
> >
> > It issues 427 "multiply-defined" error lines, similar to the ones above, before eventually ending with this final block of output:
> > ld: fatal: symbol 'g_bit_nth_lsf' is multiply-defined:
> > (file .libs/afinter.o type=FUNC; file .libs/rewrite-expr-grammar.o type=FUNC);
> > ld: fatal: symbol 'g_bit_storage' is multiply-defined:
> > (file .libs/afinter.o type=FUNC; file .libs/rewrite-expr-grammar.o type=FUNC);
> > ld: fatal: file processing errors. No output written to .libs/libsyslog-ng-3.3.2.so
>
> hmm.. this seems like a glib header <> library mess, as if it was
> defined in the header, and becoming part of the compiled module
> everywhere where the header was included into.
>
> This is how g_trash_stack_push() is defined in the glib headers (we're
> using 2.14.6):
>
> G_INLINE_FUNC void g_trash_stack_push (GTrashStack **stack_p,
> gpointer data_p);
>
> G_INLINE_FUNC is defined as a macro in the same file, this way:
>
> #if defined (G_HAVE_INLINE) && defined (__GNUC__) && defined (__STRICT_ANSI__)
> # undef inline
> # define inline __inline__
> #elif !defined (G_HAVE_INLINE)
> # undef inline
> # if defined (G_HAVE___INLINE__)
> # define inline __inline__
> # elif defined (G_HAVE___INLINE)
> # define inline __inline
> # else /* !inline && !__inline__ && !__inline */
> # define inline /* don't inline, then */
> # endif
> #endif
> #ifdef G_IMPLEMENT_INLINES
> # define G_INLINE_FUNC
> # undef G_CAN_INLINE
> #elif defined (__GNUC__)
> # if __GNUC_PREREQ (4,2) && defined (__STDC_VERSION__) \
> && __STDC_VERSION__ >= 199901L
> # define G_INLINE_FUNC extern __inline __attribute__ ((__gnu_inline__))
> # else
> # define G_INLINE_FUNC extern __inline
> # endif
> #elif defined (G_CAN_INLINE)
> # define G_INLINE_FUNC static inline
> #else /* can't inline */
> # define G_INLINE_FUNC
> #endif /* !G_INLINE_FUNC */
>
> The macros that this macro definition depends on are defined
> in glibconfig.h, usually installed in $prefix/lib/glib-2.0/include/glibconfig.h
>
> How do you tell syslog-ng where the glib libraries are? Are
> those autodetected? are you using pkg-config?
> [Marvin Nipper]
> I'm using pkg-config.
> When building GLIB, I set this variable, before the configure: PKG_CONFIG=/usr/local/bin/pkg-config
> And when building syslog-ng, I set these variables before running configure:
> PKG_CONFIG=/usr/local/bin/pkg-config
> PKG_CONFIG_PATH="$PKG_CONFIG_PATH:/usr/local/lib/pkgconfig:/usr/local/ssl/lib/pkgconfig"
>
> BUT... I downloaded and built the exact version of GLIB that you are using. I then kicked off the 3.3.2 build process again. I did NOT get the "multiply-defined" errors from before, so that seems to be resolved. But annoyingly, I now am having what seems to be a whole new, libnet related problem. It all starts with this stuff:
> libtool: link: gcc -std=gnu99 -shared -Wl,-z -Wl,text -Wl,-h -Wl,libafsocket-not
> ls.so -o .libs/libafsocket-notls.so .libs/libafsocket_notls_la-afsocket.o .libs
> /libafsocket_notls_la-afunix.o .libs/libafsocket_notls_la-afinet.o .libs/libafso
> cket_notls_la-afsocket-grammar.o .libs/libafsocket_notls_la-afsocket-parser.o .l
> ibs/libafsocket_notls_la-afsocket-plugin.o -R/var/opt/packages/syslog-ng-3.3.2
> /lib/.libs -R/usr/local/lib ../../lib/.libs/libsyslog-ng.so -L/usr/local/lib -L/
> var/opt/packages/syslog-ng-3.3.2/lib/ivykis/lib -L/var/opt/packages/syslog-ng-3.
> 3.2/lib/ivykis/modules -L/var/opt/packages/syslog-ng-3.3.2/lib/ivykis/lib/.libs
> -lsocket -lnsl -lnet -lpthread -lc -pthread -pthread^M
> Text relocation remains referenced^M
> against symbol offset in file^M
> .rodata (section) 0x7 /usr/lib/libnet.a(libnet_build_i
> p.o)^M
> .rodata.str1.1 (merged string section) 0xc /usr/lib/libnet.a(libnet
> _build_ip.o)^M
> .rodata (section) 0xdd /usr/lib/libnet.a(libnet_build_i
> p.o)^M
> .rodata.str1.1 (merged string section) 0xe2 /usr/lib/libnet.a(libnet
> _build_ip.o)^M
> .rodata (section) 0x122 /usr/lib/libnet.a(libnet_build_i
> p.o)^M
> .rodata.str1.1 (merged string section) 0x127 /usr/lib/libnet.a(libnet
> _build_ip.o)^M
>
>
> After MANY more similar lines (i.e. similar to the lines above), it eventually ends up here:
> errno 0x375 /usr/lib/libnet.a(libnet_if_addr
> .o)^M
> strpbrk 0x318 /usr/lib/libnet.a(libnet_link_dl
> pi.o)^M
> ld: fatal: relocations remain against allocatable but non-writable sections^M
> collect2: ld returned 1 exit status^M
> gmake[4]: *** [libafsocket-notls.la] Error 1^M
> gmake[4]: Leaving directory `/var/opt/packages/syslog-ng-3.3.2/modules/afsocket'
> ^M
> gmake[3]: *** [all] Error 2^M
> gmake[3]: Leaving directory `/var/opt/packages/syslog-ng-3.3.2/modules/afsocket'
> ^M
> gmake[2]: *** [all-recursive] Error 1^M
> gmake[2]: Leaving directory `/var/opt/packages/syslog-ng-3.3.2/modules'^M
> gmake[1]: *** [all-recursive] Error 1^M
> gmake[1]: Leaving directory `/var/opt/packages/syslog-ng-3.3.2'^M
> gmake: *** [all] Error 2^M
>
>
> I just feel like I'm going down the rabbit hole with all of this. I really do wonder how you build your Solaris 10 components, because no matter what I do (newest version of Solaris 10, newly built GCC 4.4.2 compiler, same GLIB, etc.), I just keep stepping into a new mess.
>
> I'm running libnet 1.1.2.1, which is the very same version that I've been using for MANY years, to build MANY versions of syslog-ng, up through 3.0.5, and it has worked flawlessly. So why this is now going bonkers, with that same version, is beyond me?? The only other version of libnet that I've ever found was libnet 0.10.11, and I could never get that to compile on Solaris.
>
> So... sorry for the whining. I've just put a lot of time and effort into trying to make this work, and feel like a dog chasing its tail. No matter what I "fix", or "upgrade", I just continue to find some "new thing" that seems to need to be changed, in order to try to make this work. If I had never previously built syslog-ng, I might understand that, but I have built many versions over the years, with no problems, and then it just seems to have "diverged" from a codebase that seems to be usable without a serious amount of augmentation to the stock Solaris components.
>
> Anyway... again, any and all input appreciated. I "seem to be" closer to getting this compiled, than I have been for a very long time. It would really be nice to see this compile clean. THANKS, as always, for your help.
>
I understand the frustration. syslog-ng has changed a lot in 3.2 and 3.3
and where only a single binary was produced, now a set of .so-s and
programs are created, not to mention threading, and TLS (=thread local
storage) variables, which we compile out unless dedected.
So building got a lot more difficult than in 3.0 times.
This time, your libnet library is probably not built with -fPIC, which
causes the linking to fail when trying to link it into libafsocket.so,
which is now a shared object, whereas it used to be part of a monolithic
executable.
Believe me, Solaris is one of the easier UNIXes to get on with, when
building relatively modern software. AIX is arcane beyond imagination
(e.g. shared objects embedded in .a files), HP-UX is buggy when it comes
to shared objects and relatively simple things like handling
LD_LIBRARY_PATH or rpath properly.
We've got dedicated staff to work with these UNIXes to keep syslog-ng in
a compilable state. We generally use a customized version of gcc,
binutils, etc. That's why I stopped providing OSE binaries: lots of work
which I have no time for, I'm sometimes happy that I can tag & publish a
release tarball. (two kids and a company to run)
The PE team is doing it as that's one of the driving force behind PE
sales, but it needs a top-notch guy who knows build tools and UNIXes
quite well.
--
Bazsi
More information about the syslog-ng
mailing list