compiling problems --enable-mongodb --enable-json --enable-redis
Hi, I am trying to get syslog-ng to be the front-end for a larger log collection architecture including mongodb, redis, elasticsearch, possibly logstash & kibana. Have been trying to build syslog-ng with the necessary modules and failing miserably. I know it must be me - that said, I have spent several hours trying to get this to compile on both RHEL 5 and Ubuntu 14.04 I ran into a few things that might be useful (or not), but since the platforms and libraries installed are different, yet the end error is the same I thought I would ask for other eyes / brains to help here... Thanks for any assistance! Jim RHEL 5: I have installed json-c & redis-stable which provides these libraries: ./redis-stable/deps/hiredis/hiredis.h ./redis-stable/deps/hiredis/libhiredis.a and created links here to try and get around issues: /usr/lib/hiredis /usr/lib/hiredis/libhiredis.a /usr/include/hiredis /usr/include/hiredis/hiredis.h configure succeeds with this: Modules: Module search path : /usr/local/lib/syslog-ng Sun STREAMS support (module): no SSL support (module) : no SQL support (module) : no PACCT module (EXPERIMENTAL) : no MongoDB destination (module): yes JSON support (module) : yes SMTP support (module) : no AMQP destination (module) : yes STOMP destination (module) : yes GEOIP support (module) : no Redis support (module) : yes Make fails here: CC modules/redis/modules_redis_libredis_la-redis-grammar.lo CC modules/redis/modules_redis_libredis_la-redis.lo CC modules/redis/modules_redis_libredis_la-redis-parser.lo CCLD modules/redis/libredis.la /usr/bin/ld: cannot find -lhiredis collect2: ld returned 1 exit status make[2]: *** [modules/redis/libredis.la] Error 1 make[1]: *** [all-recursive] Error 1 make: *** [all] Error 2 [n0142566@VDDP13E-F1A47ED syslog-ng-3.5.4.1]$ Ubuntu: I have installed json-c, redis-stable & libredis: /usr/local/lib/libredis.so.1.0.0 /usr/local/lib/libredis.1 /usr/local/lib/libredis.la /usr/local/include/redis.h configure succeeds with this: Modules: Module search path : /usr/local/lib/syslog-ng Sun STREAMS support (module): no SSL support (module) : yes SQL support (module) : no PACCT module (EXPERIMENTAL) : no MongoDB destination (module): yes JSON support (module) : yes SMTP support (module) : no AMQP destination (module) : yes STOMP destination (module) : yes GEOIP support (module) : no Redis support (module) : yes Make fails with: CC modules/redis/modules_redis_libredis_la-redis-grammar.lo CC modules/redis/modules_redis_libredis_la-redis.lo CC modules/redis/modules_redis_libredis_la-redis-parser.lo CCLD modules/redis/libredis.la /usr/bin/ld: cannot find -lhiredis collect2: ld returned 1 exit status make[2]: *** [modules/redis/libredis.la] Error 1 make[1]: *** [all-recursive] Error 1 make: *** [all] Error 2 [n0142566@picard:~/src/syslog-ng-3.5.5$ ===========================================
<jrhendri@roadrunner.com> writes:
RHEL 5: I have installed json-c & redis-stable which provides these libraries:
./redis-stable/deps/hiredis/hiredis.h ./redis-stable/deps/hiredis/libhiredis.a
and created links here to try and get around issues: /usr/lib/hiredis /usr/lib/hiredis/libhiredis.a /usr/include/hiredis /usr/include/hiredis/hiredis.h
[...]
Make fails here:
CC modules/redis/modules_redis_libredis_la-redis-grammar.lo CC modules/redis/modules_redis_libredis_la-redis.lo CC modules/redis/modules_redis_libredis_la-redis-parser.lo CCLD modules/redis/libredis.la /usr/bin/ld: cannot find -lhiredis collect2: ld returned 1 exit status make[2]: *** [modules/redis/libredis.la] Error 1 make[1]: *** [all-recursive] Error 1 make: *** [all] Error 2 [n0142566@VDDP13E-F1A47ED syslog-ng-3.5.4.1]$
Can you run make with a V=1 parameter? That will tell us all the parameters it passes to gcc when compiling and linking. The problem is likely that the library is not on the search path, and for some reason, we don't put it there either. Running with V=1 should reveal what the issue is, more clearly. -- |8]
Hi, I made a binary rpm from sources'rpm of syslog-ng-3.5.4.1 For a rhel 6.x. I had two issues : 1_ removing things I don't want or doesn't compile as geoip and others 2_ backporting to an old filesystem < 3. Recent syslog-ng are made for a rhel's filesystem v3.x, where rhel 6.x is un v2.x In short and from memory, it simply moves /lib to /usr/lib and same For /bin. I had to set some variable on my spec file and remove dependency to filesystem v3.x. Variables are like : Prefix=/ Libdir=/lib Execdir=/bin In holidays, I am away from keyboard. So all this is only pointing a direction for you and from memory. Regards, Nicolas. Le 22 juil. 2014 14:22, "Gergely Nagy" <algernon@balabit.hu> a écrit :
<jrhendri@roadrunner.com> writes:
RHEL 5: I have installed json-c & redis-stable which provides these libraries:
./redis-stable/deps/hiredis/hiredis.h ./redis-stable/deps/hiredis/libhiredis.a
and created links here to try and get around issues: /usr/lib/hiredis /usr/lib/hiredis/libhiredis.a /usr/include/hiredis /usr/include/hiredis/hiredis.h
[...]
Make fails here:
CC modules/redis/modules_redis_libredis_la-redis-grammar.lo CC modules/redis/modules_redis_libredis_la-redis.lo CC modules/redis/modules_redis_libredis_la-redis-parser.lo CCLD modules/redis/libredis.la /usr/bin/ld: cannot find -lhiredis collect2: ld returned 1 exit status make[2]: *** [modules/redis/libredis.la] Error 1 make[1]: *** [all-recursive] Error 1 make: *** [all] Error 2 [n0142566@VDDP13E-F1A47ED syslog-ng-3.5.4.1]$
Can you run make with a V=1 parameter? That will tell us all the parameters it passes to gcc when compiling and linking. The problem is likely that the library is not on the search path, and for some reason, we don't put it there either.
Running with V=1 should reveal what the issue is, more clearly.
-- |8]
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
Thanks! that was helpful - now get back on holiday :-) ---- "Nicolas Fédou" <nicolas.fedou@gmail.com> wrote:
Hi, I made a binary rpm from sources'rpm of syslog-ng-3.5.4.1 For a rhel 6.x. I had two issues : 1_ removing things I don't want or doesn't compile as geoip and others 2_ backporting to an old filesystem < 3.
Recent syslog-ng are made for a rhel's filesystem v3.x, where rhel 6.x is un v2.x In short and from memory, it simply moves /lib to /usr/lib and same For /bin. I had to set some variable on my spec file and remove dependency to filesystem v3.x. Variables are like : Prefix=/ Libdir=/lib Execdir=/bin
In holidays, I am away from keyboard. So all this is only pointing a direction for you and from memory.
Regards, Nicolas. Le 22 juil. 2014 14:22, "Gergely Nagy" <algernon@balabit.hu> a écrit :
<jrhendri@roadrunner.com> writes:
RHEL 5: I have installed json-c & redis-stable which provides these libraries:
./redis-stable/deps/hiredis/hiredis.h ./redis-stable/deps/hiredis/libhiredis.a
and created links here to try and get around issues: /usr/lib/hiredis /usr/lib/hiredis/libhiredis.a /usr/include/hiredis /usr/include/hiredis/hiredis.h
[...]
Make fails here:
CC modules/redis/modules_redis_libredis_la-redis-grammar.lo CC modules/redis/modules_redis_libredis_la-redis.lo CC modules/redis/modules_redis_libredis_la-redis-parser.lo CCLD modules/redis/libredis.la /usr/bin/ld: cannot find -lhiredis collect2: ld returned 1 exit status make[2]: *** [modules/redis/libredis.la] Error 1 make[1]: *** [all-recursive] Error 1 make: *** [all] Error 2 [n0142566@VDDP13E-F1A47ED syslog-ng-3.5.4.1]$
Can you run make with a V=1 parameter? That will tell us all the parameters it passes to gcc when compiling and linking. The problem is likely that the library is not on the search path, and for some reason, we don't put it there either.
Running with V=1 should reveal what the issue is, more clearly.
-- |8]
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
OK - on the ubuntu box I ran a "make distclean" then re-ran the configure and ran: make V=1 > make.out 2>make.err then make V=1 > make.all 2>& all three gzipped attached. I really appreciate the help! Jim ---- Gergely Nagy <algernon@balabit.hu> wrote:
<jrhendri@roadrunner.com> writes:
RHEL 5: I have installed json-c & redis-stable which provides these libraries:
./redis-stable/deps/hiredis/hiredis.h ./redis-stable/deps/hiredis/libhiredis.a
and created links here to try and get around issues: /usr/lib/hiredis /usr/lib/hiredis/libhiredis.a /usr/include/hiredis /usr/include/hiredis/hiredis.h
[...]
Make fails here:
CC modules/redis/modules_redis_libredis_la-redis-grammar.lo CC modules/redis/modules_redis_libredis_la-redis.lo CC modules/redis/modules_redis_libredis_la-redis-parser.lo CCLD modules/redis/libredis.la /usr/bin/ld: cannot find -lhiredis collect2: ld returned 1 exit status make[2]: *** [modules/redis/libredis.la] Error 1 make[1]: *** [all-recursive] Error 1 make: *** [all] Error 2 [n0142566@VDDP13E-F1A47ED syslog-ng-3.5.4.1]$
Can you run make with a V=1 parameter? That will tell us all the parameters it passes to gcc when compiling and linking. The problem is likely that the library is not on the search path, and for some reason, we don't put it there either.
Running with V=1 should reveal what the issue is, more clearly.
-- |8] ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
<jrhendri@roadrunner.com> writes:
OK - on the ubuntu box I ran a "make distclean" then re-ran the configure and ran: make V=1 > make.out 2>make.err
It looks like you need to pass LDFLAGS="-L/usr/lib/hiredis" (or LDFLAGS="-L$(pwd)/redis-stable/deps/hiredis", if you don't want to symlink it) to configure in order for make to find the library later. Using --with-libhiredisdir=$(pwd)/redis-stable/deps/hiredis would also work, if you had the hiredis.h header in an "include" subdirectory there, and libhiredis.a in a "lib" subdir. -- |8]
That was it! (it's been too long since I did any c/C++ development) I will pass along that I needed to specify the full path on the RHEL box. That is: This worked on Ubuntu make LDFLAGS="-L../redis-stable/deps/hiredis" but RHEL didn't like the relative path so I needed to be explicit Now I can start playing again :-) I have been working on things like parsing -line logs that are interleaved, using an external parsing program to build the related log lines into a structure then writing to mongodb (specifically email logs where an external connection may carry many messages, and you see interleaved syslog coming from several mail gateways) I also have started playing with ELK (elasticsearsh, logstash, kibana) and I think there is some interesting stuff using redis as the frontend for elasticsearch (maybe not needing logstash) I think these modules for syslog-ng will be very useful. Thanks again!! Jim ---- Gergely Nagy <algernon@balabit.hu> wrote:
<jrhendri@roadrunner.com> writes:
OK - on the ubuntu box I ran a "make distclean" then re-ran the configure and ran: make V=1 > make.out 2>make.err
It looks like you need to pass LDFLAGS="-L/usr/lib/hiredis" (or LDFLAGS="-L$(pwd)/redis-stable/deps/hiredis", if you don't want to symlink it) to configure in order for make to find the library later.
Using --with-libhiredisdir=$(pwd)/redis-stable/deps/hiredis would also work, if you had the hiredis.h header in an "include" subdirectory there, and libhiredis.a in a "lib" subdir.
-- |8] ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
participants (3)
-
Gergely Nagy
-
jrhendri＠roadrunner.com
-
Nicolas Fédou