[PATCH] afmongodb: Support system-installed libmongo-client at configure time.
Modify configure.in to recognise a suitable version (>= 0.1.0) of the libmongo-client library, and use it instead of the internal version if requested so. Signed-off-by: Gergely Nagy <algernon@balabit.hu> --- configure.in | 10 ++++++---- 1 files changed, 6 insertions(+), 4 deletions(-) diff --git a/configure.in b/configure.in index e5bf174..e7038fb 100644 --- a/configure.in +++ b/configure.in @@ -28,6 +28,7 @@ IVYKIS_MIN_VERSION="0.18" JSON_C_MIN_VERSION="0.7" JSON_GLIB_MIN_VERSION="0.12" PCRE_MIN_VERSION="6.1" +LMC_MIN_VERSION="0.1.0" dnl *************************************************************************** dnl Initial setup @@ -743,12 +744,13 @@ if test "x$with_libmongo_client" = "xinternal"; then AC_MSG_WARN([Internal libmongo-client sources not found in modules/afmongodb/libmongo-client]) with_libmongo_client="no" fi -else - AC_MSG_WARN([Don't know how to locate system installed libmongo-client.]) - with_libmongo_client="no" +elif test "x$with_libmongo_client" = "xsystem"; then + PKG_CHECK_MODULES(LIBMONGO, libmongo-client >= $LMC_MIN_VERSION,,with_libmongo_client="no") fi - +if test "x$with_libmongo_client" = "xno"; then + enable_mongodb="no" +fi dnl *************************************************************************** dnl misc features to be enabled -- 1.7.2.5
Gergely Nagy <algernon@balabit.hu> writes:
Modify configure.in to recognise a suitable version (>= 0.1.0) of the libmongo-client library, and use it instead of the internal version if requested so.
Also, please update the internal libmongo-client to v0.1.0, released a couple of hours ago. (It is tagged v0.1.0 in the upstream git tree at git://github.com/algernon/libmongo-client.git) The new version is API compatible (but not ABI compatible) with the current internal version, at least when considering the functions that the mongodb driver uses, so no other changes are neccessary. If the internal version needs to be statically linked by default, then a configure.gnu file should be added, similar to how it's done with lib/ivykis (the same file should work, actually). Compared to the current internal version, tons of fixes and enhancements went into the library. Though, most of those do not affect syslog-ng, as it uses very little of the library (so far, but I have plans to add a lot of neat stuff to the mongodb destination that depend on the new library version). Notable improvements that affect the current mongodb destination driver: * IPv6 and hostname support. Previously, the library - and in turn, the mongodb destination - only recognised dotted-decimal IPv4 addresses. * Endianness fixes. Parts of the Mongo wire protocol were sent in host endianness instead of little-endian (the request ID, in particular). * Use sendmsg() to send packets over the wire, which allows us to ignore SIGPIPE easily (and make the call return an error instead of raising a signal) on systems where MSG_NOSIGNAL is supported. * Faster and better memory allocations: instead of going to great lengths to ensure that memory allocations succeed, leave it up to glib to handle OOM cases and abort. This gets rid of a lot of if branches on fast paths. In certain cases, allocations were adjusted to reserve the proper amount of memory as soon as possible, so arrays won't have to grow later on. * Better test suite! 68% code coverage without tests that require a working mongodb setup, and over 95% when the full test suite is ran. This new version has also been tested on a number of platforms (including, but not limited to recent Linux platforms (debian stable & unstable), FreeBSD 8, Solaris (OpenSolaris and OpenIndiana), Debian GNU/kFreeBSD, different architectures (amd64, x86, ppc, armel)) and the test suite passed flawlessly on all of them. Gotchas: * The new library requires glib 2.12+ (compared to syslog-ng's 2.10), because of using some base64 functions introduced in that version. Though, that's only used in an example program, so with a little bit of hackery (disabling said program, and lowering the requirement) this can be worked around. -- |8]
Gergely Nagy <algernon@balabit.hu> writes:
Gergely Nagy <algernon@balabit.hu> writes:
Modify configure.in to recognise a suitable version (>= 0.1.0) of the libmongo-client library, and use it instead of the internal version if requested so.
Also, please update the internal libmongo-client to v0.1.0, released a couple of hours ago. (It is tagged v0.1.0 in the upstream git tree at git://github.com/algernon/libmongo-client.git)
Actually, since this hasn't been done yet - I plan to release 0.1.1 tomorrow, which would be a better candidate: it has a few new things that I plan to use in the mongodb driver in 3.4, and also fixes a problem on a few platform + gcc combinations. I played too many pointer games, and on a few architectures, a certain GCC version overoptimised it, and the result wasn't pretty. That fixed, and the new stuff would be great to have in the internal library. The NEWS file in my git tree has more info about what 0.1.1 will bring. It's API and ABI compatible with 0.1.0, and so on and so forth. So updating to 0.1.1 would be better in the long run, as I won't have to ask for another update once I get to publishing the interesting mongodb stuff for 3.4 ;) -- |8]
Gergely Nagy <algernon@balabit.hu> writes:
Gergely Nagy <algernon@balabit.hu> writes:
Gergely Nagy <algernon@balabit.hu> writes:
Modify configure.in to recognise a suitable version (>= 0.1.0) of the libmongo-client library, and use it instead of the internal version if requested so.
Also, please update the internal libmongo-client to v0.1.0, released a couple of hours ago. (It is tagged v0.1.0 in the upstream git tree at git://github.com/algernon/libmongo-client.git)
Actually, since this hasn't been done yet - I plan to release 0.1.1 tomorrow, which would be a better candidate: it has a few new things that I plan to use in the mongodb driver in 3.4, and also fixes a problem on a few platform + gcc combinations. I played too many pointer games, and on a few architectures, a certain GCC version overoptimised it, and the result wasn't pretty.
That fixed, and the new stuff would be great to have in the internal library.
The NEWS file in my git tree has more info about what 0.1.1 will bring. It's API and ABI compatible with 0.1.0, and so on and so forth.
So updating to 0.1.1 would be better in the long run, as I won't have to ask for another update once I get to publishing the interesting mongodb stuff for 3.4 ;)
...and today 0.1.2 is out, and here goes another ping to update the internal copy, if possible. An updated MongoDB destination is available in my modules collection, that makes use of some of the new features that are not present in the internal copy yet. The downside is that it requires glib >= 2.16, slightly newer than what syslog-ng requires right now, but still pretty damn old (released 2008-03-10, so more than 3 years old). -- |8]
On Fri, 2011-07-01 at 14:49 +0200, Gergely Nagy wrote:
Gergely Nagy <algernon@balabit.hu> writes:
Gergely Nagy <algernon@balabit.hu> writes:
Gergely Nagy <algernon@balabit.hu> writes:
Modify configure.in to recognise a suitable version (>= 0.1.0) of the libmongo-client library, and use it instead of the internal version if requested so.
Also, please update the internal libmongo-client to v0.1.0, released a couple of hours ago. (It is tagged v0.1.0 in the upstream git tree at git://github.com/algernon/libmongo-client.git)
Actually, since this hasn't been done yet - I plan to release 0.1.1 tomorrow, which would be a better candidate: it has a few new things that I plan to use in the mongodb driver in 3.4, and also fixes a problem on a few platform + gcc combinations. I played too many pointer games, and on a few architectures, a certain GCC version overoptimised it, and the result wasn't pretty.
That fixed, and the new stuff would be great to have in the internal library.
The NEWS file in my git tree has more info about what 0.1.1 will bring. It's API and ABI compatible with 0.1.0, and so on and so forth.
So updating to 0.1.1 would be better in the long run, as I won't have to ask for another update once I get to publishing the interesting mongodb stuff for 3.4 ;)
...and today 0.1.2 is out, and here goes another ping to update the internal copy, if possible.
An updated MongoDB destination is available in my modules collection, that makes use of some of the new features that are not present in the internal copy yet.
The downside is that it requires glib >= 2.16, slightly newer than what syslog-ng requires right now, but still pretty damn old (released 2008-03-10, so more than 3 years old).
I've updated the internal copy (just pushed out the submodules change), however I'd be interested how much it depends on glib 2.16? Our build systems have 2.14 deployed, and our last attempt to update that failed, because newer glib is not trivial to compile on some of our non-Linux platforms (at least in its pure, upstream form, sunfreeware/perzl carries a quite recent glib). So is the glib dependency easy to get rid of, or it is something more inherent? -- Bazsi
Balazs Scheidler <bazsi@balabit.hu> writes:
The downside is that it requires glib >= 2.16, slightly newer than what syslog-ng requires right now, but still pretty damn old (released 2008-03-10, so more than 3 years old).
I've updated the internal copy (just pushed out the submodules change), however I'd be interested how much it depends on glib 2.16? Our build systems have 2.14 deployed, and our last attempt to update that failed, because newer glib is not trivial to compile on some of our non-Linux platforms (at least in its pure, upstream form, sunfreeware/perzl carries a quite recent glib).
So is the glib dependency easy to get rid of, or it is something more inherent?
It's needed for authentication (not used by the mongodb destination at the moment, but I plan to add authentication support later), and later on for GridFS (but that's irrelevant for syslog-ng). I suppose I can add a little compat layer that'd fall back to OpenSSL if glib is too old. It's only some MD5 stuff I use from 2.16. For 0.1.3 (ETA: July 22nd), I'll drop back to requiring glib >= 2.12 only, and using either GChecksum from glib >= 2.16 if present, or OpenSSL otherwise. All in all, it's reasonably easy to get rid of the hard 2.16 dependency, and I'll do that for the next release. However, libmongo-client will depend on either glib >= 2.16 OR OpenSSL, unless you have strong objections, in which case I'll make the depending parts of it optional (but I'd like to not do that, if possible). -- |8]
Gergely Nagy <algernon@balabit.hu> writes:
Balazs Scheidler <bazsi@balabit.hu> writes:
The downside is that it requires glib >= 2.16, slightly newer than what syslog-ng requires right now, but still pretty damn old (released 2008-03-10, so more than 3 years old).
[...] So is the glib dependency easy to get rid of, or it is something more inherent? [...] All in all, it's reasonably easy to get rid of the hard 2.16 dependency, and I'll do that for the next release.
Done. libmongo-client 0.1.3 has been released as of an hour or so ago, tagged in git and all that. It removes the need for glib >= 2.16, and will work with 2.12 and up. When building with < 2.16, OpenSSL becomes a requirement, as I use its MD5 functions to emulate GChecksum. Any version of OpenSSL will do, but the configure script was set up for 0.9.8. As far as I remember, that's the same as syslog-ng's, so it should be fine. -- |8]
thanks. it really helps me, as upgrading glib would draw a lot of effort, which I'd like to spend elsewhere. ----- Original message -----
Gergely Nagy <algernon@balabit.hu> writes:
Balazs Scheidler <bazsi@balabit.hu> writes:
The downside is that it requires glib >= 2.16, slightly newer than what syslog-ng requires right now, but still pretty damn old (released 2008-03-10, so more than 3 years old).
[...] So is the glib dependency easy to get rid of, or it is something more inherent? [...] All in all, it's reasonably easy to get rid of the hard 2.16 dependency, and I'll do that for the next release.
Done. libmongo-client 0.1.3 has been released as of an hour or so ago, tagged in git and all that. It removes the need for glib >= 2.16, and will work with 2.12 and up. When building with < 2.16, OpenSSL becomes a requirement, as I use its MD5 functions to emulate GChecksum.
Any version of OpenSSL will do, but the configure script was set up for 0.9.8. As far as I remember, that's the same as syslog-ng's, so it should be fine.
-- |8]
Hi, On Wed, 2011-07-20 at 19:44 +0200, Balazs Scheidler wrote:
thanks. it really helps me, as upgrading glib would draw a lot of effort, which I'd like to spend elsewhere.
I've updated the internal libmongo-client to 0.1.3. I've needed a fix for g_checksum_new() autodetection, as it didn't use the proper -L linker option, thus it used the system installed glib, instead of the one in my private prefix. Otherwise afmongodb compiled nicely with 0.1.3, I haven't tested though. -- Bazsi
Balazs Scheidler <bazsi@balabit.hu> writes:
Hi,
On Wed, 2011-07-20 at 19:44 +0200, Balazs Scheidler wrote:
thanks. it really helps me, as upgrading glib would draw a lot of effort, which I'd like to spend elsewhere.
I've updated the internal libmongo-client to 0.1.3. I've needed a fix for g_checksum_new() autodetection, as it didn't use the proper -L linker option, thus it used the system installed glib, instead of the one in my private prefix.
Argh, right. I forgot to test this case. I'll pick your patch over, thank you! -- |8]
Hi, first of all, sorry for bringing this old topic back. Tuesday 19 of July 2011 20:55:54 Gergely Nagy wrote:
Gergely Nagy <algernon@balabit.hu> writes:
Balazs Scheidler <bazsi@balabit.hu> writes:
The downside is that it requires glib >= 2.16, slightly newer than what syslog-ng requires right now, but still pretty damn old (released 2008-03-10, so more than 3 years old). [...] So is the glib dependency easy to get rid of, or it is something more inherent? [...] All in all, it's reasonably easy to get rid of the hard 2.16 dependency, and I'll do that for the next release.
Done. libmongo-client 0.1.3 has been released as of an hour or so ago, tagged in git and all that. It removes the need for glib >= 2.16, and will work with 2.12 and up. When building with < 2.16, OpenSSL becomes a requirement, as I use its MD5 functions to emulate GChecksum.
It turns out this is not entirely true. HEAD of 3.3 does not compile on RHEL5, which has glib 2.12, because those 3 examples: modules/afmongodb/libmongo-client/examples/mongo-dump.c modules/afmongodb/libmongo-client/examples/gridfs.c modules/afmongodb/libmongo-client/examples/bson-inspect.c use g_option_context_get_help(), which - according to documentation at http://developer.gimp.org/api/2.0/glib/ix09.html - is new in 2.14. I've replaced g_option_context_get_help() occurences with simple static gchar *help = "You must specify --this and --that\n"; and it seems to build fine. Removind dependency on glib 2.14 would surely help some RHEL5-dependent folks like me. Please consider this, preferably before 3.3.0 final is out. :) Regards, Jakub. -- Jakub Jankowski|shasta@toxcorp.com|http://toxcorp.com/ GPG: FCBF F03D 9ADB B768 8B92 BB52 0341 9037 A875 942D
Jakub Jankowski <shasta@toxcorp.com> writes:
Done. libmongo-client 0.1.3 has been released as of an hour or so ago, tagged in git and all that. It removes the need for glib >= 2.16, and will work with 2.12 and up. When building with < 2.16, OpenSSL becomes a requirement, as I use its MD5 functions to emulate GChecksum.
It turns out this is not entirely true. HEAD of 3.3 does not compile on RHEL5, which has glib 2.12, because those 3 examples:
modules/afmongodb/libmongo-client/examples/mongo-dump.c modules/afmongodb/libmongo-client/examples/gridfs.c modules/afmongodb/libmongo-client/examples/bson-inspect.c
use g_option_context_get_help(), which - according to documentation at http://developer.gimp.org/api/2.0/glib/ix09.html - is new in 2.14.
I've replaced g_option_context_get_help() occurences with simple static gchar *help = "You must specify --this and --that\n"; and it seems to build fine.
Thanks for catching this! I'll fix the examples accordingly. -- |8]
Gergely Nagy <algernon@balabit.hu> writes:
Done. libmongo-client 0.1.3 has been released as of an hour or so ago, tagged in git and all that. It removes the need for glib >= 2.16, and will work with 2.12 and up. When building with < 2.16, OpenSSL becomes a requirement, as I use its MD5 functions to emulate GChecksum.
It turns out this is not entirely true. HEAD of 3.3 does not compile on RHEL5, which has glib 2.12, because those 3 examples:
modules/afmongodb/libmongo-client/examples/mongo-dump.c modules/afmongodb/libmongo-client/examples/gridfs.c modules/afmongodb/libmongo-client/examples/bson-inspect.c
use g_option_context_get_help(), which - according to documentation at http://developer.gimp.org/api/2.0/glib/ix09.html - is new in 2.14.
I've replaced g_option_context_get_help() occurences with simple static gchar *help = "You must specify --this and --that\n"; and it seems to build fine.
Thanks for catching this! I'll fix the examples accordingly.
Done in my git repo. I solved the problem by calling g_option_context_parse() again, with an explicitly constructed argv, that contains "--help", in case the help needs to be displayed. This is a gross and ugly hack, but has the advantage of working on 2.12 aswell as anything newer, and will display the correct help output without manually having to keep the 2.12 and post-2.12 version in sync. Bazsi, can you update the internal libmongo-client to my latest master? (only documentation & packaging related changes since 0.1.3 + the above hack) -- |8]
On Tue, 2011-09-06 at 09:40 +0200, Gergely Nagy wrote:
Gergely Nagy <algernon@balabit.hu> writes:
Done. libmongo-client 0.1.3 has been released as of an hour or so ago, tagged in git and all that. It removes the need for glib >= 2.16, and will work with 2.12 and up. When building with < 2.16, OpenSSL becomes a requirement, as I use its MD5 functions to emulate GChecksum.
It turns out this is not entirely true. HEAD of 3.3 does not compile on RHEL5, which has glib 2.12, because those 3 examples:
modules/afmongodb/libmongo-client/examples/mongo-dump.c modules/afmongodb/libmongo-client/examples/gridfs.c modules/afmongodb/libmongo-client/examples/bson-inspect.c
use g_option_context_get_help(), which - according to documentation at http://developer.gimp.org/api/2.0/glib/ix09.html - is new in 2.14.
I've replaced g_option_context_get_help() occurences with simple static gchar *help = "You must specify --this and --that\n"; and it seems to build fine.
Thanks for catching this! I'll fix the examples accordingly.
Done in my git repo. I solved the problem by calling g_option_context_parse() again, with an explicitly constructed argv, that contains "--help", in case the help needs to be displayed.
This is a gross and ugly hack, but has the advantage of working on 2.12 aswell as anything newer, and will display the correct help output without manually having to keep the 2.12 and post-2.12 version in sync.
Bazsi, can you update the internal libmongo-client to my latest master? (only documentation & packaging related changes since 0.1.3 + the above hack)
I pulled your latest changes, it is tagged with "syslog-ng-3.3-ref2", whereas the old, previous state has become tagged too (with syslog-ng-3.3-ref1), so that its commit is not lost in the public repo. Thanks, Gergely. -- Bazsi
Balazs Scheidler <bazsi@balabit.hu> writes:
I pulled your latest changes, it is tagged with "syslog-ng-3.3-ref2", whereas the old, previous state has become tagged too (with syslog-ng-3.3-ref1), so that its commit is not lost in the public repo.
-ref2 does not seem to be pushed, and git submodule update fails. Which is weird, as I see ref2 on http://git.balabit.hu/ -- |8]
participants (4)
-
Balazs Scheidler
-
Gergely Nagy
-
Gergely Nagy
-
Jakub Jankowski