[syslog-ng] MongoDB destination driver

[Gergely Nagy]

Greetings!

I was contemplating switching one of the websites I maintain from
postgresql to one of these document store things, and while wondering
about how to evaluate the options best, I stumbled into a short
article on the MongoDB website:
http://blog.mongodb.org/post/172254834/mongodb-is-fantastic-for-logging,
which promptly led me to start pondering about how hard - or easy, as
it turns out - would it be to write a MongoDB destination driver for
syslog-ng?

Only four hours later, I can present you the mongodb destination
driver, available from my git repo (note the -b option, it is
important):

  $ git clone -b algernon/dest/mongodb
git://git.madhouse-project.org/syslog-ng/syslog-ng-3.2.git

Of course, one can browse the sources on the web too:
http://git.madhouse-project.org/syslog-ng/syslog-ng-3.2/tree/modules/afmongodb?h=algernon/dest/mongodb

It is using the MongoDB C client library
(http://www.mongodb.org/display/DOCS/C+Language+Center) - I simply
embedded the sources for now, lacking a better option. Once compiled,
one can already begin using it with the default options:

  destination d_mongodb { mongodb(); };

This will try to connect to localhost:27017, and use the logs
collection in the syslog-ng database, and will log all the standard
fields. Of course, all of those are configurable!

To demonstrate all the - currently - available options, the
destination definition above is the same as the following:

  destination d_mongodb {
    mongodb(
      host("localhost")
      port(27017)
      database("syslog-ng")
      collection("logs")
      keys("date", "facility", "level", "host", "program", "pid", "message")
      values("${R_YEAR}-${R_MONTH}-${R_DAY}
${R_HOUR}:${R_MIN}:${R_SEC}", "$FACILITY", "$LEVEL", "$HOST",
"$PROGRAM", "$PID", "$MSGONLY")
    );
  };

A few things, like authentication and some template options are not
configurable yet, partly because I didn't figure out what they're good
for, or how they work. But I will get there at some point, especially
if there's interest in said features.

All in all, I'm very happy that I could cook up a fairly simple
destination driver within a few hours, having no prior experience
writing one. The syslog-ng code is amazing, by the way, it was a
breeze to navigate through and find the stuff I needed to make this
driver work.

Mind you, this is very new code, and I haven't tested it extensively,
but I do have some great plans involving syslog-ng and mongodb >;)

Hope you like the code, and perhaps find it useful!

-- 
|8]