[syslog-ng] MongoDB destination driver

[Martin Holste]

Great idea to have a dedicated, user-configurable sub-key.  One
suggestion: I think that key names cannot contain dots in Mongo.  They
don't really make sense because this:

"patterndb" : {
     ".classifier.class" : "system",
     ".classifier.rule_id" : "4dd5a329-da83-4876-a431-ddcb59c2858c",
     "usracct.authmethod" : "publickey for algernon from ::1 port 59690 ssh2",
     "usracct.username" : "algernon from ::1 port 59690 ssh2",
     "usracct.device" : "::1 port 59690 ssh2",
     "usracct.service" : "ssh2",
     "usracct.type" : "login",
     "usracct.sessionid" : "12674",
     "usracct.application" : "sshd",
     "secevt.verdict" : "ACCEPT"
 }

should really look like this:

"patterndb" : {
     "classifier": {
        "class" : "system",
        "rule_id" : "4dd5a329-da83-4876-a431-ddcb59c2858c"
      },
     "usracct": {
       "authmethod" : "publickey for algernon from ::1 port 59690 ssh2",
       "username" : "algernon from ::1 port 59690 ssh2",
       "device" : "::1 port 59690 ssh2",
       "service" : "ssh2",
       "type" : "login",
       "sessionid" : "12674",
       "application" : "sshd",
     },
     "secevt":{
       "verdict" : "ACCEPT"
     }
 }

I recognize, however, that this is not a trivial conversion.  As a
start, just doing a simple substitution of "." for "_" on keys would
probably work just fine.

On Mon, Jan 3, 2011 at 3:02 PM, Gergely Nagy <algernon at balabit.hu> wrote:
>> The patch looks good on first read, but I'll have a closer look tonight,
>> and run a quick benchmark aswell, if all goes well.
>
> The patch looked fine on the second read too, and I integrated it, with
> a few changes:
>
> Instead of using a flag, I introduced a patterndb_key("foo") setting,
> which, if turned on, will put the patterndb results under the specified
> key, as a sub-document. If not specified, it will do nothing extra.
>
> In my opinion, this solution is clearer, and results in a better
> structured log entry.
>
> Usage is like this:
>
> destination d_mongo {
>  mongodb(
>    patterndb_key("patterndb")
>  );
> };
>
> The resulting log entry in mongodb looks something like this:
>
>> db.logs.find()
> { "_id" : ObjectId("4d2235525edd07af78f648f9"), "date" : "2011-01-03 21:45:06", "facility" : "auth",
>  "level" : "info", "host" : "localhost", "program" : "sshd", "pid" : "12674",
>  "message" : "Accepted publickey for algernon from ::1 port 59690 ssh2",
>  "patterndb" : {
>      ".classifier.class" : "system",
>      ".classifier.rule_id" : "4dd5a329-da83-4876-a431-ddcb59c2858c",
>      "usracct.authmethod" : "publickey for algernon from ::1 port 59690 ssh2",
>      "usracct.username" : "algernon from ::1 port 59690 ssh2",
>      "usracct.device" : "::1 port 59690 ssh2",
>      "usracct.service" : "ssh2",
>      "usracct.type" : "login",
>      "usracct.sessionid" : "12674",
>      "usracct.application" : "sshd",
>      "secevt.verdict" : "ACCEPT"
>  }
> }
> { "_id" : ObjectId("4d2235525edd07af78f648fa"), "date" : "2011-01-03 21:45:06", "facility" : "authpriv",
>  "level" : "info", "host" : "localhost", "program" : "sshd", "pid" : "12674",
>  "message" : "pam_unix(sshd:session): session opened for user algernon by (uid=0)",
>  "patterndb" : { ".classifier.class" : "unknown" }
> }
>
> As you can see, the second log entry is not recognised by patterndb,
> thus only an unknown classifier.class is logged, and nothing else.
>
> It also highlights a few problems in the patterndb I used for sshd,
> namely that it doesn't like ipv6 all that much.
>
> The changes are now pushed to my repository. I'll do a couple of
> benchmarks later tonight.
>
> --
> |8]
>
>
>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.campin.net/syslog-ng/faq.html
>
>