[syslog-ng] Syslog-NG with MongoDB

[Budai, László]

Hi,

which syslog-ng do you use and which mongodb?

Previous syslog-ng versions supports only the 2.x mongodb series, 3.2.x
series is supported only by the syslog-ng upstream (version 3.8).
We moved to the official mongo-c library and -temporary- broke the
configuration (we have a PR[1] that will support the previous config format
- we will merge this soon). Until we won't merge the patch, you can use the
new config options.

So if you want to use the 3.2.x mongodb, you have to compile syslog-ng from
source.

Small tutorial:
1. docker: install mongodb 3.2.5

https://docs.mongodb.org/master/tutorial/install-mongodb-on-ubuntu/?_ga=1.190134827.1652922526.1461162613


2. start mongodb

root at a3e9571e39eb:/# mongod --bind_ip 172.17.0.2


3. build & configure

git clone https://github.com/balabit/syslog-ng
cd syslog-ng
./autogen.sh
cd ..
mkdir build
cd build
../syslog-ng/configure --prefix /opt/syslog-ng.upstream
make
make install

[if you need help on howto compile syslog-ng, we can help]

/opt/syslog-ng.upstream/etc/syslog-ng.conf:
@version: 3.8
@include "scl.conf"


source s_network {
    network(port(5555));
};

destination d_mongo {
    mongodb (
       uri('mongodb://172.17.0.2:27017/syslog-ng')
       collection('test')
        value-pairs(
            scope("selected-macros" "nv-pairs" "sdata")
        )
    );
};

log {
   source(s_network);
   destination(d_mongo);
   flags(flow-control);
};

4. send logs

stentor at T440s:~$ logger -s MSG:$RANDOM 2>&1 | nc localhost 5555

5. query the results
root at a3e9571e39eb:/# mongo 172.17.0.2/syslog-ng

MongoDB shell version: 3.2.5
connecting to: 172.17.0.2/syslog-ng
> db.test.find({'PROGRAM':'stentor'})
{ "_id" : ObjectId("57179ae358d80574d81440e1"), "TAGS" :
> ".source.s_network", "SOURCEIP" : "127.0.0.1", "SOURCE" : "s_network",
> "SEQNUM" : "1", "PROGRAM" : "stentor", "PRIORITY" : "notice", "MESSAGE" :
> "MSG:4854", "LEGACY_MSGHDR" : "stentor: ", "HOST_FROM" : "localhost",
> "HOST" : "localhost", "FACILITY" : "user", "DATE" : "Apr 20 17:06:11" }
{ "_id" : ObjectId("57179ae858d80574d81440e2"), "TAGS" :
> ".source.s_network", "SOURCEIP" : "127.0.0.1", "SOURCE" : "s_network",
> "SEQNUM" : "2", "PROGRAM" : "stentor", "PRIORITY" : "notice", "MESSAGE" :
> "MSG:31751", "LEGACY_MSGHDR" : "stentor: ", "HOST_FROM" : "localhost",
> "HOST" : "localhost", "FACILITY" : "user", "DATE" : "Apr 20 17:06:16" }
{ "_id" : ObjectId("57179b1958d80574ff238bf1"), "TAGS" :
> ".source.s_network", "SOURCEIP" : "127.0.0.1", "SOURCE" : "s_network",
> "SEQNUM" : "1", "PROGRAM" : "stentor", "PRIORITY" : "notice", "MESSAGE" :
> "MSG:13698", "LEGACY_MSGHDR" : "stentor: ", "HOST_FROM" : "localhost",
> "HOST" : "localhost", "FACILITY" : "user", "DATE" : "Apr 20 17:07:05" }
>

[1] https://github.com/balabit/syslog-ng/pull/981


regards,
Laszlo Budai


On Wed, Apr 20, 2016 at 1:29 PM, Ivan Adji - Krstev <akivanradix at gmail.com>
wrote:

> Hi all,
> I'm trying to change the DB from InooDB to MongoDB, and i have install the
> MongoDB but i can't configure/create the database and tables on it. Can
> some one provide with simple tutorial or the best way to create and
> configure the MongoDB database and tables so that can be used by syslog-ng ?
>
> Kind regards
> Ivan
>
>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation:
> http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.balabit.hu/pipermail/syslog-ng/attachments/20160420/93b6982e/attachment-0001.htm