[syslog-ng] Syslog-NG with MongoDB

[Ivan Adji - Krstev]

Hi all,

Thanks for the answers,
I thought that i have to create first tables and databases etc. But seem
like its just writing. Can you give me some tips on how to protect the
mongo with Usernames and Password ?

Im using Mongo 2.6.11 with Syslog-NG version 3.8.0alpha0 with
LogAnalyzer too. I have installed previous and works perfect with
MariaDB, but after some time start to eat mu CPU and RAM so i decide to
go with Mongo etc.


Kind regards
Ivan


On 04/20/2016 05:32 PM, Budai, László wrote:
> Hi,
>
> which syslog-ng do you use and which mongodb?
>
> Previous syslog-ng versions supports only the 2.x mongodb series,
> 3.2.x series is supported only by the syslog-ng upstream (version 3.8).
> We moved to the official mongo-c library and -temporary- broke the
> configuration (we have a PR[1] that will support the previous config
> format - we will merge this soon). Until we won't merge the patch, you
> can use the new config options. 
>
> So if you want to use the 3.2.x mongodb, you have to compile syslog-ng
> from source.
>
> Small tutorial:
> 1. docker: install mongodb 3.2.5
>
> https://docs.mongodb.org/master/tutorial/install-mongodb-on-ubuntu/?_ga=1.190134827.1652922526.1461162613
>
>
> 2. start mongodb
>
> root at a3e9571e39eb:/# mongod --bind_ip 172.17.0.2
>
>
> 3. build & configure
>
> git clone https://github.com/balabit/syslog-ng
> cd syslog-ng
> ./autogen.sh
> cd ..
> mkdir build
> cd build
> ../syslog-ng/configure --prefix /opt/syslog-ng.upstream
> make 
> make install
>
> [if you need help on howto compile syslog-ng, we can help]
>
> /opt/syslog-ng.upstream/etc/syslog-ng.conf:
> @version: 3.8
> @include "scl.conf"
>
>
> source s_network {
>     network(port(5555));
> };
>
> destination d_mongo {
>     mongodb (
>        uri('mongodb://172.17.0.2:27017/syslog-ng
> <http://172.17.0.2:27017/syslog-ng>')
>        collection('test')
>         value-pairs(
>             scope("selected-macros" "nv-pairs" "sdata")
>         )
>     );
> };
>
> log {
>    source(s_network);
>    destination(d_mongo);
>    flags(flow-control);
> };
>
> 4. send logs
>
> stentor at T440s:~$ logger -s MSG:$RANDOM 2>&1 | nc localhost 5555
>
> 5. query the results
> root at a3e9571e39eb:/# mongo 172.17.0.2/syslog-ng
> <http://172.17.0.2/syslog-ng>
>
> MongoDB shell version: 3.2.5
> connecting to: 172.17.0.2/syslog-ng <http://172.17.0.2/syslog-ng>
> > db.test.find({'PROGRAM':'stentor'})
> { "_id" : ObjectId("57179ae358d80574d81440e1"), "TAGS" :
> > ".source.s_network", "SOURCEIP" : "127.0.0.1", "SOURCE" : "s_network",
> > "SEQNUM" : "1", "PROGRAM" : "stentor", "PRIORITY" : "notice",
> "MESSAGE" :
> > "MSG:4854", "LEGACY_MSGHDR" : "stentor: ", "HOST_FROM" : "localhost",
> > "HOST" : "localhost", "FACILITY" : "user", "DATE" : "Apr 20 17:06:11" }
> { "_id" : ObjectId("57179ae858d80574d81440e2"), "TAGS" :
> > ".source.s_network", "SOURCEIP" : "127.0.0.1", "SOURCE" : "s_network",
> > "SEQNUM" : "2", "PROGRAM" : "stentor", "PRIORITY" : "notice",
> "MESSAGE" :
> > "MSG:31751", "LEGACY_MSGHDR" : "stentor: ", "HOST_FROM" : "localhost",
> > "HOST" : "localhost", "FACILITY" : "user", "DATE" : "Apr 20 17:06:16" }
> { "_id" : ObjectId("57179b1958d80574ff238bf1"), "TAGS" :
> > ".source.s_network", "SOURCEIP" : "127.0.0.1", "SOURCE" : "s_network",
> > "SEQNUM" : "1", "PROGRAM" : "stentor", "PRIORITY" : "notice",
> "MESSAGE" :
> > "MSG:13698", "LEGACY_MSGHDR" : "stentor: ", "HOST_FROM" : "localhost",
> > "HOST" : "localhost", "FACILITY" : "user", "DATE" : "Apr 20 17:07:05" }
> > 
>
> [1] https://github.com/balabit/syslog-ng/pull/981
>
>
> regards,
> Laszlo Budai
>
>
> On Wed, Apr 20, 2016 at 1:29 PM, Ivan Adji - Krstev
> <akivanradix at gmail.com <mailto:akivanradix at gmail.com>> wrote:
>
>     Hi all,
>     I'm trying to change the DB from InooDB to MongoDB, and i have
>     install the MongoDB but i can't configure/create the database and
>     tables on it. Can some one provide with simple tutorial or the
>     best way to create and configure the MongoDB database and tables
>     so that can be used by syslog-ng ?
>
>     Kind regards
>     Ivan
>
>     ______________________________________________________________________________
>     Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>     Documentation:
>     http://www.balabit.com/support/documentation/?product=syslog-ng
>     FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>
>
>
>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.balabit.hu/pipermail/syslog-ng/attachments/20160421/646a6361/attachment.htm