<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
Hmmm - I have done some testing but not settled on the perfect
solution (yet).<br>
<br>
- I have multi-line logs being parsed by a program destination and
then writing to mongodb (perl parser)<br>
- I have started testing straight mongo, but getting the right data
in the right fields seems important<br>
- I have done very basic testing with syslog-ng --> redis which I
am planning on then --> elasticsearch <br>
(I may need to stick Logstash in between redis and elasticsearch)<br>
<br>
In general, any relational database (SQL) wins when you can
represent the relationships in the schema.<br>
<br>
"nosql" like mongo typically wins with unstructured data, but at a
space penalty (needing to store json format)<br>
<br>
If I had to pick one *right now* I would probably use syslog-ng
--> redis --> logstash --> elasticsearch --> kibana<br>
The (R) ELK stack has a lot of support and development, and is
pretty close to a free splunk.<br>
<br>
Although I can see using a sharded/replicated mongodb having some
basic advantages, but I have not (yet) found the perfect way to do
ad-hoc queries against the store.<br>
<br>
Good luck (and report back!)<br>
<br>
Thanks,<br>
Jim<br>
<br>
<br>
<div class="moz-cite-prefix">On 08/18/2014 05:06 PM, VMI X wrote:<br>
</div>
<blockquote
cite="mid:CAA1YcJFLi-HWrkW0gQHNx8aVhmaPywqX3yiHsxLcvsem+A315Q@mail.gmail.com"
type="cite">
<div dir="ltr">Hi,
<div>Currently we're logging everything to text files for a few
LAN clients.</div>
<div>We're considering using a database instead and have a few
questions to help us decide:</div>
<div>
<ul>
<li>Would a database be a good option to replace existing
text files for long term storage considering storage
space?</li>
<li>Would mongo OR mysql be better suited for storing system
logs? </li>
</ul>
I understand answers to these questions can vary depending on
specific use case but seeking a general recommendation to see
what's typically being used and what the most stable/supported
options would be.<br>
</div>
<div><br clear="all">
<div><br>
</div>
-- <br>
<i><font face="tahoma, sans-serif">Nullius In Verba</font></i>
</div>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">______________________________________________________________________________
Member info: <a class="moz-txt-link-freetext" href="https://lists.balabit.hu/mailman/listinfo/syslog-ng">https://lists.balabit.hu/mailman/listinfo/syslog-ng</a>
Documentation: <a class="moz-txt-link-freetext" href="http://www.balabit.com/support/documentation/?product=syslog-ng">http://www.balabit.com/support/documentation/?product=syslog-ng</a>
FAQ: <a class="moz-txt-link-freetext" href="http://www.balabit.com/wiki/syslog-ng-faq">http://www.balabit.com/wiki/syslog-ng-faq</a>
</pre>
</blockquote>
<br>
</body>
</html>