[syslog-ng] Recommended Data base

[Gergely Nagy]

VMI X <vmixus at gmail.com> writes:

> Currently we're logging everything to text files for a few LAN clients.
> We're considering using a database instead and have a few questions to help
> us decide:
>
>    - Would a database be a good option to replace existing text files for
>    long term storage considering storage space?
>    - Would mongo OR mysql be better suited for storing system logs?
>
> I understand answers to these questions can vary depending on specific use
> case but seeking a general recommendation to see what's typically being
> used and what the most stable/supported options would be.

As I recommended on IRC, I would suggest using syslog-ng (+ incubator)
-> ElasticSearch -> Kibana for visualisation and shorter-term (a few
months, maybe, depending on the amount of logs you have) storage. For
archival purposes, I'd use text files with rotation and compression,
alongside ES+Kibana.

I found that text files are much more efficiently compressed than
databases, so if your concern is size, then by all means, use files for
archival. Nothing stops you from using a DB alongside it for other
purposes.

Which DB? That depends on a lot of things. The DB your tools are
prepared for. If you use Kibana, that's going to be ElasticSearch. But
MongoDB has a fair amount of good tools that can help you work with your
log data. But then, so does SQL (and when it comes to SQL, I always
recommend Postgres over MySQL).

Hope that helps!

-- 
|8]