VMI X <vmixus@gmail.com> writes:
Currently we're logging everything to text files for a few LAN clients. We're considering using a database instead and have a few questions to help us decide:
- Would a database be a good option to replace existing text files for long term storage considering storage space? - Would mongo OR mysql be better suited for storing system logs?
I understand answers to these questions can vary depending on specific use case but seeking a general recommendation to see what's typically being used and what the most stable/supported options would be.
As I recommended on IRC, I would suggest using syslog-ng (+ incubator) -> ElasticSearch -> Kibana for visualisation and shorter-term (a few months, maybe, depending on the amount of logs you have) storage. For archival purposes, I'd use text files with rotation and compression, alongside ES+Kibana. I found that text files are much more efficiently compressed than databases, so if your concern is size, then by all means, use files for archival. Nothing stops you from using a DB alongside it for other purposes. Which DB? That depends on a lot of things. The DB your tools are prepared for. If you use Kibana, that's going to be ElasticSearch. But MongoDB has a fair amount of good tools that can help you work with your log data. But then, so does SQL (and when it comes to SQL, I always recommend Postgres over MySQL). Hope that helps! -- |8]