Mysql is missing log on my Ubuntu 18.04.4LTS
Hi I have installed syslog-ng on a ubuntu machine. Mysql has been used to store the logs. But i keep seeing the logs are missed. that means logs are not storing properly. any idea what is wrong with it. it normally happened at the end of the day close to 12am. Kindly help. Thanks
Hi, mysql (or any other traditional SQL database for that matter) is not an ideal storage for logs, their performance range is usually much lower to what a log infrastructure can collect. Assuming mysql is the botlleneck in your installation, you can tell syslog-ng to slow down incoming logs to the pace of mysql using the flags(flow-control) in your log {} statement, however that will mean that everything will slow down to what mysql can give you. In any other case, you will need to improve mysql performance (larger box, better disks, tuning parameters etc). Or you start using something else for logs, ElasticSearch is much better suited for this use-case. Splunk is better, but is quite expensive. One Identity (formerly Balabit) has SSB that can cope with a lot of logs and can be a good step forward, but that's proprietary too. Cheers, Bazsi On Sun, Mar 22, 2020 at 8:34 PM Sync IT <syncit-bd@live.com> wrote:
Hi I have installed syslog-ng on a ubuntu machine. Mysql has been used to store the logs. But i keep seeing the logs are missed. that means logs are not storing properly. any idea what is wrong with it. it normally happened at the end of the day close to 12am. Kindly help. Thanks
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
-- Bazsi
Hi, Forget any SQL to store logs, it is a bad juju as Bazsi mentioned. It will be painfully slow and far from ideal. Move to ElasticSearch or even Graylog (both free with paid options), you'll be more happier :) My recent personal favorite is Graylog, which can scale both horizontally, vertically and financially :) https://www.graylog.org/downloads#open-source L: On Sun, Mar 22, 2020 at 8:34 PM Sync IT <syncit-bd@live.com> wrote:
Hi I have installed syslog-ng on a ubuntu machine. Mysql has been used to store the logs. But i keep seeing the logs are missed. that means logs are not storing properly. any idea what is wrong with it. it normally happened at the end of the day close to 12am. Kindly help. Thanks
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
participants (3)
-
Balazs Scheidler
-
Pal, Laszlo
-
Sync IT