[syslog-ng] Mysql is missing log on my Ubuntu 18.04.4LTS

[Balazs Scheidler]

Hi,

mysql (or any other traditional SQL database for that matter) is not an
ideal storage for logs, their performance range is usually much lower to
what a log infrastructure can collect.

Assuming mysql is the botlleneck in your installation, you can tell
syslog-ng to slow down incoming logs to the pace of mysql using the
flags(flow-control) in your log {} statement, however that will mean that
everything will slow down to what mysql can give you.

In any other case, you will need to improve mysql performance (larger box,
better disks, tuning parameters etc).

Or you start using something else for logs, ElasticSearch is much better
suited for this use-case. Splunk is better, but is quite expensive. One
Identity (formerly Balabit) has SSB that can cope with a lot of logs and
can be a good step forward, but that's proprietary too.

Cheers,
Bazsi


On Sun, Mar 22, 2020 at 8:34 PM Sync IT <syncit-bd at live.com> wrote:

> Hi
> I have installed syslog-ng on a ubuntu machine. Mysql has been used to
> store the logs. But i keep seeing the logs are missed. that means logs are
> not storing properly. any idea what is wrong with it. it normally happened
> at the end of the day close to 12am. Kindly help. Thanks
>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation:
> http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>

-- 
Bazsi
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20200323/ba37cc0d/attachment.html>