[syslog-ng] syslog-ng with sqlite 100% cpu

[Balazs Scheidler]

The problem is that older syslog-ng versions didn't have transaction
support, so even if you set flush-lines() higher, it may not be starting
transactions.

On Tue, Feb 14, 2017 at 11:57 PM, work ran <work.greeran at gmail.com> wrote:

> thanks for the response
> For now our architecture is limiting us to an old syslog-ng version so we
> cannot upgrade.
> as for the flush we are running it with 200 and with timeout 200 also. we
> are trying to increase the flush-lines to 1000 and see if it helps.
> i also googled the problem and saw a couple of mails that said that there
> was a bug with syslog-ng 3.2.5/4 and pcre that created a bug with the cpu
> usage. does someone if it might be the problem and how to fix it.
>
> thanks
> Ran
>
>
> On Tue, Feb 14, 2017 at 5:53 PM, Scheidler, Balázs <
> balazs.scheidler at balabit.com> wrote:
>
>> It might be because of transaction size. Sqlite is very inefficient if
>> the transaction size is small.
>>
>> Newer syslog-ng versions support batching, set flush-lines to something
>> like 100 and it would improve a lot.
>>
>> On Feb 14, 2017 06:31, "Fekete, Róbert" <robert.fekete at balabit.com>
>> wrote:
>>
>>> Hi,
>>>
>>> It is not normal, it might be a bug. However, version 3.2 is ancient,
>>> I'd recommend upgrading to a newer version. You can find release tarballs
>>> on github, or links to binaries here: https://syslog-ng.org/3r
>>> d-party-binaries/
>>>
>>> Regards,
>>>
>>> Robert
>>>
>>> On Tue, Feb 14, 2017 at 3:19 PM, work ran <work.greeran at gmail.com>
>>> wrote:
>>>
>>>> hi
>>>> i am running syslog-ng with sqlite
>>>> i am sending 50 logs per second and i am getting 80% cpu usage.
>>>> could someone tell me how i can fix this or is it normal
>>>> thanks
>>>> Ran
>>>>
>>>>
>>>>
>>>> syslog-ng 3.2.5
>>>> Installer-Version: 3.2.5
>>>> Revision: ssh+git://bazsi@git.balabit//var/scm/git/syslog-ng/syslog-ng
>>>> -ose--mainline--3.2#master#9d4bea28198bd731df1a61e980a2af5b88d81116
>>>> Compile-Date: Jul 25 2014 15:20:50
>>>> Enable-Threads: on
>>>> Enable-Debug: off
>>>> Enable-GProf: off
>>>> Enable-Memtrace: off
>>>> Enable-Sun-STREAMS: off
>>>> Enable-IPv6: on
>>>> Enable-Spoof-Source: on
>>>> Enable-TCP-Wrapper: on
>>>> Enable-SSL: off
>>>> Enable-SQL: on
>>>> Enable-Linux-Caps: off
>>>> Enable-Pcre: on
>>>> Enable-Pacct: off
>>>>
>>>>
>>>>
>>>> ____________________________________________________________
>>>> __________________
>>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>>> Documentation: http://www.balabit.com/support
>>>> /documentation/?product=syslog-ng
>>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>>>
>>>>
>>>>
>>>
>>> ____________________________________________________________
>>> __________________
>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>> Documentation: http://www.balabit.com/support
>>> /documentation/?product=syslog-ng
>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>>
>>>
>>>
>> ____________________________________________________________
>> __________________
>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>> Documentation: http://www.balabit.com/support/documentation/?product=
>> syslog-ng
>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>
>>
>>
>
> ____________________________________________________________
> __________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?
> product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>
>


-- 
Bazsi
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20170216/df99ecf5/attachment.html>