[syslog-ng] improving SQLite writing performances (WAS: Re: inserts into a sqlite3 database are not delayed)

[Davide Alberani]

On Wed, Oct 15, 2014 at 10:56 AM, Balazs Scheidler <bazsi77 at gmail.com> wrote:
>
> It immediately runs the query, flush_lines() controls the transaction size,
> which can be enabled using
> flags(explicit-commits).

Ok, I see and I can confirm noticeable improvements on a system
under heavy I/O and CPU load using explicit-commits (or better: I see
it gets a lot worse disabling it, since I already had it on ;))

Plus, I'm seeing minor improvements with session_statements set
to "PRAGMA synchronous=OFF; PRAGMA count_changes=OFF;"

Beside this, has anyone found other ways to improve writing performances
using syslog-ng with sqlite?

On (physical) systems with high load, I've noticed that the system is much more
responsive if the elevator is set to noop or deadline instead of cfq.
But I guess this is highly dependent on the specific circumstances of
the system.


Thanks!

-- 
Davide Alberani <davide.alberani at gmail.com>  [PGP KeyID: 0x465BFD47]
http://www.mimante.net/