[syslog-ng]FYI: Performance

[Balazs Scheidler]

On Wed, 2004-12-01 at 02:50, Timothy Webster wrote:
> Currently logging 39G a day and just keeping up, with
> minirsyslog. The problem is the disk io is killing me. :(
> 
> each byte requires 1 write then 1 read to process and 1 read to
> compress and finally 1/3 write 
> 1 + 1/3 write and 2 reads
> 
> I am using lzop compression, gzip just won't keep up.
> And I avoid perl like the plague.
> I have one second stage data analyzer script in perl and it
> takes a good chunk of the resources.
> 
> Thinking of adding lzo compression directly in to syslog-ng and with
> a fifo I can reduce my disk io to ~ 1/3 compressed byte per raw byte. 
> 
> Does syslog-ng have a hope here performance wise?

Sigh. yes and no. I have a nearly complete reimplementation of syslog-ng
with nifty features and probably better performance and the possibility
to add more features easier. It's been sitting on my hard disk more or
less unmodified for about two years now.

-- 
Bazsi