[syslog-ng]FYI: Performance

[Dave Johnson]

Jay---

 Yesterday, our email log server here did 47069024518  bytes or 
176818253 lines a day.


Balazs -- Unfortunately we can't run non-production stuff to test out
              that secret code you have there, but would like to see
that stuff make it
              to the stable series in the next year... =)

I'm sure people wouldn't mind testing... ?  In the lab, at least I
can...  Which reminds me..  I  still need to get some more namedpipe
logging information, which seems to get lost somewhere....

- Dave


On Wed, 01 Dec 2004 09:24:23 +0100, Balazs Scheidler <bazsi@balabit.hu> wrote:
> On Wed, 2004-12-01 at 02:50, Timothy Webster wrote:
> 
> 
> > Currently logging 39G a day and just keeping up, with
> > minirsyslog. The problem is the disk io is killing me. :(
> >
> > each byte requires 1 write then 1 read to process and 1 read to
> > compress and finally 1/3 write
> > 1 + 1/3 write and 2 reads
> >
> > I am using lzop compression, gzip just won't keep up.
> > And I avoid perl like the plague.
> > I have one second stage data analyzer script in perl and it
> > takes a good chunk of the resources.
> >
> > Thinking of adding lzo compression directly in to syslog-ng and with
> > a fifo I can reduce my disk io to ~ 1/3 compressed byte per raw byte.
> >
> > Does syslog-ng have a hope here performance wise?
> 
> Sigh. yes and no. I have a nearly complete reimplementation of syslog-ng
> with nifty features and probably better performance and the possibility
> to add more features easier. It's been sitting on my hard disk more or
> less unmodified for about two years now.
> 
> --
> Bazsi
> 
> 
> 
> 
> _______________________________________________
> syslog-ng maillist  -  syslog-ng@lists.balabit.hu
> https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Frequently asked questions at http://www.campin.net/syslog-ng/faq.html
> 
>