Jay--- Yesterday, our email log server here did 47069024518 bytes or 176818253 lines a day. Balazs -- Unfortunately we can't run non-production stuff to test out that secret code you have there, but would like to see that stuff make it to the stable series in the next year... =) I'm sure people wouldn't mind testing... ? In the lab, at least I can... Which reminds me.. I still need to get some more namedpipe logging information, which seems to get lost somewhere.... - Dave On Wed, 01 Dec 2004 09:24:23 +0100, Balazs Scheidler <bazsi@balabit.hu> wrote:
On Wed, 2004-12-01 at 02:50, Timothy Webster wrote:
Currently logging 39G a day and just keeping up, with minirsyslog. The problem is the disk io is killing me. :(
each byte requires 1 write then 1 read to process and 1 read to compress and finally 1/3 write 1 + 1/3 write and 2 reads
I am using lzop compression, gzip just won't keep up. And I avoid perl like the plague. I have one second stage data analyzer script in perl and it takes a good chunk of the resources.
Thinking of adding lzo compression directly in to syslog-ng and with a fifo I can reduce my disk io to ~ 1/3 compressed byte per raw byte.
Does syslog-ng have a hope here performance wise?
Sigh. yes and no. I have a nearly complete reimplementation of syslog-ng with nifty features and probably better performance and the possibility to add more features easier. It's been sitting on my hard disk more or less unmodified for about two years now.
-- Bazsi
_______________________________________________ syslog-ng maillist - syslog-ng@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html