Wouldn't the log_fetch_limit set to a large number allow for 1 second follow to process huge amounts of data. You would have to set the log_fifo_size large enough to host this number as well, but I don't see the requirement for the millisecond polling. Evan. Joe Shaw wrote:
Hi,
On Mon, Aug 25, 2008 at 3:07 AM, Balazs Scheidler <bazsi@balabit.hu> wrote:
I'm not sure about the patch though, is millisecond polling really needed? It might increase the load on the host significantly to check files several times a second.
The reason this patch was born was solved in an unrelated manner (e.g. increasing FIFO size).
The reasoning behind the patch really isn't increasing the FIFO size (although that is why I did it initially) but because when you are pushing large amounts of data, a one second delay can cause the data to take hours to get from the source to the destination. This is what I was referring to when I was talking about my lack of patience. :)
It's true that decreasing the interval increases the load -- and I think anyone who decreased it to, say, 10ms would see that -- but it's another area in which people can make that trade-off and optimize their deployment. Dropping it to 500ms doubles the throughput but doesn't add much load. I was running at 200ms rather happily.
If this turns out really useful, I'd not add a separate keyword, but would permit the use of floating point numbers, e.g. follow_freq(0.1) instead of follow_freq_ms(100)
Makes a lot of sense.
Joe ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html