[syslog-ng]Filter enhancement

[Gregor Binder]

Jay Guerette on Thu, Dec 06, 2001 at 08:32:33AM -0500:

Jay,

> The only problem with this implementation is the double hit for every
> message.I have 53 hosts that generated 2.5 million log entries yesterday; so having
> a 'loopback' filter could theoretically throw up to 5 million at
> syslog-ng. I don'twant to overload the system. Also, when an 'event' happens, and a cluster of
> boxes all start logging furiously for a few seconds, I think that the
> burst * 2will temporarily overwhelm it.

are you sure? Actually, one would have to look at the source. But the
way I see it, there can't be MUCH difference given the configuration is
optimized, since reading from a FIFO is not much different than reading
program output performancewise. Obviously the message re-enters syslog-
ng message processing, but I'd guess this path is short if the log
statements are not too exciting (at least not two times the processing).

I believe context switching between possibly multiple filter processes
with this sort of relatively slow IPC will already cost you so much
that the additional overhead produced by this missing feature is
neglectible. I admit this is just a feeling :)

To really save cycles, message rewriting should be inline, I think there
even might be something like this in the development release? I don't
know exactly, I believe there was some discussion about this, but I only
use production releases. Somebody else?

As another alternative: Have you thought about passing the intended
filename as an argument to your program() destination and write directly
from the filter program? That should be just about the same thing as the
inline filtering. Obviously you'd have to implement sync() et al 
yourself.

Regards,

-- 
 ____ ____ 
/  _/| -  >  Gregor Binder <gb@(rootnexus.net|sysfive.com)>
| / || _\ \
\__ Id: 0xE2F31C4B Fp: 8B8A 5CE3 B79B FBF1 5518 8871 0EFB AFA3 E2F3 1C4B