[syslog-ng] Using pipes for prefiltering?

10 Nov 2009

      Hello,

Recently I've been trying to squeeze out the most of syslog-ng 3.0 and one of 
problems pestering me is this: there are some messages, which I would like 
never to appear in any 
log file, so I created filters for them, directed real sources through them to 
pipe destination and created prefiltered sources pointing to those same 
pipes. Well and what 
happens is - some logs reach the final destinations, but some not. For 
example, nothing at all now gets logged form kernel through pipe, even if 
filters get thrown out!

Its on Linux... Also I had to disable syslogng logs, because otherwise CPU was 
loaded 100% sometimes (one core).

After all, how should I do prefiltering?
Strange thing is, that even with pipes first of these works, while the other 
doesn't:

log { source(s_compname_usp_f); source(s_compname_kern_f); 
destination(d_compname_vt12); };
log { source(s_compname_kern_f); filter(f__kern); 
destination(d_compname_kern); };

filter f__kern { facility(0); };

You can see pipes fragment below.
Hope someone has suggestion...
Best, 
Kārlis

# prefiltering
#
destination d_compname_usp_f { pipe("/var/log/filtered-source"); };
destination d_compname_kern_f { pipe("/var/log/filtered-kernel-source"); };

# abusers/spammers dropping here
#
filter f_compname_antispam_usp {
                  ( not
                    (
                      program(upsd)
                      and
                      message("some shit there")
                    )
                  )
                  and
                  ( not
                    program("syslog")
                  )
                  and
                  ( not
                    (
                      program("cron")
                      and
                      (
                        message(".*rm \-f /var/spool/cron/lastrun/cron\..*")
                        or
                        message(".*test \-x /usr/sbin/run-crons 
\&\& /usr/sbin/run\-crons.*")
                      )
                    )
                  )
                  and
                  ( not
                    (
                      program("acpid")
                      and
                      (
                        message(".*client connected from.*")
                        or
                        message(".*1 client rule loaded")
                        or
                        message(".*client .* has disconnected")
                      )
                    )
                  )
;
};

filter f_compname_antispam_kern {
                  ( not message(".*ata1: exception .*") )
                  and
                  ( not message(".*ata1: .*connection status changed") )
                  and
                  ( not message(".*Hangcheck\: hangcheck value past 
margin\!") )
;
};

log { source(s_compname_usp); filter(f_compname_antispam_usp); 
destination(d_compname_usp_f); };
log { source(s_compname_kern); destination(d_compname_kern_f); };

source s_compname_usp_f { pipe("/var/log/filtered-source"); };
source s_compname_kern_f { pipe("/var/log/filtered-kernel-source"); };

[syslog-ng] Using pipes for prefiltering?

Kārlis Repsons