[syslog-ng] filtering messages by context

[Balazs Scheidler]

On Wed, 2010-01-20 at 11:25 +0100, Guillaume Rousse wrote:
> Hello list.
> 
> We use hardware-based SLB for our critical services (ldap, radius, dns, 
> ...). It means we have probes testing our services every X seconds, 
> polluting our logs. It's quite easy to catch those traces when they 
> consist of a single message, as for our kerberos servers
> 
> Jan 20 11:17:02 avron1 kdc[477]: connection closed before end of data 
> after 0 bytes from IPv4:10.202.11.254 to tcp/0
> Jan 20 11:17:04 avron1 kdc[477]: connection closed before end of data 
> after 0 bytes from IPv4:10.202.11.254 to tcp/0
> Jan 20 11:17:06 avron1 kdc[477]: connection closed before end of data 
> after 0 bytes from IPv4:10.202.11.254 to tcp/0
> 
> This simple setup is enough to drop them:
> destination d_drop {
> }
> 
> filter f_slb_kerberos_probe {
>      message("connection closed before end of data");
> };
> 
> log {
>      source(s_sys);
>      filter(f_kerberos);
>      filter(f_slb_kerberos_probe);
>      destination(d_drop);
>      flags(final);
> };
> 
> However, when those traces actually consist of two messages, as in our 
> LDAP servers, it's a bit more difficult:
> 
> Jan 20 11:23:04 avron1 slapd[13802]: conn=68089 fd=51 ACCEPT from 
> IP=10.202.11.254:39428 (IP=10.202.11.8:389)
> Jan 20 11:23:04 avron1 slapd[13802]: conn=68089 fd=51 closed (connection 
> lost)
>   Jan 20 11:23:06 avron1 slapd[13802]: conn=68090 fd=51 ACCEPT from 
> IP=10.202.11.254:39434 (IP=10.202.11.8:389)
> Jan 20 11:23:06 avron1 slapd[13802]: conn=68090 fd=51 closed (connection 
> lost)
> 
> I can filter out the first message, using the probe IP adress as 
> criteria, but not the second one:
> 
> filter f_slb_ldap_probe {
>      message("ACCEPT from IP=10.202.11.254");
> };
> 
> log {
>      source(s_sys);
>      filter(f_ldap);
>      filter(f_slb_ldap_probe);
>      destination(d_drop);
>      flags(final);
> };
> 
> is there any way to catch the connection id in the first message 
> (68090), so as to filter out any following one refering to the same 
> connection ?

Well, syslog-ng doesn't have an builtin correllation engine, so as of
now you can't filter those out, at least not without programming.


-- 
Bazsi