[syslog-ng] logic and duplicate suppression

[Balazs Scheidler]

On Thu, 2010-07-29 at 09:02 -0500, John Kristoff wrote:
> On Thu, 29 Jul 2010 10:03:37 +0200
> Balazs Scheidler <bazsi at balabit.hu> wrote:
> 
> > I had a similar idea for a while and as an incentive for you to try
> > the latest-greatest stuff, I've implemented it in OSE 3.2:
> 
> Thank you, thought I can't promise I'll try it in the immediate
> future.  By way of another example for the sort of thing I was thinking
> of here are a couple of patterndb rules for BIND 9.x messages:
> 
>  <ruleset id='c7a73d172adcd7c5' name='named'>
>     <pattern>named</pattern>
>     <rules>
>       <rule id='3409bd184b4dd59c' provider='drg.lamer' class='info'>
>         <description>BIND 9.x lame server - see lib/dns/resolver.c</description>
>         <patterns>
>           <pattern>lame-servers: info: @ESTRING:LAMER.REASON: resolving@ '@ESTRING:LAMER.QNAME:/@@STRING:LAMER.TYPE@/@STRING:LAMER.CLASS@': @IPvANY:LAMER.SADDR@#@NUMBER@</pattern>
>           <pattern>lame-servers: info: lame server resolving @QSTRING:LAMER.QNAME:'@ (in @QSTRING:LAMER.ZONE:'@?): @IPvANY:LAMER.SADDR@#@NUMBER@</pattern>
>         </patterns>
>         <examples>
>           <example>
>             <test_message>lame-servers: info: unexpected RCODE (REFUSED) resolving 'ns1.example.org/AAAA/IN': 192.0.2.1#53</test_message>
>             <test_message>lame-servers: info: lame server resolving '1.2.0.192.in-addr.arpa' (in '2.0.192.in-addr.arpa'?): 192.2.0.1#53</test_message>
>           </example>
>         </examples>
>       </rule>
> 
>       <rule id='2531e2f1a9304259' provider='drg.query' class='info'>
>         <description>BIND 9.x query</description>
>         <patterns>
>           <pattern>queries: info: client @IPvANY:QUERY.SADDR@#@NUMBER:QUERY.SPORT@: view @STRING:QUERY.VIEW@: query: @ESTRING:QUERY.QNAME: @@STRING:QUERY.CLASS@ @STRING:QUERY.TYPE@ @ANYSTRING:QUERY.FLAGS@</pattern>
>         </patterns>
>         <examples>
>           <example>
>             <test_message>queries: info: client 192.0.2.1#1024: view external: query: . IN NS -</test_message>
>             <test_message>queries: info: client 127.0.0.1#1024: view loopback: query: 1.0.0.127.in-addr.arpa IN PTR +</test_message>
>           </example>
>         </examples>
>       </rule>
> 
>     </rules>
>   </ruleset>
> 
> I'll happily take improvements to the above patterns.  Do note, the
> prefix strings (e.g. 'queries: info: ' and 'lame-servers: info: ') will
> only be present if BIND print-severity and print-category options are
> set.  A better set of named rules to account for various combinations of
> those options may be desirable.  Likewise, to account for the existence
> or lack of using views for queries.

you can simply duplicate the pattern rules one including the "queries:
info" part, the other that doesn't. they can even trigger the same rule.

btw, the issue with integrating these into the patterndb ruleset is that
there's no schema for this yet.

can you describe what 'generic' application level events these do
describe? For example, user login/logout are described using the
"usracct" schema, which defines which name-value pairs need to be marked
in the incoming log message. Does this idea apply to here as well?

> 
> In the case of the query pattern, being able to set a MACRO based on
> the presence of a flag (e.g. if FLAGS =~ /\+/ then RD=1 else RD=0).

I don't understand this, can you elaborate please?

> 
> > This is more difficult. The sane way of doing this is to keep state
> > on a per-host basis, which is the area of correllation. Of course
> > this is on the radar for syslog-ng, but we're not there yet.
> 
> I'm already doing as Martin suggested (thanks for your reply Martin)
> and will just log everything and optimize queries and reporting.  If
> really needed, I could post process entries in the database to keep
> only the first and last seen log of interest.  Though 

ok.

-- 
Bazsi