[syslog-ng] [patterndb] classification

[Balazs Scheidler]

On Fri, 2010-09-03 at 13:25 -0700, Anton Chuvakin wrote:
> > However quickly browsing through the PDF I couldn't find the taxonomy
> > portion, is this "almost ready" stuff available somewhere?
> 
> Not public yet, but will be very soon. Let me see what I can send over
> at this stage. The main idea for CEE taxonomy is "OAS" for
> object/action/status "tags" being mandatory for each message. We found
> this to be both more useful and more doable than a single class for
> the message. Essentially, you should be able unambiguously determine
> what every log message in the world (!) means by reading the OAS
> triad.
> 
> >> Tags can be organized in 'bunches' that serve as classes.
> > You mean, every tag would belong to a bunch and a given message could
> > only be part of a single bunch?
> 
> No, it will be many-to-many where a message can carry many tags, but
> it can be filtered both by tags and bunches. Bunch of tags is simply a
> "next level tag" like:
> 
> message 1 linux user login  failed
> tagged: authentication, user, failure, PCI DSS compliance
> 
> authentication tag is part of "AAA bunch", "Action" bunches
> PCI DSS compliance tag is part of "Regulations" bunch
> failure is part of "status"
> 
> In CEE, OAS triad will likely be used as "default tags" for all messages.

Is it a recursive hierarchy? e.g. is it possible to organize bunches to
even higher level bunches?

Also what I see unsolved is how the user can easily sort messages into
files/tables by bunch.

E.g. something like:

destination d_files_by_bunch { file("/var/log/messages.$bunch"); };

Although if I were to define multi-value name-value pairs the one above
could expand to multiple file writes. This way writing by tags or by
bunches should be very simple.

Interesting idea...

> 
> >> > "importance", in a similar spirit to syslog severity, but one that works
> >> > even if the application developer uses a bogus severity when sending
> >> > syslog messages.
> >>
> >> Important is HUGE challenge. Now sure what to add to this one as it is
> >> largely an unsolved problem due to very different contexts for message
> >> analysis. Even mere 'connection established' can be 10 of 10 for
> >> somebody in some circumstances. One can try to glue important to tags
> >> (like exploit > connection) and not to individual messages, it might
> >> work sometimes.
> >
> > Hmm... good idea.
> 
> Maybe.. this issue took about 3 years of discussion among CEE team -
> and there is still no resolution to "universal syslog/log message
> severity scoring"
> 
> Let me know how else I can help.

Yeah, but using tags/bunches one can define which is more important to
her.

-- 
Bazsi