[syslog-ng] pdbtool 'patternize'

[Péter Gyöngyösi]

Hi,

On Wed, Apr 16, 2014 at 6:15 PM, David Hauck <davidh at netacquire.com> wrote:

> Another couple questions regarding 'patternize'.
>
> Why does the 'patternize' output not include additionally relevant parts
> of the schema? In particular the 'program pattern' is not output as part of
> the result? It's my understanding that this is key matching criteria when
> determining matches and I'm unsure what would happen with the pattern db
> that contains rulesets with no program pattern specifiers (note: the
> documentation does talk about the matching behaviour when ${PROGRAM} is
> empty, but this is different - i.e., I assume rules with empty program
> patterns don't get matched/looked at when ${PROGRAM} is non-empty).
>

That's because the clustering algorithm used within patternize itself does
not take the program field into account, so including that in the pattern
database would create erroneous results. It wouldn't be that difficult to
update the algorithm to use the program field and only group logs together
if they have the same value there but I won't have time to get to it in the
upcoming weeks. It's a low hanging fruit if you are willing to code, I am
happy to help if you get stuck :)

If the {$PROGRAM} is non-empty but there's no "program" entry defined in
the pattern, the message does get matched, although I am pretty sure that
the patterns where the "program" entry is specified are stronger, but I am
not 100% about that priority order. Actually, that's what happens if you
run "pdbtool test" on an XML generated by patternize: as you can see it
contains examples in which the program field is set to the bogus
"patternize" value manually, and the patterns match those examples
nevertheless. Probably the documentation should be updated to describe that
scenario, too.



>
> Also, where is the actual schema (the xsd file) that defines the pattern
> db format (and the semantics of each element)? I've found the admin guide
> documentation lacking in terms of explicit description of the patter db
> format (the brief section that attempts to describe this is very thin).
>

Well, a human-readable description can indeed never be as precise as a
formal definition :) I don't know how the version you are using is
packaged, but in the source tree these XSDs are in "/doc/xsd":
https://github.com/balabit/syslog-ng/tree/master/doc/xsd These are pretty
well annotated XSDs which should be quite self-explaining when it comes to
the semantics, too.

greets,
Peter
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.balabit.hu/pipermail/syslog-ng/attachments/20140416/1dcdbe14/attachment-0001.htm