[syslog-ng] dbparser problem isolating messages

[Evan Rempel]

I am confused by the silence on this issue.

Should I open this as a bug, or is this the expected behaviour?

If this is the expected behaviour, how do other work around this issue?

Thanks,

On 10/31/2012 09:01 AM, Evan Rempel wrote:
> I am having a problem with the dbparser when messages have identical leading portions.
>
> In my case I have two messages
>
> xlog: backup pg_xlog/000000010000014700000076
> xlog: backup pg_xlog/000000010000014700000076 failed
>
>
> The first of these two messages is a success, and should be safely ignored.
> The second of these two messages is a failure, and I want to tag it and route it to a ticket creation program
> so that our Operations Center can investigate.
>
> Patterns are;
>
> xlog: backup pg_xlog/@SET:xid:0123456789ABCDEF@
> xlog: backup pg_xlog/@SET:xid:0123456789ABCDEF@ failed
>
> If these patterns are placed into the pattern database in this order, then the first pattern will match the failed log
> message, which is completely wrong. I would ignore the message rather than route it to my ticketing system.
>
> If I change the order of them, then all works correctly.
>
> I don't think that "hand tuning" the xml file is the correct approach, and in my case can not actually be done.
> My patterndb.xml file is created by a program that reads all of the patterns and tests messages from an external database.
>
> I've worked around this by sorting the patterns descending by the length of the patterns, but if I had the two patterns;
>
> xlog: backup pg_xlog/@SET:xid:0123456789ABCDEF@
> xlog: backup pg_xlog/@ESTRING:: @failed
>
> then that "trick" does not work either.
>
> As I see it, the dbparser should match complete messages only. That would make the order completely irrelevant.
>
>
>
>
>
>
>
> Incidentally, I have the same problem with the "program" matching that the dbparser users.
>
> Take the mimedefang program as an example. There are multiple components with program names of;
>
> mimedefang
> mimedefang-multiplexor
> mimedefang.pl
>
> but the program mimedefang will match them all. This means that the pattern database file must either;
>
> 1. Have all of the message patterns under mimedefang
> 2. have the program sections sorted in reverse order so that the shortest program matches last.
>
> Again, I think that the pattern database should match the complete program name. Using multiple patterns for the program
> name is technically sufficient, but it would be nice to use a regular expression so that I can match
>
> imapd
> imapds
>
> with the program
> impads?
>
>
> As usual, all comments welcome, especially those that explain the history of why it was done this way and
> how the issue can be addressed.
>
>
> Thanks to everyone for such an active community.
>
> Evan.
>


-- 
Evan Rempel                               erempel at uvic.ca
Senior Systems Administrator                 250.721.7691
Unix Services, University Systems, University of Victoria