[syslog-ng] Troubles with the pattern database

[ILLES, Marton]

Hi,

See my comments inline.

On Mon, 2009-10-12 at 16:32 +0200, Guillaume Rousse wrote:
> Hello list.
> 
> I'm the mandriva maintainer for syslog-ng.
> 
> I'm trying to play with the pattern database, with syslog-ng 3.0.4. I 
> rebuild the package with the attached patch, so as to use 
> /usr/share/syslog-ng a database location (a bit more FHS-compliant than 
> /var), and extracted the files downloaded from 
> http://www.balabit.com/downloads/files/patterndb there.
> 

The reason for using /var location is that the patterndb could be
updated by the user, so I though /usr is not the appropriate place for
such files as they could be changed by the user. For pre-packaged
patterns it might be a good place on the other hand. The general idea
(which is not ready yet) is to have a bunch of pattern files (for
different applications and site specific ones) which are merged into one
big file which is loaded by syslog-ng. This generated file would be
in /var.

This merge functionality is missing from the 3.0 version and is only
available in my 3.1 tree (hopefully soon merged to mainline by Bazsi). A
new pdbtool is responsible for merging and other misc stuff. See my blog
on this:
http://marci.blogs.balabit.com/2009/08/db-parser-new-utility-pdbtool.html

I am not an FHS export so please correct me if I am wrong.

> However, loading them fails with this message:
> Error parsing pattern database file; 
> filename='/usr/share/syslog-ng/patterndb.xml', error='Unexpected <rule> 
> element'
> Error reloading pattern database, no pattern recognition will be done;
> 
> It looks like some DB format issue. According to
> http://www.balabit.com/dl/html/syslog-ng-v3.0-guide-admin-en.html/ch08s06.html#reference_parsers_pattern_databases
> 
> the supported format is v1 until syslog-ng 3.0.2, and the NEWS file 
> doesn't list any change here, while the patterndb file is already using 
> v2. Am I correct ? And in this case, is there any way to easily convert 
> the base to the old format ?

syslog-ng 3.0 only supports patterndb format version 1. syslog-ng 3.1
supoprts patterndb format 3. Format 3 is backward compatible with format
2, but not with format 1. The published patterns are in format 2, so you
can only use if the 3.1 line. The pdbtool merge command could be used -
besides merging patterns - to upgrade the patterndbs to the latest
format 3. The new format provides many new features and advantages over
the old one so it is probably a good idea to use it.

Regarding the published patterns we are working on reviewing and doing
editorial work on them to provide a better quality as the currently
published ones are generated automatically with a script from logcheck
regexp based patterns and contains some errors. I try to publish them as
soon as possible, but I am bit overloaded now. :(

M
-- 
Key fingerprint = F78C 25CA 5F88 6FAF EA21 779D 3279 9F9E 1155 670D