[syslog-ng] syslog-ng-3.38.1 causing type hint errors in patterndb - bug?
Evan Rempel
erempel at uvic.ca
Tue Nov 22 15:16:30 UTC 2022
I feel that we are able to do "whatever we want" with the syntax of the
syslog-ng configuration file and the add-contextual-data because those
are 100% syslog-ng constructs. The patterndb is an XML file which in my
opinion should attempt to stay within the spirit of the XML community as
much as possible. To that end I feel that using XML attributes is more
appropriate than placing type hints into the data values.
Using third party tools to view or even edit a patterndb file would be
more intuitive if the reader does not have to continuously re-interpret
values with type hints.
The other benefit of using XML attributes is that it can be documented
in the XML schema which document objects which can have a type and which
can not. Right now I'm really confused as to which objects can have type
hints.
Evan.
On 2022-11-22 07:03, Balazs Scheidler wrote:
>
> Hi,
>
> You are right. This was a bug in 3.38 which was fixed in PR #4158,
> which has already been merged to master, and I was testing with the
> current master and not 3.38.1
>
> Even though I did that PR, I forgot we merged a fix for this already.
>
> With the current master the same is like this:
>
> $ bin/pdbtool test --validate sample.pdb
> WARNING: the template specified in value()/<value> options for your
> grouping-by() or db-parser() configuration has been changed to support
> typing from syslog-ng 4.0. You are using an older config version and
> your template contains an unrecognized type-cast, probably a
> parenthesis in the value field. This will be interpreted in the
> `type(value)' format in future versions. Please add an explicit
> string() cast as shown in the 'fixed-value' tag of this log message or
> remove the parenthesis. The value will be processed as a 'string'
> expression; config-version='3.38', name='AUTHPROGRAM',
> value='su(pam_unix)', fixed-value='string(su(pam_unix))'
> $ echo $?
> 0
>
> The issue is that with the release of v4.0 (which we started to
> prepare for), this would become an error in earnest, as pdbtool does
> not allow you to specify the config version and uses the syslog-ng
> version number by default. And I don't see too much value in extending
> pdbtool in this direction, as having to add an explicit config version
> number to pdbtool when using it is not very intuitive.
>
> The PR I mentioned in my previous email (#4222) improves this further
> by checking the pdb version field (in the topmost <patterndb> tag) and
> uses string based values if it's below v6.
>
> What I am thinking about is whether to revert to using an attribute to
> specify the type, e.g.
>
> <values>
> <value name="AUTHPROGRAM"
> type="whatevertype-syslog-ng-supports">su(pam_unix)</value>
> </values>
>
> We would default to using a string. The only downside is that
> add-contextual-data() uses the parenthesis based format in a CSV file
> and I can't extend the CSV format as easily as XML.
>
> So by changing to the use of an attribute types would mean we have
> different syntaxes for these use-cases where one can use typed templates:
>
> - config file anywhere: `type(value)`
> - add-contextual-data, 3rd column in the CSV file: `type(value)`
> - db-parser(): `<value name="foo" type="whatever"/>`
>
>
>
>
> On Tue, Nov 22, 2022 at 3:14 PM Evan Rempel <erempel at uvic.ca> wrote:
>
> I'm not sure if you still think this is only a warning.
>
> [xerr at pangolin etc]$ rpm -q syslog-ng
> syslog-ng-3.38.1-1.el8.x86_64
> [xerr at pangolin etc]$ rpm -qi syslog-ng
> Name : syslog-ng
> Version : 3.38.1
> Release : 1.el8
> Architecture: x86_64
> Install Date: Thu 03 Nov 2022 04:08:02 AM PDT
> Group : System Environment/Daemons
> Size : 3651296
> License : GPLv2+
> Signature : RSA/SHA256, Mon 03 Oct 2022 03:00:51 AM PDT, Key ID
> c96af856c87e88fe
> Source RPM : syslog-ng-3.38.1-1.el8.src.rpm
> Build Date : Mon 03 Oct 2022 03:00:39 AM PDT
> Build Host : copr-hv-x86-64-01-prod-02441780-20221003-095224
> Relocations : (not relocatable)
> Vendor : Fedora Copr - user czanik
> URL : http://www.balabit.com/network-security/syslog-ng
> Summary : Next-generation syslog server
>
> A very small patterndb
>
> <patterndb version="4" pub_date="2009-09-01">
> <ruleset name="su(pam_unix)"
> id="RS-d2fae001-a7f6-4e39-ae62-5658906fe48c">
> <pattern>su_pam_unix_</pattern>
> <rules>
> <rule id="unmatched-program-su(pam_unix)" class="unknown"
> provider="UVic">
> <patterns>
> <pattern>@ANYSTRING@</pattern>
> </patterns>
> <values>
> <value name="AUTHPROGRAM">su(pam_unix)</value>
> </values>
> </rule>
> </rules>
> </ruleset>
> </patterndb>
>
> [xerr at pangolin etc]$ /usr/bin/pdbtool test --validate test.xml
> Error parsing pattern database file; filename='test.xml',
> error='test.xml:10:51: Error compiling value template,
> rule=unmatched-program-su(pam_unix), name=AUTHPROGRAM,
> value=su(pam_unix), error=Unknown type specified in type hinting: su'
>
> Evan.
>
> On 2022-11-22 01:39, Balazs Scheidler wrote:
>>
>> I only got warnings and not errors, when trying to use this in my
>> XML file:
>>
>> <value name="seq-matches">su(pam_unix)</value>
>>
>> how did you execute syslog-ng so you got the error above? The
>> only case where this is an error, if you are using "@version: 4.0"
>>
>> This was the warning produced (both by syslog-ng and pdbtool):
>>
>> WARNING: the template specified in value()/<value> options for
>> your grouping-by() or db-parser() configuration has been changed
>> to support typing from syslog-ng 4.0. You are using an older
>> config version and your template contains an unrecognized
>> type-cast, probably a parenthesis in the value field. This will
>> be interpreted in the `type(value)' format in future versions.
>> Please add an explicit string() cast as shown in the
>> 'fixed-value' tag of this log message or remove the parenthesis.
>> The value will be processed as a 'string' expression;
>> config-version='3.38', name='seq-matches', value='su(pam_unix)',
>> fixed-value='string(su(pam_unix))'
>>
>> I chose to embed the type-hint field in the body of the <value>
>> tag, as this is the format that is used everywhere else. I might
>> be able to bump the db-parser XML file version, in the file header:
>>
>> <patterndb version='5'>
>>
>> I could bump this up to version 6, in which case you'd only need
>> to add the type-hint if you also bumped the version number. That
>> I think is doable.
>>
>> On the documentation front, there's an XML schema in the source
>> tree under the doc/xsd/ directory for each version of patterndb,
>> and I assume the documentation also has a chapter on the
>> db-parser() format.
>>
>> On Mon, Nov 21, 2022 at 5:22 PM Evan Rempel <erempel at uvic.ca> wrote:
>>
>> We have a patterndb file that contains both patterns and
>> values with
>> fixed text that include a strings of the form "xxxx (yyy)"
>> which now can
>> not be loaded by the patterndb.
>>
>> Syslog-ng throws the error
>>
>> Error parsing pattern database file; ... Error compiling
>> value template,
>> rule=FLARE-3543, name=AUTHPROGRAM, value=su(pam_unix),
>> error=Unknown
>> type specified in type hinting: su'
>>
>> I have not enabled the version 4.0 testing of type hinting.
>> The version
>> configuration is
>>
>> @version: 3.36
>>
>> I am unable to find documentation for the full syntax of the
>> patterndb
>> file (seems to have been lost from the docs since perhaps 3.16?)
>>
>> Is there a patterndb syntax specification document?
>>
>> I would expect that even when type hinting is enabled in the
>> patterndb
>> file, given that it is an XML document, that the type hinting
>> would be
>> part of the XML tag metadata, rather than part of the XML
>> static data.
>>
>> --
>> Evan
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20221122/76fc275b/attachment.htm>
More information about the syslog-ng
mailing list