<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<div class="moz-cite-prefix">I feel that we are able to do "whatever
we want" with the syntax of the syslog-ng configuration file and
the add-contextual-data because those are 100% syslog-ng
constructs. The patterndb is an XML file which in my opinion
should attempt to stay within the spirit of the XML community as
much as possible. To that end I feel that using XML attributes is
more appropriate than placing type hints into the data values.</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">Using third party tools to view or even
edit a patterndb file would be more intuitive if the reader does
not have to continuously re-interpret values with type hints.<br>
</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">The other benefit of using XML
attributes is that it can be documented in the XML schema which
document objects which can have a type and which can not. Right
now I'm really confused as to which objects can have type hints.</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">Evan.<br>
</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">On 2022-11-22 07:03, Balazs Scheidler
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CAKcfE+YLFv3kjY2nr1d=mnbbQhT83iC9+aGJRZaS05-4M+ZjqQ@mail.gmail.com">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<br>
<div>
<div dir="ltr">
<div>Hi,</div>
<div><br>
</div>
<div>You are right. This was a bug in 3.38 which was fixed in
PR #4158, which has already been merged to master, and I was
testing with the current master and not 3.38.1</div>
<div><br>
</div>
<div>Even though I did that PR, I forgot we merged a fix for
this already.<br>
</div>
<div><br>
</div>
<div>With the current master the same is like this:</div>
<div><br>
</div>
<div>$ bin/pdbtool test --validate sample.pdb <br>
WARNING: the template specified in value()/<value>
options for your grouping-by() or db-parser() configuration
has been changed to support typing from syslog-ng 4.0. You
are using an older config version and your template contains
an unrecognized type-cast, probably a parenthesis in the
value field. This will be interpreted in the `type(value)'
format in future versions. Please add an explicit string()
cast as shown in the 'fixed-value' tag of this log message
or remove the parenthesis. The value will be processed as a
'string' expression; config-version='3.38',
name='AUTHPROGRAM', value='su(pam_unix)',
fixed-value='string(su(pam_unix))'<br>
$ echo $?<br>
0<br>
</div>
<div><br>
</div>
<div>The issue is that with the release of v4.0 (which we
started to prepare for), this would become an error in
earnest, as pdbtool does not allow you to specify the config
version and uses the syslog-ng version number by default.
And I don't see too much value in extending pdbtool in this
direction, as having to add an explicit config version
number to pdbtool when using it is not very intuitive.<br>
</div>
<div><br>
</div>
<div>The PR I mentioned in my previous email (#4222) improves
this further by checking the pdb version field (in the
topmost <patterndb> tag) and uses string based values
if it's below v6.</div>
<div><br>
</div>
<div>What I am thinking about is whether to revert to using an
attribute to specify the type, e.g.</div>
<div><br>
</div>
<div> <values><br>
<value name="AUTHPROGRAM"
type="whatevertype-syslog-ng-supports">su(pam_unix)</value><br>
</values></div>
<div><br>
</div>
<div>We would default to using a string. The only downside is
that add-contextual-data() uses the parenthesis based format
in a CSV file and I can't extend the CSV format as easily as
XML.
<br>
</div>
<div><br>
</div>
<div>So by changing to the use of an attribute types would
mean we have different syntaxes for these use-cases where
one can use typed templates:</div>
<div><br>
</div>
<div>- config file anywhere: `type(value)`</div>
<div>- add-contextual-data, 3rd column in the CSV file:
`type(value)`</div>
<div>- db-parser(): `<value name="foo"
type="whatever"/>`<br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Tue, Nov 22, 2022 at 3:14
PM Evan Rempel <<a href="mailto:erempel@uvic.ca"
moz-do-not-send="true" class="moz-txt-link-freetext">erempel@uvic.ca</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div>
<div>I'm not sure if you still think this is only a
warning.</div>
<div><br>
</div>
<div>[xerr@pangolin etc]$ rpm -q syslog-ng<br>
syslog-ng-3.38.1-1.el8.x86_64</div>
<div>[xerr@pangolin etc]$ rpm -qi syslog-ng<br>
Name : syslog-ng<br>
Version : 3.38.1<br>
Release : 1.el8<br>
Architecture: x86_64<br>
Install Date: Thu 03 Nov 2022 04:08:02 AM PDT<br>
Group : System Environment/Daemons<br>
Size : 3651296<br>
License : GPLv2+<br>
Signature : RSA/SHA256, Mon 03 Oct 2022 03:00:51 AM
PDT, Key ID c96af856c87e88fe<br>
Source RPM : syslog-ng-3.38.1-1.el8.src.rpm<br>
Build Date : Mon 03 Oct 2022 03:00:39 AM PDT<br>
Build Host :
copr-hv-x86-64-01-prod-02441780-20221003-095224<br>
Relocations : (not relocatable)<br>
Vendor : Fedora Copr - user czanik<br>
URL : <a
href="http://www.balabit.com/network-security/syslog-ng"
target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">
http://www.balabit.com/network-security/syslog-ng</a><br>
Summary : Next-generation syslog server<br>
</div>
<div><br>
</div>
<div>A very small patterndb</div>
<div><br>
</div>
<div><patterndb version="4" pub_date="2009-09-01"><br>
<ruleset name="su(pam_unix)"
id="RS-d2fae001-a7f6-4e39-ae62-5658906fe48c"><br>
<pattern>su_pam_unix_</pattern><br>
<rules><br>
<rule id="unmatched-program-su(pam_unix)"
class="unknown" provider="UVic"><br>
<patterns><br>
<pattern>@ANYSTRING@</pattern><br>
</patterns><br>
<values><br>
<value
name="AUTHPROGRAM">su(pam_unix)</value><br>
</values><br>
</rule><br>
</rules><br>
</ruleset><br>
</patterndb><br>
</div>
<div><br>
</div>
<div>[xerr@pangolin etc]$ /usr/bin/pdbtool test --validate
test.xml <br>
Error parsing pattern database file;
filename='test.xml', error='test.xml:10:51: Error
compiling value template,
rule=unmatched-program-su(pam_unix), name=AUTHPROGRAM,
value=su(pam_unix), error=Unknown type specified in type
hinting: su'<br>
</div>
<div><br>
</div>
<div>Evan.<br>
</div>
<div><br>
</div>
<div>On 2022-11-22 01:39, Balazs Scheidler wrote:<br>
</div>
<blockquote type="cite"><br>
<div>
<div dir="ltr">
<div>I only got warnings and not errors, when trying
to use this in my XML file:</div>
<div><br>
</div>
<div> <value
name="seq-matches">su(pam_unix)</value></div>
<div><br>
</div>
<div>how did you execute syslog-ng so you got the
error above? The only case where this is an error,
if you are using "@version: 4.0"<br>
</div>
<div><br>
</div>
<div>This was the warning produced (both by
syslog-ng and pdbtool):</div>
<div><br>
</div>
<div>WARNING: the template specified in
value()/<value> options for your
grouping-by() or db-parser() configuration has
been changed to support typing from syslog-ng 4.0.
You are using an older config version and your
template contains an unrecognized type-cast,
probably a parenthesis in the value field. This
will be interpreted in the `type(value)' format in
future versions. Please add an explicit string()
cast as shown in the 'fixed-value' tag of this log
message or remove the parenthesis. The value will
be processed as a 'string' expression;
config-version='3.38', name='seq-matches',
value='su(pam_unix)',
fixed-value='string(su(pam_unix))'</div>
<div><br>
</div>
<div>I chose to embed the type-hint field in the
body of the <value> tag, as this is the
format that is used everywhere else. I might be
able to bump the db-parser XML file version, in
the file header:</div>
<div><br>
</div>
<div><patterndb version='5'></div>
<div><br>
</div>
<div>I could bump this up to version 6, in which
case you'd only need to add the type-hint if you
also bumped the version number. That I think is
doable.</div>
<div><br>
</div>
<div>On the documentation front, there's an XML
schema in the source tree under the doc/xsd/
directory for each version of patterndb, and I
assume the documentation also has a chapter on the
db-parser() format.<br>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Mon, Nov 21,
2022 at 5:22 PM Evan Rempel <<a
href="mailto:erempel@uvic.ca" target="_blank"
moz-do-not-send="true"
class="moz-txt-link-freetext">erempel@uvic.ca</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px
0px 0px 0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
We have a patterndb file that contains both
patterns and values with <br>
fixed text that include a strings of the form
"xxxx (yyy)" which now can <br>
not be loaded by the patterndb.<br>
<br>
Syslog-ng throws the error<br>
<br>
Error parsing pattern database file; ... Error
compiling value template, <br>
rule=FLARE-3543, name=AUTHPROGRAM,
value=su(pam_unix), error=Unknown <br>
type specified in type hinting: su'<br>
<br>
I have not enabled the version 4.0 testing of type
hinting. The version <br>
configuration is<br>
<br>
@version: 3.36<br>
<br>
I am unable to find documentation for the full
syntax of the patterndb <br>
file (seems to have been lost from the docs since
perhaps 3.16?)<br>
<br>
Is there a patterndb syntax specification
document?<br>
<br>
I would expect that even when type hinting is
enabled in the patterndb <br>
file, given that it is an XML document, that the
type hinting would be <br>
part of the XML tag metadata, rather than part of
the XML static data.<br>
<br>
-- <br>
Evan</blockquote>
</div>
</div>
</blockquote>
<br>
</div>
</blockquote>
</div>
</div>
</blockquote>
<br>
</body>
</html>