[syslog-ng] Warnings and error while loading default.xml in syslog-ng-3.25.1
Evan Rempel
erempel at uvic.ca
Mon Feb 17 16:50:22 UTC 2020
Syslog messages have multiple components to the. Date/time, host,
program, message and a few others.
The important point is that the program and message are independent
items. This means that the pattern database must use a two phase
matching method, first matching the program and then matching the message.
Another way to look at it is that the same message for two different
programs means two independent matching rules.
Taking this into account you need to write your patterndb differently.
Either merging the rulesets (as proposed by aszakacs), or breaking apart
the duplicate rule (one rule for two programs) as shown below. Although
it may have worked (or appeared to have worked) in syslog-ng 3.6 or 3.7,
it was against the patterndb specification.
<ruleset id="f582419b3baa42d4a57e42b89704e38c" description="">
<pattern>proxysg</pattern>
<rules>
<rule id="f1e2bfd7bb85402a88d0b732821a0f94">
<patterns>
<pattern>foo</pattern>
</patterns>
</rule>
<rule id="a681963842014480a83a2a2e38875439">
<patterns>
<pattern>anything</pattern>
</patterns>
</rule>
<rule id="bb169f917216467985cc16e28015f598">
<patterns>
<pattern>bar</pattern>
</patterns>
</rule>
<rule id="94d4a0c324c8-44a88cf3d4640477d399">
<patterns>
<pattern>something</pattern>
</patterns>
</rule>
</rules>
</ruleset>
<ruleset id="17fae6edff32a53f9f294ab21240fc2641e7a4db" description="">
<pattern>ProxySG</pattern>
<rules>
<rule id="bb169f917216467985cc16e28015f5fa">
<patterns>
<pattern>bar</pattern>
</patterns>
</rule>
<rule id="94d4a0c324c8-44a88cf3d4640477d35e">
<patterns>
<pattern>something</pattern>
</patterns>
</rule>
</rules>
</ruleset>
Both this solution and the one by aszakacs have ONE ruleset for any
given program (ruleset pattern).
I don't see any other alternatives.
Evan.
On 2/17/20 1:29 AM, Nitish Saboo wrote:
> +Syslog-ng users' and developers' mailing list
> <mailto:syslog-ng at lists.balabit.hu>
>
> On Mon, Feb 17, 2020 at 2:37 PM Nitish Saboo <nitish.saboo55 at gmail.com
> <mailto:nitish.saboo55 at gmail.com>> wrote:
>
> Hi Attila,
>
> 1,2) The issue is that you used "proxysg" in two different
> rulesets. I believe syslog-ng could handle this, but we do not
> handle it right now.
> >> But the same default.xml file is working fine in version
> syslog-ng-3,6,2 and syslog-ng-3.7.1 where we have same program in
> different rulesets.The behavior in version syslog-ng-3,6,2 and
> syslog-ng-3.7.1 is the expected behavior or is the current
> behavior in syslog-ng-3.25.1 version is having a Bug ?
>
> 3/4) You can merge the two rulesets, if it is okay for you:
> >>Merging the rulesets is not possible at this moment.
> Do we have any workaround for this scenario other than merging the
> rulesets?
> When can we expect a fix for this in near future ?
>
> Thanks,
> Nitish
>
> On Mon, Feb 17, 2020 at 2:23 PM Attila Szakacs (aszakacs)
> <Attila.Szakacs at oneidentity.com
> <mailto:Attila.Szakacs at oneidentity.com>> wrote:
>
> Hi!
>
> 1,2) The issue is that you used "proxysg" in two different
> rulesets. I believe syslog-ng could handle this, but we do not
> handle it right now.
> 3/4) You can merge the two rulesets, if it is okay for you:
>
> <?xml version='1.0' encoding='UTF-8'?>
> <patterndb version="4">
> <ruleset id="f582419b3baa42d4a57e42b89704e38c"
> description="">
> <pattern>proxysg</pattern>
> <pattern>ProxySG</pattern>
> <rules>
> <rule id="f1e2bfd7bb85402a88d0b732821a0f94">
> <patterns>
> <pattern>foo</pattern>
> </patterns>
> </rule>
> <rule id="a681963842014480a83a2a2e38875439">
> <patterns>
> <pattern>anything</pattern>
> </patterns>
> </rule>
> <rule id="bb169f917216467985cc16e28015f5fa">
> <patterns>
> <pattern>bar</pattern>
> </patterns>
> </rule>
> <rule id="94d4a0c324c8-44a88cf3d4640477d35e">
> <patterns>
> <pattern>something</pattern>
> </patterns>
> </rule>
> </rules>
> </ruleset>
> </patterndb>
>
> Regards,
> Attila
>
> ------------------------------------------------------------------------
> *From:* syslog-ng <syslog-ng-bounces at lists.balabit.hu
> <mailto:syslog-ng-bounces at lists.balabit.hu>> on behalf of
> Nitish Saboo <nitish.saboo55 at gmail.com
> <mailto:nitish.saboo55 at gmail.com>>
> *Sent:* Monday, February 17, 2020 6:30 AM
> *To:* Syslog-ng users' and developers' mailing list
> <syslog-ng at lists.balabit.hu <mailto:syslog-ng at lists.balabit.hu>>
> *Subject:* Re: [syslog-ng] Warnings and error while loading
> default.xml in syslog-ng-3.25.1
> CAUTION: This email originated from outside of the
> organization. Do not follow guidance, click links, or open
> attachments unless you recognize the sender and know the
> content is safe.
>
> Hi Evan,
>
> Apologies for the confusion but I did close the rule tag in my
> default.xml , missed adding it here.
> The error does not seem to be related to improper closing of tags.
>
> This is how it looks :
>
> <ruleset id="f582419b3baa42d4a57e42b89704e38c" description="">
> <pattern>proxysg</pattern>
> <rules>
> <rule id="f1e2bfd7bb85402a88d0b732821a0f94">
> <patterns>
> <pattern>foo</pattern>
> </patterns>
> </rule>
> <rule id="a681963842014480a83a2a2e38875439">
> <patterns>
> <pattern>anything</pattern>
> </patterns>
> </rule>
> </rules>
> </ruleset>
> <ruleset id="17fae6edff32a53f9f294ab21240fc2641e7a4db"
> description="">
> <pattern>ProxySG</pattern>
> <pattern>proxysg</pattern>
> <rules>
> <rule id="bb169f917216467985cc16e28015f5fa">
> <patterns>
> <pattern>bar</pattern>
> </patterns>
> </rule>
> <rule id="94d4a0c324c8-44a88cf3d4640477d35e">
> <patterns>
> <pattern>something</pattern>
> </patterns>
> </rule>
> </rules>
> </ruleset>
>
>
> I am getting the following error message:
>
> 2020-02-13T10:47:29.631090] Error parsing pattern database
> file; filename='/home/nsaboo/abc/default.xml',
> error='/home/nsaboo/abc/default.xml:17274:22: Joining rulesets
> with mismatching program name sets, program=proxysg'.
>
> My hunch is the issue is related to merging of two rulesets
> but I am not able to understand why is there mismatching of
> program names.
>
> 1) Can someone please help me understand the issue here ?
>
> 2) Is the issue seen because a ruleset has multiple programs
> in it or is it because the same program 'proxysg' is being
> used in different rulesets ?
>
> 3) From the above snippet of default.xml, what changes can I
> make into default.xml to avoid the error ?
>
> 4) Is there a workaround for this issue ?
>
> Thanks,
> Nitish
>
>
> On Sun, Feb 16, 2020 at 12:40 AM Evan Rempel <erempel at uvic.ca
> <mailto:erempel at uvic.ca>> wrote:
>
> I'm not exactly sure what is or is not permitted in the
> pattern database but I two comments.
>
> 1. you need end your rule tag and your rules tag before
> you start a new ruleset tag.
>
> 2. What I do in my pattern database is of the form.
>
> <ruleset id="f582419b3baa42d4a57e42b89704e38c" description="">
> <pattern>proxysg</pattern>
> <rules>
> <rule id="f582419b3baa42d4a57e42b89704e38c">
> <patterns>
> <pattern>foo</pattern>
> </patterns>
> </rule>
> <rule id="bb169f917216467985cc16e28015f5fa">
> <patterns>
> <pattern>bar</pattern>
> </patterns>
> </rules>
> </ruleset>
>
>
> Note:
> 1. the closing tag of </rule> before a new starting tag of
> <rule>
> 2. Multiple "rule" entries inside the the "rules" entry.
> 3. the closing tag of </rule> before the closing tag of
> </rules>
> 4. the closing tag of </rules> before the closing tag of
> </ruleset>
>
> I hope that helps.
>
> Evan.
>
> On 2/15/20 12:43 AM, Nitish Saboo wrote:
>> Hi,
>>
>> After debugging further into the issue looks like there
>> was a fix for patterndb rule clash in syslog-ng-3.8 and
>> this is the commit-id
>> '12cd960c8f47260b0b0d4154b096994d66fe345'
>> for the fix. And for this reason I am getting the
>> following error for same default.xml in syslog-ng-3.25.1
>> version and not in syslog-ng3.6.2 and syslog-ng3.7.1.
>>
>> 2020-02-13T10:47:29.631090] Error parsing pattern
>> database file; filename='/home/nsaboo/abc/default.xml',
>> error='/home/nsaboo/abc/default.xml:17274:22: Joining
>> rulesets with mismatching program name sets,
>> program=proxysg'.
>>
>> Snippet from default.xml
>> ==========================
>>
>> <ruleset id="f582419b3baa42d4a57e42b89704e38c"
>> description="">
>> <pattern>proxysg</pattern>
>> <rules>
>> <rule id="f582419b3baa42d4a57e42b89704e38c">
>> <patterns>
>> <pattern>foo</pattern>
>> </patterns>
>>
>> <ruleset id="8d633c824e844a559088d803464e507a"
>> description="">
>> <pattern>ProxySG</pattern>
>> <pattern>proxysg</pattern>
>> <rules>
>> <rule id="bb169f917216467985cc16e28015f5fa">
>> <patterns>
>> <pattern>bar</pattern>
>> </patterns>
>>
>> I am not able to understand the error message clearly.
>>
>> 1) Can someone please help me understand the issue here ?
>>
>> 2) Is the issue seen because a ruleset has multiple
>> programs in it or is it because the same program
>> 'proxysg' is being used in different rulesets ?
>>
>> 3) From the above snippet of default.xml, what changes
>> can I make into default.xml to avoid the error ?
>>
>> 4) Is there a workaround for this issue ?
>>
>> Thanks,
>> Nitish
>>
>> On Fri, Feb 14, 2020 at 2:40 PM Nitish Saboo
>> <nitish.saboo55 at gmail.com
>> <mailto:nitish.saboo55 at gmail.com>> wrote:
>>
>> Hi Attila,
>>
>> Thanks for your response.
>>
>> And what about the following error:
>>
>> 2020-02-13T10:47:29.631090] Error parsing pattern
>> database file;
>> filename='/home/nsaboo/abc/default.xml',
>> error='/home/nsaboo/abc/default.xml:17274:22: Joining
>> rulesets with mismatching program name sets,
>> program=proxysg'.
>>
>> The same default.xml file was getting loaded
>> correctly in syslog-ng-3.6.2 and syslog-ng-3.7.1 but
>> getting following error while loading same
>> default.xml in syslog-ng3.25.1
>>
>> I came across a similar issue on githib
>> 'https://github.com/syslog-ng/syslog-ng/issues/2763
>> <https://nam05.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fsyslog-ng%2Fsyslog-ng%2Fissues%2F2763&data=02%7C01%7CAttila.Szakacs%40oneidentity.com%7C7984ff7f5a4241fdae2208d7b36a9848%7C91c369b51c9e439c989c1867ec606603%7C0%7C0%7C637175142727691628&sdata=WdmKYNdboeB7Y26uuDXhmR225uPqIfX6x4U3Sff2qbY%3D&reserved=0>'
>> .I see the issue is still in open state.Is there a
>> workaround for this issue?
>>
>> Thanks,
>> Nitish
>>
>> On Fri, Feb 14, 2020 at 1:12 PM Attila Szakacs
>> (aszakacs) <Attila.Szakacs at oneidentity.com
>> <mailto:Attila.Szakacs at oneidentity.com>> wrote:
>>
>> Hi!
>>
>> WARNING: due to a bug in versions before
>> syslog-ng 3.8numeric comparison operators like
>> '!=' in filter expressions were evaluated as
>> string operators. This is fixed in syslog-ng 3.8.
>> As we are operating in compatibility mode,
>> syslog-ng will exhibit the buggy behaviour as
>> previous versions until you bump the @version
>> value in your configuration file;
>>
>> ^^^ This refers to the syslog-ng.conf file version.
>>
>> The correct way to resolve it, and fix the buggy
>> behavior of != and ==, should be to change the !=
>> operators between strings to neq in your filters.
>>
>> Regards,
>> Attila
>> ------------------------------------------------------------------------
>> *From:* syslog-ng
>> <syslog-ng-bounces at lists.balabit.hu
>> <mailto:syslog-ng-bounces at lists.balabit.hu>> on
>> behalf of Nitish Saboo <nitish.saboo55 at gmail.com
>> <mailto:nitish.saboo55 at gmail.com>>
>> *Sent:* Thursday, February 13, 2020 12:17 PM
>> *To:* Syslog-ng users' and developers' mailing
>> list <syslog-ng at lists.balabit.hu
>> <mailto:syslog-ng at lists.balabit.hu>>
>> *Subject:* [syslog-ng] Warnings and error while
>> loading default.xml in syslog-ng-3.25.1
>> Hi,
>>
>> I am using syslog-ng version 3.25.1.Getting
>> following Warnings and error while initialising
>> syslog-ng engine:
>>
>> [2020-02-13T10:47:29.627899] WARNING: due to a
>> bug in versions before syslog-ng 3.8numeric
>> comparison operators like '!=' in filter
>> expressions were evaluated as string operators.
>> This is fixed in syslog-ng 3.8. As we are
>> operating in compatibility mode, syslog-ng will
>> exhibit the buggy behaviour as previous versions
>> until you bump the @version value in your
>> configuration file;
>> [2020-02-13T10:47:29.627968] WARNING: due to a
>> bug in versions before syslog-ng 3.8numeric
>> comparison operators like '!=' in filter
>> expressions were evaluated as string operators.
>> This is fixed in syslog-ng 3.8. As we are
>> operating in compatibility mode, syslog-ng will
>> exhibit the buggy behaviour as previous versions
>> until you bump the @version value in your
>> configuration file;
>> [2020-02-13T10:47:29.628059] WARNING: due to a
>> bug in versions before syslog-ng 3.8numeric
>> comparison operators like '!=' in filter
>> expressions were evaluated as string operators.
>> This is fixed in syslog-ng 3.8. As we are
>> operating in compatibility mode, syslog-ng will
>> exhibit the buggy behaviour as previous versions
>> until you bump the @version value in your
>> configuration file;
>> [2020-02-13T10:47:29.631090] Error parsing
>> pattern database file;
>> filename='/opt/tap-parsing/patterns/default.xml',
>> error='/opt/tap-parsing/patterns/default.xml:17274:22:
>> Joining rulesets with mismatching program name
>> sets, program=proxysg'
>>
>>
>> 1)For the following warnings, to which version I
>> have to bump up the configuration file ?
>>
>> 2020-02-13T10:47:29.627899] WARNING: due to a bug
>> in versions before syslog-ng 3.8numeric
>> comparison operators like '!=' in filter
>> expressions were evaluated as string operators.
>> This is fixed in syslog-ng 3.8. As we are
>> operating in compatibility mode, syslog-ng will
>> exhibit the buggy behaviour as previous versions
>> until you bump the @version value in your
>> configuration file;
>> [2020-02-13T10:47:29.627968] WARNING: due to a
>> bug in versions before syslog-ng 3.8numeric
>> comparison operators like '!=' in filter
>> expressions were evaluated as string operators.
>> This is fixed in syslog-ng 3.8. As we are
>> operating in compatibility mode, syslog-ng will
>> exhibit the buggy behaviour as previous versions
>> until you bump the @version value in your
>> configuration file;
>> [2020-02-13T10:47:29.628059] WARNING: due to a
>> bug in versions before syslog-ng 3.8numeric
>> comparison operators like '!=' in filter
>> expressions were evaluated as string operators.
>> This is fixed in syslog-ng 3.8. As we are
>> operating in compatibility mode, syslog-ng will
>> exhibit the buggy behaviour as previous versions
>> until you bump the @version value in your
>> configuration file;
>>
>> Currrently the configuration version is the
>> following:
>>
>> configuration = cfg_new(0x0302)
>>
>> Do I have to change it to '0x0319' as defined in
>> 'lib/versioning.h' ?
>>
>> 2)The same default.xml file was getting loaded
>> correctly in syslog-ng-3.6.2 and syslog-ng-3.7.1
>> but getting following error while loading same
>> default.xml in syslog-ng3.25.1
>>
>>
>> 2020-02-13T10:47:29.631090] Error parsing pattern
>> database file;
>> filename='/home/nsaboo/abc/default.xml',
>> error='/home/nsaboo/abc/default.xml:17274:22:
>> Joining rulesets with mismatching program name
>> sets, program=proxysg'.
>>
>> What can be the reason for this error ?
>>
>>
>> Thanks,
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20200217/c56ee620/attachment-0001.html>
More information about the syslog-ng
mailing list