[syslog-ng] Warnings and error while loading default.xml in syslog-ng-3.25.1
Nitish Saboo
nitish.saboo55 at gmail.com
Tue Feb 18 05:17:59 UTC 2020
Thank you Evan for you response.Looks like changes in .xml file is the only
option.
Thanks,
Nitish
On Mon, Feb 17, 2020 at 10:20 PM Evan Rempel <erempel at uvic.ca> wrote:
> Syslog messages have multiple components to the. Date/time, host, program,
> message and a few others.
> The important point is that the program and message are independent items.
> This means that the pattern database must use a two phase matching method,
> first matching the program and then matching the message.
>
> Another way to look at it is that the same message for two different
> programs means two independent matching rules.
>
> Taking this into account you need to write your patterndb differently.
> Either merging the rulesets (as proposed by aszakacs), or breaking apart
> the duplicate rule (one rule for two programs) as shown below. Although it
> may have worked (or appeared to have worked) in syslog-ng 3.6 or 3.7, it
> was against the patterndb specification.
>
> <ruleset id="f582419b3baa42d4a57e42b89704e38c" description="">
> <pattern>proxysg</pattern>
> <rules>
> <rule id="f1e2bfd7bb85402a88d0b732821a0f94">
> <patterns>
> <pattern>foo</pattern>
> </patterns>
> </rule>
> <rule id="a681963842014480a83a2a2e38875439">
> <patterns>
> <pattern>anything</pattern>
> </patterns>
> </rule>
> <rule id="bb169f917216467985cc16e28015f598">
> <patterns>
> <pattern>bar</pattern>
> </patterns>
> </rule>
> <rule id="94d4a0c324c8-44a88cf3d4640477d399">
> <patterns>
> <pattern>something</pattern>
> </patterns>
> </rule>
> </rules>
> </ruleset>
> <ruleset id="17fae6edff32a53f9f294ab21240fc2641e7a4db" description="">
> <pattern>ProxySG</pattern>
> <rules>
> <rule id="bb169f917216467985cc16e28015f5fa">
> <patterns>
> <pattern>bar</pattern>
> </patterns>
> </rule>
> <rule id="94d4a0c324c8-44a88cf3d4640477d35e">
> <patterns>
> <pattern>something</pattern>
> </patterns>
> </rule>
> </rules>
> </ruleset>
>
>
>
> Both this solution and the one by aszakacs have ONE ruleset for any given
> program (ruleset pattern).
>
> I don't see any other alternatives.
>
> Evan.
>
>
> On 2/17/20 1:29 AM, Nitish Saboo wrote:
>
> +Syslog-ng users' and developers' mailing list
> <syslog-ng at lists.balabit.hu>
>
> On Mon, Feb 17, 2020 at 2:37 PM Nitish Saboo <nitish.saboo55 at gmail.com>
> wrote:
>
>> Hi Attila,
>>
>> 1,2) The issue is that you used "proxysg" in two different rulesets. I
>> believe syslog-ng could handle this, but we do not handle it right now.
>> >> But the same default.xml file is working fine in version
>> syslog-ng-3,6,2 and syslog-ng-3.7.1 where we have same program in different
>> rulesets.The behavior in version syslog-ng-3,6,2 and syslog-ng-3.7.1 is the
>> expected behavior or is the current behavior in syslog-ng-3.25.1 version is
>> having a Bug ?
>>
>> 3/4) You can merge the two rulesets, if it is okay for you:
>> >>Merging the rulesets is not possible at this moment.
>> Do we have any workaround for this scenario other than merging the
>> rulesets?
>> When can we expect a fix for this in near future ?
>>
>> Thanks,
>> Nitish
>>
>> On Mon, Feb 17, 2020 at 2:23 PM Attila Szakacs (aszakacs) <
>> Attila.Szakacs at oneidentity.com> wrote:
>>
>>> Hi!
>>>
>>> 1,2) The issue is that you used "proxysg" in two different rulesets. I
>>> believe syslog-ng could handle this, but we do not handle it right now.
>>> 3/4) You can merge the two rulesets, if it is okay for you:
>>>
>>> <?xml version='1.0' encoding='UTF-8'?>
>>> <patterndb version="4">
>>> <ruleset id="f582419b3baa42d4a57e42b89704e38c" description="">
>>> <pattern>proxysg</pattern>
>>> <pattern>ProxySG</pattern>
>>> <rules>
>>> <rule id="f1e2bfd7bb85402a88d0b732821a0f94">
>>> <patterns>
>>> <pattern>foo</pattern>
>>> </patterns>
>>> </rule>
>>> <rule id="a681963842014480a83a2a2e38875439">
>>> <patterns>
>>> <pattern>anything</pattern>
>>> </patterns>
>>> </rule>
>>> <rule id="bb169f917216467985cc16e28015f5fa">
>>> <patterns>
>>> <pattern>bar</pattern>
>>> </patterns>
>>> </rule>
>>> <rule id="94d4a0c324c8-44a88cf3d4640477d35e">
>>> <patterns>
>>> <pattern>something</pattern>
>>> </patterns>
>>> </rule>
>>> </rules>
>>> </ruleset>
>>> </patterndb>
>>>
>>> Regards,
>>> Attila
>>>
>>> ------------------------------
>>> *From:* syslog-ng <syslog-ng-bounces at lists.balabit.hu> on behalf of
>>> Nitish Saboo <nitish.saboo55 at gmail.com>
>>> *Sent:* Monday, February 17, 2020 6:30 AM
>>> *To:* Syslog-ng users' and developers' mailing list <
>>> syslog-ng at lists.balabit.hu>
>>> *Subject:* Re: [syslog-ng] Warnings and error while loading default.xml
>>> in syslog-ng-3.25.1
>>>
>>> CAUTION: This email originated from outside of the organization. Do not
>>> follow guidance, click links, or open attachments unless you recognize the
>>> sender and know the content is safe.
>>>
>>> Hi Evan,
>>>
>>> Apologies for the confusion but I did close the rule tag in my
>>> default.xml , missed adding it here.
>>> The error does not seem to be related to improper closing of tags.
>>>
>>> This is how it looks :
>>>
>>> <ruleset id="f582419b3baa42d4a57e42b89704e38c" description="">
>>> <pattern>proxysg</pattern>
>>> <rules>
>>> <rule id="f1e2bfd7bb85402a88d0b732821a0f94">
>>> <patterns>
>>> <pattern>foo</pattern>
>>> </patterns>
>>> </rule>
>>> <rule id="a681963842014480a83a2a2e38875439">
>>> <patterns>
>>> <pattern>anything</pattern>
>>> </patterns>
>>> </rule>
>>> </rules>
>>> </ruleset>
>>> <ruleset id="17fae6edff32a53f9f294ab21240fc2641e7a4db" description="">
>>> <pattern>ProxySG</pattern>
>>> <pattern>proxysg</pattern>
>>> <rules>
>>> <rule id="bb169f917216467985cc16e28015f5fa">
>>> <patterns>
>>> <pattern>bar</pattern>
>>> </patterns>
>>> </rule>
>>> <rule id="94d4a0c324c8-44a88cf3d4640477d35e">
>>> <patterns>
>>> <pattern>something</pattern>
>>> </patterns>
>>> </rule>
>>> </rules>
>>> </ruleset>
>>>
>>>
>>> I am getting the following error message:
>>>
>>> 2020-02-13T10:47:29.631090] Error parsing pattern database file;
>>> filename='/home/nsaboo/abc/default.xml',
>>> error='/home/nsaboo/abc/default.xml:17274:22: Joining rulesets with
>>> mismatching program name sets, program=proxysg'.
>>>
>>> My hunch is the issue is related to merging of two rulesets but I am not
>>> able to understand why is there mismatching of program names.
>>>
>>> 1) Can someone please help me understand the issue here ?
>>>
>>> 2) Is the issue seen because a ruleset has multiple programs in it or
>>> is it because the same program 'proxysg' is being used in different
>>> rulesets ?
>>>
>>> 3) From the above snippet of default.xml, what changes can I make into
>>> default.xml to avoid the error ?
>>>
>>> 4) Is there a workaround for this issue ?
>>>
>>> Thanks,
>>> Nitish
>>>
>>>
>>> On Sun, Feb 16, 2020 at 12:40 AM Evan Rempel <erempel at uvic.ca> wrote:
>>>
>>> I'm not exactly sure what is or is not permitted in the pattern database
>>> but I two comments.
>>>
>>> 1. you need end your rule tag and your rules tag before you start a new
>>> ruleset tag.
>>>
>>> 2. What I do in my pattern database is of the form.
>>>
>>> <ruleset id="f582419b3baa42d4a57e42b89704e38c" description="">
>>> <pattern>proxysg</pattern>
>>> <rules>
>>> <rule id="f582419b3baa42d4a57e42b89704e38c">
>>> <patterns>
>>> <pattern>foo</pattern>
>>> </patterns>
>>> </rule>
>>> <rule id="bb169f917216467985cc16e28015f5fa">
>>> <patterns>
>>> <pattern>bar</pattern>
>>> </patterns>
>>> </rules>
>>> </ruleset>
>>>
>>>
>>> Note:
>>> 1. the closing tag of </rule> before a new starting tag of <rule>
>>> 2. Multiple "rule" entries inside the the "rules" entry.
>>> 3. the closing tag of </rule> before the closing tag of </rules>
>>> 4. the closing tag of </rules> before the closing tag of </ruleset>
>>>
>>> I hope that helps.
>>>
>>> Evan.
>>>
>>> On 2/15/20 12:43 AM, Nitish Saboo wrote:
>>>
>>> Hi,
>>>
>>> After debugging further into the issue looks like there was a fix for
>>> patterndb rule clash in syslog-ng-3.8 and this is the commit-id
>>> '12cd960c8f47260b0b0d4154b096994d66fe345'
>>> for the fix. And for this reason I am getting the following error for
>>> same default.xml in syslog-ng-3.25.1 version and not in syslog-ng3.6.2 and
>>> syslog-ng3.7.1.
>>>
>>> 2020-02-13T10:47:29.631090] Error parsing pattern database file;
>>> filename='/home/nsaboo/abc/default.xml',
>>> error='/home/nsaboo/abc/default.xml:17274:22: Joining rulesets with
>>> mismatching program name sets, program=proxysg'.
>>>
>>> Snippet from default.xml
>>> ==========================
>>>
>>> <ruleset id="f582419b3baa42d4a57e42b89704e38c" description="">
>>> <pattern>proxysg</pattern>
>>> <rules>
>>> <rule id="f582419b3baa42d4a57e42b89704e38c">
>>> <patterns>
>>> <pattern>foo</pattern>
>>> </patterns>
>>>
>>> <ruleset id="8d633c824e844a559088d803464e507a" description="">
>>> <pattern>ProxySG</pattern>
>>> <pattern>proxysg</pattern>
>>> <rules>
>>> <rule id="bb169f917216467985cc16e28015f5fa">
>>> <patterns>
>>> <pattern>bar</pattern>
>>> </patterns>
>>>
>>> I am not able to understand the error message clearly.
>>>
>>> 1) Can someone please help me understand the issue here ?
>>>
>>> 2) Is the issue seen because a ruleset has multiple programs in it or
>>> is it because the same program 'proxysg' is being used in different
>>> rulesets ?
>>>
>>> 3) From the above snippet of default.xml, what changes can I make into
>>> default.xml to avoid the error ?
>>>
>>> 4) Is there a workaround for this issue ?
>>>
>>> Thanks,
>>> Nitish
>>>
>>> On Fri, Feb 14, 2020 at 2:40 PM Nitish Saboo <nitish.saboo55 at gmail.com>
>>> wrote:
>>>
>>> Hi Attila,
>>>
>>> Thanks for your response.
>>>
>>> And what about the following error:
>>>
>>> 2020-02-13T10:47:29.631090] Error parsing pattern database file;
>>> filename='/home/nsaboo/abc/default.xml',
>>> error='/home/nsaboo/abc/default.xml:17274:22: Joining rulesets with
>>> mismatching program name sets, program=proxysg'.
>>>
>>> The same default.xml file was getting loaded correctly in
>>> syslog-ng-3.6.2 and syslog-ng-3.7.1 but getting following error while
>>> loading same default.xml in syslog-ng3.25.1
>>>
>>> I came across a similar issue on githib '
>>> https://github.com/syslog-ng/syslog-ng/issues/2763
>>> <https://nam05.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fsyslog-ng%2Fsyslog-ng%2Fissues%2F2763&data=02%7C01%7CAttila.Szakacs%40oneidentity.com%7C7984ff7f5a4241fdae2208d7b36a9848%7C91c369b51c9e439c989c1867ec606603%7C0%7C0%7C637175142727691628&sdata=WdmKYNdboeB7Y26uuDXhmR225uPqIfX6x4U3Sff2qbY%3D&reserved=0>'
>>> .I see the issue is still in open state.Is there a workaround for
>>> this issue?
>>>
>>> Thanks,
>>> Nitish
>>>
>>> On Fri, Feb 14, 2020 at 1:12 PM Attila Szakacs (aszakacs) <
>>> Attila.Szakacs at oneidentity.com> wrote:
>>>
>>> Hi!
>>>
>>> WARNING: due to a bug in versions before syslog-ng 3.8numeric comparison
>>> operators like '!=' in filter expressions were evaluated as string
>>> operators. This is fixed in syslog-ng 3.8. As we are operating in
>>> compatibility mode, syslog-ng will exhibit the buggy behaviour as previous
>>> versions until you bump the @version value in your configuration file;
>>>
>>> ^^^ This refers to the syslog-ng.conf file version.
>>>
>>> The correct way to resolve it, and fix the buggy behavior of != and ==,
>>> should be to change the != operators between strings to neq in your filters.
>>>
>>> Regards,
>>> Attila
>>> ------------------------------
>>> *From:* syslog-ng <syslog-ng-bounces at lists.balabit.hu> on behalf of
>>> Nitish Saboo <nitish.saboo55 at gmail.com>
>>> *Sent:* Thursday, February 13, 2020 12:17 PM
>>> *To:* Syslog-ng users' and developers' mailing list <
>>> syslog-ng at lists.balabit.hu>
>>> *Subject:* [syslog-ng] Warnings and error while loading default.xml in
>>> syslog-ng-3.25.1
>>>
>>> Hi,
>>>
>>> I am using syslog-ng version 3.25.1.Getting following Warnings and error
>>> while initialising syslog-ng engine:
>>>
>>> [2020-02-13T10:47:29.627899] WARNING: due to a bug in versions before
>>> syslog-ng 3.8numeric comparison operators like '!=' in filter expressions
>>> were evaluated as string operators. This is fixed in syslog-ng 3.8. As we
>>> are operating in compatibility mode, syslog-ng will exhibit the buggy
>>> behaviour as previous versions until you bump the @version value in your
>>> configuration file;
>>> [2020-02-13T10:47:29.627968] WARNING: due to a bug in versions before
>>> syslog-ng 3.8numeric comparison operators like '!=' in filter expressions
>>> were evaluated as string operators. This is fixed in syslog-ng 3.8. As we
>>> are operating in compatibility mode, syslog-ng will exhibit the buggy
>>> behaviour as previous versions until you bump the @version value in your
>>> configuration file;
>>> [2020-02-13T10:47:29.628059] WARNING: due to a bug in versions before
>>> syslog-ng 3.8numeric comparison operators like '!=' in filter expressions
>>> were evaluated as string operators. This is fixed in syslog-ng 3.8. As we
>>> are operating in compatibility mode, syslog-ng will exhibit the buggy
>>> behaviour as previous versions until you bump the @version value in your
>>> configuration file;
>>> [2020-02-13T10:47:29.631090] Error parsing pattern database file;
>>> filename='/opt/tap-parsing/patterns/default.xml',
>>> error='/opt/tap-parsing/patterns/default.xml:17274:22: Joining rulesets
>>> with mismatching program name sets, program=proxysg'
>>>
>>>
>>> 1)For the following warnings, to which version I have to bump up the
>>> configuration file ?
>>>
>>> 2020-02-13T10:47:29.627899] WARNING: due to a bug in versions before
>>> syslog-ng 3.8numeric comparison operators like '!=' in filter expressions
>>> were evaluated as string operators. This is fixed in syslog-ng 3.8. As we
>>> are operating in compatibility mode, syslog-ng will exhibit the buggy
>>> behaviour as previous versions until you bump the @version value in your
>>> configuration file;
>>> [2020-02-13T10:47:29.627968] WARNING: due to a bug in versions before
>>> syslog-ng 3.8numeric comparison operators like '!=' in filter expressions
>>> were evaluated as string operators. This is fixed in syslog-ng 3.8. As we
>>> are operating in compatibility mode, syslog-ng will exhibit the buggy
>>> behaviour as previous versions until you bump the @version value in your
>>> configuration file;
>>> [2020-02-13T10:47:29.628059] WARNING: due to a bug in versions before
>>> syslog-ng 3.8numeric comparison operators like '!=' in filter expressions
>>> were evaluated as string operators. This is fixed in syslog-ng 3.8. As we
>>> are operating in compatibility mode, syslog-ng will exhibit the buggy
>>> behaviour as previous versions until you bump the @version value in your
>>> configuration file;
>>>
>>> Currrently the configuration version is the following:
>>>
>>> configuration = cfg_new(0x0302)
>>>
>>> Do I have to change it to '0x0319' as defined in 'lib/versioning.h' ?
>>>
>>> 2)The same default.xml file was getting loaded correctly in
>>> syslog-ng-3.6.2 and syslog-ng-3.7.1 but getting following error while
>>> loading same default.xml in syslog-ng3.25.1
>>>
>>>
>>> 2020-02-13T10:47:29.631090] Error parsing pattern database file;
>>> filename='/home/nsaboo/abc/default.xml',
>>> error='/home/nsaboo/abc/default.xml:17274:22: Joining rulesets with
>>> mismatching program name sets, program=proxysg'.
>>>
>>> What can be the reason for this error ?
>>>
>>>
>>> Thanks,
>>>
>>>
>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation:
> http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20200218/96e627d7/attachment-0001.html>
More information about the syslog-ng
mailing list