[syslog-ng] Warnings and error while loading default.xml in syslog-ng-3.25.1
Balazs Scheidler
bazsi77 at gmail.com
Fri Feb 21 10:06:30 UTC 2020
My guess is: because in one ruleset you only have one program whereas in
the other you have multiple. Make sure the list of programs are the same.
I dont have source code nor the abioity to try it at the moment, as I am
standing on the top of a hill in the Alps :)
On Fri, Feb 21, 2020, 10:59 Nitish Saboo <nitish.saboo55 at gmail.com> wrote:
> Hi Evan, Attila,
>
> I got the following Note from the following link.I have made the line
> bold.This is from syslog-ng Adminitrator guide from 3.25 version
>
>
> https://support.oneidentity.com/technical-documents/syslog-ng-open-source-edition/3.25/administration-guide/77#TOPIC-1349587
>
> NOTE:
> If the ${PROGRAM} part of a message is empty, rules with an empty Program
> Pattern are used to classify the message.
>
>
> *If the same Program Pattern is used in multiple rulesets, the rules of
> these rulesets are merged, and every rule is used to classify the message.
> Note that message patterns must be unique within the merged rulesets, but
> the currently only one ruleset is checked for uniqueness.*
> If the content of the ${PROGRAM} macro is not the proper name of the
> application, you can use the program-template() option to specify it.
>
>
> 1) According to that line( bold one) you should accept multiple program
> pattern in different rulesets ..Am i correct? Please correct me if I am
> missing something here.
>
> 2) If my understanding is correct for [1], then why is my default.xml
> failing with the following error while loading:
>
> 2020-02-13T10:47:29.631090] Error parsing pattern database file;
> filename='/home/nsaboo/abc/default.xml',
> error='/home/nsaboo/abc/default.xml:17274:22: Joining rulesets with
> mismatching program name sets, program=proxysg'.
>
> Thanks,
> Nitish
>
> On Tue, Feb 18, 2020 at 10:47 AM Nitish Saboo <nitish.saboo55 at gmail.com>
> wrote:
>
>> Thank you Evan for you response.Looks like changes in .xml file is the
>> only option.
>>
>> Thanks,
>> Nitish
>>
>> On Mon, Feb 17, 2020 at 10:20 PM Evan Rempel <erempel at uvic.ca> wrote:
>>
>>> Syslog messages have multiple components to the. Date/time, host,
>>> program, message and a few others.
>>> The important point is that the program and message are independent
>>> items. This means that the pattern database must use a two phase matching
>>> method, first matching the program and then matching the message.
>>>
>>> Another way to look at it is that the same message for two different
>>> programs means two independent matching rules.
>>>
>>> Taking this into account you need to write your patterndb differently.
>>> Either merging the rulesets (as proposed by aszakacs), or breaking apart
>>> the duplicate rule (one rule for two programs) as shown below. Although it
>>> may have worked (or appeared to have worked) in syslog-ng 3.6 or 3.7, it
>>> was against the patterndb specification.
>>>
>>> <ruleset id="f582419b3baa42d4a57e42b89704e38c" description="">
>>> <pattern>proxysg</pattern>
>>> <rules>
>>> <rule id="f1e2bfd7bb85402a88d0b732821a0f94">
>>> <patterns>
>>> <pattern>foo</pattern>
>>> </patterns>
>>> </rule>
>>> <rule id="a681963842014480a83a2a2e38875439">
>>> <patterns>
>>> <pattern>anything</pattern>
>>> </patterns>
>>> </rule>
>>> <rule id="bb169f917216467985cc16e28015f598">
>>> <patterns>
>>> <pattern>bar</pattern>
>>> </patterns>
>>> </rule>
>>> <rule id="94d4a0c324c8-44a88cf3d4640477d399">
>>> <patterns>
>>> <pattern>something</pattern>
>>> </patterns>
>>> </rule>
>>> </rules>
>>> </ruleset>
>>> <ruleset id="17fae6edff32a53f9f294ab21240fc2641e7a4db" description="">
>>> <pattern>ProxySG</pattern>
>>> <rules>
>>> <rule id="bb169f917216467985cc16e28015f5fa">
>>> <patterns>
>>> <pattern>bar</pattern>
>>> </patterns>
>>> </rule>
>>> <rule id="94d4a0c324c8-44a88cf3d4640477d35e">
>>> <patterns>
>>> <pattern>something</pattern>
>>> </patterns>
>>> </rule>
>>> </rules>
>>> </ruleset>
>>>
>>>
>>>
>>> Both this solution and the one by aszakacs have ONE ruleset for any
>>> given program (ruleset pattern).
>>>
>>> I don't see any other alternatives.
>>>
>>> Evan.
>>>
>>>
>>> On 2/17/20 1:29 AM, Nitish Saboo wrote:
>>>
>>> +Syslog-ng users' and developers' mailing list
>>> <syslog-ng at lists.balabit.hu>
>>>
>>> On Mon, Feb 17, 2020 at 2:37 PM Nitish Saboo <nitish.saboo55 at gmail.com>
>>> wrote:
>>>
>>>> Hi Attila,
>>>>
>>>> 1,2) The issue is that you used "proxysg" in two different rulesets. I
>>>> believe syslog-ng could handle this, but we do not handle it right now.
>>>> >> But the same default.xml file is working fine in version
>>>> syslog-ng-3,6,2 and syslog-ng-3.7.1 where we have same program in different
>>>> rulesets.The behavior in version syslog-ng-3,6,2 and syslog-ng-3.7.1 is the
>>>> expected behavior or is the current behavior in syslog-ng-3.25.1 version is
>>>> having a Bug ?
>>>>
>>>> 3/4) You can merge the two rulesets, if it is okay for you:
>>>> >>Merging the rulesets is not possible at this moment.
>>>> Do we have any workaround for this scenario other than merging the
>>>> rulesets?
>>>> When can we expect a fix for this in near future ?
>>>>
>>>> Thanks,
>>>> Nitish
>>>>
>>>> On Mon, Feb 17, 2020 at 2:23 PM Attila Szakacs (aszakacs) <
>>>> Attila.Szakacs at oneidentity.com> wrote:
>>>>
>>>>> Hi!
>>>>>
>>>>> 1,2) The issue is that you used "proxysg" in two different rulesets. I
>>>>> believe syslog-ng could handle this, but we do not handle it right now.
>>>>> 3/4) You can merge the two rulesets, if it is okay for you:
>>>>>
>>>>> <?xml version='1.0' encoding='UTF-8'?>
>>>>> <patterndb version="4">
>>>>> <ruleset id="f582419b3baa42d4a57e42b89704e38c" description="">
>>>>> <pattern>proxysg</pattern>
>>>>> <pattern>ProxySG</pattern>
>>>>> <rules>
>>>>> <rule id="f1e2bfd7bb85402a88d0b732821a0f94">
>>>>> <patterns>
>>>>> <pattern>foo</pattern>
>>>>> </patterns>
>>>>> </rule>
>>>>> <rule id="a681963842014480a83a2a2e38875439">
>>>>> <patterns>
>>>>> <pattern>anything</pattern>
>>>>> </patterns>
>>>>> </rule>
>>>>> <rule id="bb169f917216467985cc16e28015f5fa">
>>>>> <patterns>
>>>>> <pattern>bar</pattern>
>>>>> </patterns>
>>>>> </rule>
>>>>> <rule id="94d4a0c324c8-44a88cf3d4640477d35e">
>>>>> <patterns>
>>>>> <pattern>something</pattern>
>>>>> </patterns>
>>>>> </rule>
>>>>> </rules>
>>>>> </ruleset>
>>>>> </patterndb>
>>>>>
>>>>> Regards,
>>>>> Attila
>>>>>
>>>>> ------------------------------
>>>>> *From:* syslog-ng <syslog-ng-bounces at lists.balabit.hu> on behalf of
>>>>> Nitish Saboo <nitish.saboo55 at gmail.com>
>>>>> *Sent:* Monday, February 17, 2020 6:30 AM
>>>>> *To:* Syslog-ng users' and developers' mailing list <
>>>>> syslog-ng at lists.balabit.hu>
>>>>> *Subject:* Re: [syslog-ng] Warnings and error while loading
>>>>> default.xml in syslog-ng-3.25.1
>>>>>
>>>>> CAUTION: This email originated from outside of the organization. Do
>>>>> not follow guidance, click links, or open attachments unless you recognize
>>>>> the sender and know the content is safe.
>>>>>
>>>>> Hi Evan,
>>>>>
>>>>> Apologies for the confusion but I did close the rule tag in my
>>>>> default.xml , missed adding it here.
>>>>> The error does not seem to be related to improper closing of tags.
>>>>>
>>>>> This is how it looks :
>>>>>
>>>>> <ruleset id="f582419b3baa42d4a57e42b89704e38c" description="">
>>>>> <pattern>proxysg</pattern>
>>>>> <rules>
>>>>> <rule id="f1e2bfd7bb85402a88d0b732821a0f94">
>>>>> <patterns>
>>>>> <pattern>foo</pattern>
>>>>> </patterns>
>>>>> </rule>
>>>>> <rule id="a681963842014480a83a2a2e38875439">
>>>>> <patterns>
>>>>> <pattern>anything</pattern>
>>>>> </patterns>
>>>>> </rule>
>>>>> </rules>
>>>>> </ruleset>
>>>>> <ruleset id="17fae6edff32a53f9f294ab21240fc2641e7a4db" description="">
>>>>> <pattern>ProxySG</pattern>
>>>>> <pattern>proxysg</pattern>
>>>>> <rules>
>>>>> <rule id="bb169f917216467985cc16e28015f5fa">
>>>>> <patterns>
>>>>> <pattern>bar</pattern>
>>>>> </patterns>
>>>>> </rule>
>>>>> <rule id="94d4a0c324c8-44a88cf3d4640477d35e">
>>>>> <patterns>
>>>>> <pattern>something</pattern>
>>>>> </patterns>
>>>>> </rule>
>>>>> </rules>
>>>>> </ruleset>
>>>>>
>>>>>
>>>>> I am getting the following error message:
>>>>>
>>>>> 2020-02-13T10:47:29.631090] Error parsing pattern database file;
>>>>> filename='/home/nsaboo/abc/default.xml',
>>>>> error='/home/nsaboo/abc/default.xml:17274:22: Joining rulesets with
>>>>> mismatching program name sets, program=proxysg'.
>>>>>
>>>>> My hunch is the issue is related to merging of two rulesets but I am
>>>>> not able to understand why is there mismatching of program names.
>>>>>
>>>>> 1) Can someone please help me understand the issue here ?
>>>>>
>>>>> 2) Is the issue seen because a ruleset has multiple programs in it or
>>>>> is it because the same program 'proxysg' is being used in different
>>>>> rulesets ?
>>>>>
>>>>> 3) From the above snippet of default.xml, what changes can I make
>>>>> into default.xml to avoid the error ?
>>>>>
>>>>> 4) Is there a workaround for this issue ?
>>>>>
>>>>> Thanks,
>>>>> Nitish
>>>>>
>>>>>
>>>>> On Sun, Feb 16, 2020 at 12:40 AM Evan Rempel <erempel at uvic.ca> wrote:
>>>>>
>>>>> I'm not exactly sure what is or is not permitted in the pattern
>>>>> database but I two comments.
>>>>>
>>>>> 1. you need end your rule tag and your rules tag before you start a
>>>>> new ruleset tag.
>>>>>
>>>>> 2. What I do in my pattern database is of the form.
>>>>>
>>>>> <ruleset id="f582419b3baa42d4a57e42b89704e38c" description="">
>>>>> <pattern>proxysg</pattern>
>>>>> <rules>
>>>>> <rule id="f582419b3baa42d4a57e42b89704e38c">
>>>>> <patterns>
>>>>> <pattern>foo</pattern>
>>>>> </patterns>
>>>>> </rule>
>>>>> <rule id="bb169f917216467985cc16e28015f5fa">
>>>>> <patterns>
>>>>> <pattern>bar</pattern>
>>>>> </patterns>
>>>>> </rules>
>>>>> </ruleset>
>>>>>
>>>>>
>>>>> Note:
>>>>> 1. the closing tag of </rule> before a new starting tag of <rule>
>>>>> 2. Multiple "rule" entries inside the the "rules" entry.
>>>>> 3. the closing tag of </rule> before the closing tag of </rules>
>>>>> 4. the closing tag of </rules> before the closing tag of </ruleset>
>>>>>
>>>>> I hope that helps.
>>>>>
>>>>> Evan.
>>>>>
>>>>> On 2/15/20 12:43 AM, Nitish Saboo wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> After debugging further into the issue looks like there was a fix for
>>>>> patterndb rule clash in syslog-ng-3.8 and this is the commit-id
>>>>> '12cd960c8f47260b0b0d4154b096994d66fe345'
>>>>> for the fix. And for this reason I am getting the following error for
>>>>> same default.xml in syslog-ng-3.25.1 version and not in syslog-ng3.6.2 and
>>>>> syslog-ng3.7.1.
>>>>>
>>>>> 2020-02-13T10:47:29.631090] Error parsing pattern database file;
>>>>> filename='/home/nsaboo/abc/default.xml',
>>>>> error='/home/nsaboo/abc/default.xml:17274:22: Joining rulesets with
>>>>> mismatching program name sets, program=proxysg'.
>>>>>
>>>>> Snippet from default.xml
>>>>> ==========================
>>>>>
>>>>> <ruleset id="f582419b3baa42d4a57e42b89704e38c" description="">
>>>>> <pattern>proxysg</pattern>
>>>>> <rules>
>>>>> <rule id="f582419b3baa42d4a57e42b89704e38c">
>>>>> <patterns>
>>>>> <pattern>foo</pattern>
>>>>> </patterns>
>>>>>
>>>>> <ruleset id="8d633c824e844a559088d803464e507a" description="">
>>>>> <pattern>ProxySG</pattern>
>>>>> <pattern>proxysg</pattern>
>>>>> <rules>
>>>>> <rule id="bb169f917216467985cc16e28015f5fa">
>>>>> <patterns>
>>>>> <pattern>bar</pattern>
>>>>> </patterns>
>>>>>
>>>>> I am not able to understand the error message clearly.
>>>>>
>>>>> 1) Can someone please help me understand the issue here ?
>>>>>
>>>>> 2) Is the issue seen because a ruleset has multiple programs in it or
>>>>> is it because the same program 'proxysg' is being used in different
>>>>> rulesets ?
>>>>>
>>>>> 3) From the above snippet of default.xml, what changes can I make
>>>>> into default.xml to avoid the error ?
>>>>>
>>>>> 4) Is there a workaround for this issue ?
>>>>>
>>>>> Thanks,
>>>>> Nitish
>>>>>
>>>>> On Fri, Feb 14, 2020 at 2:40 PM Nitish Saboo <nitish.saboo55 at gmail.com>
>>>>> wrote:
>>>>>
>>>>> Hi Attila,
>>>>>
>>>>> Thanks for your response.
>>>>>
>>>>> And what about the following error:
>>>>>
>>>>> 2020-02-13T10:47:29.631090] Error parsing pattern database file;
>>>>> filename='/home/nsaboo/abc/default.xml',
>>>>> error='/home/nsaboo/abc/default.xml:17274:22: Joining rulesets with
>>>>> mismatching program name sets, program=proxysg'.
>>>>>
>>>>> The same default.xml file was getting loaded correctly in
>>>>> syslog-ng-3.6.2 and syslog-ng-3.7.1 but getting following error while
>>>>> loading same default.xml in syslog-ng3.25.1
>>>>>
>>>>> I came across a similar issue on githib '
>>>>> https://github.com/syslog-ng/syslog-ng/issues/2763
>>>>> <https://nam05.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fsyslog-ng%2Fsyslog-ng%2Fissues%2F2763&data=02%7C01%7CAttila.Szakacs%40oneidentity.com%7C7984ff7f5a4241fdae2208d7b36a9848%7C91c369b51c9e439c989c1867ec606603%7C0%7C0%7C637175142727691628&sdata=WdmKYNdboeB7Y26uuDXhmR225uPqIfX6x4U3Sff2qbY%3D&reserved=0>'
>>>>> .I see the issue is still in open state.Is there a workaround for
>>>>> this issue?
>>>>>
>>>>> Thanks,
>>>>> Nitish
>>>>>
>>>>> On Fri, Feb 14, 2020 at 1:12 PM Attila Szakacs (aszakacs) <
>>>>> Attila.Szakacs at oneidentity.com> wrote:
>>>>>
>>>>> Hi!
>>>>>
>>>>> WARNING: due to a bug in versions before syslog-ng 3.8numeric
>>>>> comparison operators like '!=' in filter expressions were evaluated as
>>>>> string operators. This is fixed in syslog-ng 3.8. As we are operating in
>>>>> compatibility mode, syslog-ng will exhibit the buggy behaviour as previous
>>>>> versions until you bump the @version value in your configuration file;
>>>>>
>>>>> ^^^ This refers to the syslog-ng.conf file version.
>>>>>
>>>>> The correct way to resolve it, and fix the buggy behavior of != and
>>>>> ==, should be to change the != operators between strings to neq in your
>>>>> filters.
>>>>>
>>>>> Regards,
>>>>> Attila
>>>>> ------------------------------
>>>>> *From:* syslog-ng <syslog-ng-bounces at lists.balabit.hu> on behalf of
>>>>> Nitish Saboo <nitish.saboo55 at gmail.com>
>>>>> *Sent:* Thursday, February 13, 2020 12:17 PM
>>>>> *To:* Syslog-ng users' and developers' mailing list <
>>>>> syslog-ng at lists.balabit.hu>
>>>>> *Subject:* [syslog-ng] Warnings and error while loading default.xml
>>>>> in syslog-ng-3.25.1
>>>>>
>>>>> Hi,
>>>>>
>>>>> I am using syslog-ng version 3.25.1.Getting following Warnings and
>>>>> error while initialising syslog-ng engine:
>>>>>
>>>>> [2020-02-13T10:47:29.627899] WARNING: due to a bug in versions before
>>>>> syslog-ng 3.8numeric comparison operators like '!=' in filter expressions
>>>>> were evaluated as string operators. This is fixed in syslog-ng 3.8. As we
>>>>> are operating in compatibility mode, syslog-ng will exhibit the buggy
>>>>> behaviour as previous versions until you bump the @version value in your
>>>>> configuration file;
>>>>> [2020-02-13T10:47:29.627968] WARNING: due to a bug in versions before
>>>>> syslog-ng 3.8numeric comparison operators like '!=' in filter expressions
>>>>> were evaluated as string operators. This is fixed in syslog-ng 3.8. As we
>>>>> are operating in compatibility mode, syslog-ng will exhibit the buggy
>>>>> behaviour as previous versions until you bump the @version value in your
>>>>> configuration file;
>>>>> [2020-02-13T10:47:29.628059] WARNING: due to a bug in versions before
>>>>> syslog-ng 3.8numeric comparison operators like '!=' in filter expressions
>>>>> were evaluated as string operators. This is fixed in syslog-ng 3.8. As we
>>>>> are operating in compatibility mode, syslog-ng will exhibit the buggy
>>>>> behaviour as previous versions until you bump the @version value in your
>>>>> configuration file;
>>>>> [2020-02-13T10:47:29.631090] Error parsing pattern database file;
>>>>> filename='/opt/tap-parsing/patterns/default.xml',
>>>>> error='/opt/tap-parsing/patterns/default.xml:17274:22: Joining rulesets
>>>>> with mismatching program name sets, program=proxysg'
>>>>>
>>>>>
>>>>> 1)For the following warnings, to which version I have to bump up the
>>>>> configuration file ?
>>>>>
>>>>> 2020-02-13T10:47:29.627899] WARNING: due to a bug in versions before
>>>>> syslog-ng 3.8numeric comparison operators like '!=' in filter expressions
>>>>> were evaluated as string operators. This is fixed in syslog-ng 3.8. As we
>>>>> are operating in compatibility mode, syslog-ng will exhibit the buggy
>>>>> behaviour as previous versions until you bump the @version value in your
>>>>> configuration file;
>>>>> [2020-02-13T10:47:29.627968] WARNING: due to a bug in versions before
>>>>> syslog-ng 3.8numeric comparison operators like '!=' in filter expressions
>>>>> were evaluated as string operators. This is fixed in syslog-ng 3.8. As we
>>>>> are operating in compatibility mode, syslog-ng will exhibit the buggy
>>>>> behaviour as previous versions until you bump the @version value in your
>>>>> configuration file;
>>>>> [2020-02-13T10:47:29.628059] WARNING: due to a bug in versions before
>>>>> syslog-ng 3.8numeric comparison operators like '!=' in filter expressions
>>>>> were evaluated as string operators. This is fixed in syslog-ng 3.8. As we
>>>>> are operating in compatibility mode, syslog-ng will exhibit the buggy
>>>>> behaviour as previous versions until you bump the @version value in your
>>>>> configuration file;
>>>>>
>>>>> Currrently the configuration version is the following:
>>>>>
>>>>> configuration = cfg_new(0x0302)
>>>>>
>>>>> Do I have to change it to '0x0319' as defined in 'lib/versioning.h' ?
>>>>>
>>>>> 2)The same default.xml file was getting loaded correctly in
>>>>> syslog-ng-3.6.2 and syslog-ng-3.7.1 but getting following error while
>>>>> loading same default.xml in syslog-ng3.25.1
>>>>>
>>>>>
>>>>> 2020-02-13T10:47:29.631090] Error parsing pattern database file;
>>>>> filename='/home/nsaboo/abc/default.xml',
>>>>> error='/home/nsaboo/abc/default.xml:17274:22: Joining rulesets with
>>>>> mismatching program name sets, program=proxysg'.
>>>>>
>>>>> What can be the reason for this error ?
>>>>>
>>>>>
>>>>> Thanks,
>>>>>
>>>>>
>>>
>>> ______________________________________________________________________________
>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>> Documentation:
>>> http://www.balabit.com/support/documentation/?product=syslog-ng
>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>>
>>>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation:
> http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20200221/e46750c8/attachment-0001.html>
More information about the syslog-ng
mailing list