[syslog-ng] Warnings and error while loading default.xml in syslog-ng-3.25.1
Nitish Saboo
nitish.saboo55 at gmail.com
Fri Feb 21 09:59:40 UTC 2020
Hi Evan, Attila,
I got the following Note from the following link.I have made the line
bold.This is from syslog-ng Adminitrator guide from 3.25 version
https://support.oneidentity.com/technical-documents/syslog-ng-open-source-edition/3.25/administration-guide/77#TOPIC-1349587
NOTE:
If the ${PROGRAM} part of a message is empty, rules with an empty Program
Pattern are used to classify the message.
*If the same Program Pattern is used in multiple rulesets, the rules of
these rulesets are merged, and every rule is used to classify the message.
Note that message patterns must be unique within the merged rulesets, but
the currently only one ruleset is checked for uniqueness.*
If the content of the ${PROGRAM} macro is not the proper name of the
application, you can use the program-template() option to specify it.
1) According to that line( bold one) you should accept multiple program
pattern in different rulesets ..Am i correct? Please correct me if I am
missing something here.
2) If my understanding is correct for [1], then why is my default.xml
failing with the following error while loading:
2020-02-13T10:47:29.631090] Error parsing pattern database file;
filename='/home/nsaboo/abc/default.xml',
error='/home/nsaboo/abc/default.xml:17274:22: Joining rulesets with
mismatching program name sets, program=proxysg'.
Thanks,
Nitish
On Tue, Feb 18, 2020 at 10:47 AM Nitish Saboo <nitish.saboo55 at gmail.com>
wrote:
> Thank you Evan for you response.Looks like changes in .xml file is the
> only option.
>
> Thanks,
> Nitish
>
> On Mon, Feb 17, 2020 at 10:20 PM Evan Rempel <erempel at uvic.ca> wrote:
>
>> Syslog messages have multiple components to the. Date/time, host,
>> program, message and a few others.
>> The important point is that the program and message are independent
>> items. This means that the pattern database must use a two phase matching
>> method, first matching the program and then matching the message.
>>
>> Another way to look at it is that the same message for two different
>> programs means two independent matching rules.
>>
>> Taking this into account you need to write your patterndb differently.
>> Either merging the rulesets (as proposed by aszakacs), or breaking apart
>> the duplicate rule (one rule for two programs) as shown below. Although it
>> may have worked (or appeared to have worked) in syslog-ng 3.6 or 3.7, it
>> was against the patterndb specification.
>>
>> <ruleset id="f582419b3baa42d4a57e42b89704e38c" description="">
>> <pattern>proxysg</pattern>
>> <rules>
>> <rule id="f1e2bfd7bb85402a88d0b732821a0f94">
>> <patterns>
>> <pattern>foo</pattern>
>> </patterns>
>> </rule>
>> <rule id="a681963842014480a83a2a2e38875439">
>> <patterns>
>> <pattern>anything</pattern>
>> </patterns>
>> </rule>
>> <rule id="bb169f917216467985cc16e28015f598">
>> <patterns>
>> <pattern>bar</pattern>
>> </patterns>
>> </rule>
>> <rule id="94d4a0c324c8-44a88cf3d4640477d399">
>> <patterns>
>> <pattern>something</pattern>
>> </patterns>
>> </rule>
>> </rules>
>> </ruleset>
>> <ruleset id="17fae6edff32a53f9f294ab21240fc2641e7a4db" description="">
>> <pattern>ProxySG</pattern>
>> <rules>
>> <rule id="bb169f917216467985cc16e28015f5fa">
>> <patterns>
>> <pattern>bar</pattern>
>> </patterns>
>> </rule>
>> <rule id="94d4a0c324c8-44a88cf3d4640477d35e">
>> <patterns>
>> <pattern>something</pattern>
>> </patterns>
>> </rule>
>> </rules>
>> </ruleset>
>>
>>
>>
>> Both this solution and the one by aszakacs have ONE ruleset for any given
>> program (ruleset pattern).
>>
>> I don't see any other alternatives.
>>
>> Evan.
>>
>>
>> On 2/17/20 1:29 AM, Nitish Saboo wrote:
>>
>> +Syslog-ng users' and developers' mailing list
>> <syslog-ng at lists.balabit.hu>
>>
>> On Mon, Feb 17, 2020 at 2:37 PM Nitish Saboo <nitish.saboo55 at gmail.com>
>> wrote:
>>
>>> Hi Attila,
>>>
>>> 1,2) The issue is that you used "proxysg" in two different rulesets. I
>>> believe syslog-ng could handle this, but we do not handle it right now.
>>> >> But the same default.xml file is working fine in version
>>> syslog-ng-3,6,2 and syslog-ng-3.7.1 where we have same program in different
>>> rulesets.The behavior in version syslog-ng-3,6,2 and syslog-ng-3.7.1 is the
>>> expected behavior or is the current behavior in syslog-ng-3.25.1 version is
>>> having a Bug ?
>>>
>>> 3/4) You can merge the two rulesets, if it is okay for you:
>>> >>Merging the rulesets is not possible at this moment.
>>> Do we have any workaround for this scenario other than merging the
>>> rulesets?
>>> When can we expect a fix for this in near future ?
>>>
>>> Thanks,
>>> Nitish
>>>
>>> On Mon, Feb 17, 2020 at 2:23 PM Attila Szakacs (aszakacs) <
>>> Attila.Szakacs at oneidentity.com> wrote:
>>>
>>>> Hi!
>>>>
>>>> 1,2) The issue is that you used "proxysg" in two different rulesets. I
>>>> believe syslog-ng could handle this, but we do not handle it right now.
>>>> 3/4) You can merge the two rulesets, if it is okay for you:
>>>>
>>>> <?xml version='1.0' encoding='UTF-8'?>
>>>> <patterndb version="4">
>>>> <ruleset id="f582419b3baa42d4a57e42b89704e38c" description="">
>>>> <pattern>proxysg</pattern>
>>>> <pattern>ProxySG</pattern>
>>>> <rules>
>>>> <rule id="f1e2bfd7bb85402a88d0b732821a0f94">
>>>> <patterns>
>>>> <pattern>foo</pattern>
>>>> </patterns>
>>>> </rule>
>>>> <rule id="a681963842014480a83a2a2e38875439">
>>>> <patterns>
>>>> <pattern>anything</pattern>
>>>> </patterns>
>>>> </rule>
>>>> <rule id="bb169f917216467985cc16e28015f5fa">
>>>> <patterns>
>>>> <pattern>bar</pattern>
>>>> </patterns>
>>>> </rule>
>>>> <rule id="94d4a0c324c8-44a88cf3d4640477d35e">
>>>> <patterns>
>>>> <pattern>something</pattern>
>>>> </patterns>
>>>> </rule>
>>>> </rules>
>>>> </ruleset>
>>>> </patterndb>
>>>>
>>>> Regards,
>>>> Attila
>>>>
>>>> ------------------------------
>>>> *From:* syslog-ng <syslog-ng-bounces at lists.balabit.hu> on behalf of
>>>> Nitish Saboo <nitish.saboo55 at gmail.com>
>>>> *Sent:* Monday, February 17, 2020 6:30 AM
>>>> *To:* Syslog-ng users' and developers' mailing list <
>>>> syslog-ng at lists.balabit.hu>
>>>> *Subject:* Re: [syslog-ng] Warnings and error while loading
>>>> default.xml in syslog-ng-3.25.1
>>>>
>>>> CAUTION: This email originated from outside of the organization. Do
>>>> not follow guidance, click links, or open attachments unless you recognize
>>>> the sender and know the content is safe.
>>>>
>>>> Hi Evan,
>>>>
>>>> Apologies for the confusion but I did close the rule tag in my
>>>> default.xml , missed adding it here.
>>>> The error does not seem to be related to improper closing of tags.
>>>>
>>>> This is how it looks :
>>>>
>>>> <ruleset id="f582419b3baa42d4a57e42b89704e38c" description="">
>>>> <pattern>proxysg</pattern>
>>>> <rules>
>>>> <rule id="f1e2bfd7bb85402a88d0b732821a0f94">
>>>> <patterns>
>>>> <pattern>foo</pattern>
>>>> </patterns>
>>>> </rule>
>>>> <rule id="a681963842014480a83a2a2e38875439">
>>>> <patterns>
>>>> <pattern>anything</pattern>
>>>> </patterns>
>>>> </rule>
>>>> </rules>
>>>> </ruleset>
>>>> <ruleset id="17fae6edff32a53f9f294ab21240fc2641e7a4db" description="">
>>>> <pattern>ProxySG</pattern>
>>>> <pattern>proxysg</pattern>
>>>> <rules>
>>>> <rule id="bb169f917216467985cc16e28015f5fa">
>>>> <patterns>
>>>> <pattern>bar</pattern>
>>>> </patterns>
>>>> </rule>
>>>> <rule id="94d4a0c324c8-44a88cf3d4640477d35e">
>>>> <patterns>
>>>> <pattern>something</pattern>
>>>> </patterns>
>>>> </rule>
>>>> </rules>
>>>> </ruleset>
>>>>
>>>>
>>>> I am getting the following error message:
>>>>
>>>> 2020-02-13T10:47:29.631090] Error parsing pattern database file;
>>>> filename='/home/nsaboo/abc/default.xml',
>>>> error='/home/nsaboo/abc/default.xml:17274:22: Joining rulesets with
>>>> mismatching program name sets, program=proxysg'.
>>>>
>>>> My hunch is the issue is related to merging of two rulesets but I am
>>>> not able to understand why is there mismatching of program names.
>>>>
>>>> 1) Can someone please help me understand the issue here ?
>>>>
>>>> 2) Is the issue seen because a ruleset has multiple programs in it or
>>>> is it because the same program 'proxysg' is being used in different
>>>> rulesets ?
>>>>
>>>> 3) From the above snippet of default.xml, what changes can I make into
>>>> default.xml to avoid the error ?
>>>>
>>>> 4) Is there a workaround for this issue ?
>>>>
>>>> Thanks,
>>>> Nitish
>>>>
>>>>
>>>> On Sun, Feb 16, 2020 at 12:40 AM Evan Rempel <erempel at uvic.ca> wrote:
>>>>
>>>> I'm not exactly sure what is or is not permitted in the pattern
>>>> database but I two comments.
>>>>
>>>> 1. you need end your rule tag and your rules tag before you start a new
>>>> ruleset tag.
>>>>
>>>> 2. What I do in my pattern database is of the form.
>>>>
>>>> <ruleset id="f582419b3baa42d4a57e42b89704e38c" description="">
>>>> <pattern>proxysg</pattern>
>>>> <rules>
>>>> <rule id="f582419b3baa42d4a57e42b89704e38c">
>>>> <patterns>
>>>> <pattern>foo</pattern>
>>>> </patterns>
>>>> </rule>
>>>> <rule id="bb169f917216467985cc16e28015f5fa">
>>>> <patterns>
>>>> <pattern>bar</pattern>
>>>> </patterns>
>>>> </rules>
>>>> </ruleset>
>>>>
>>>>
>>>> Note:
>>>> 1. the closing tag of </rule> before a new starting tag of <rule>
>>>> 2. Multiple "rule" entries inside the the "rules" entry.
>>>> 3. the closing tag of </rule> before the closing tag of </rules>
>>>> 4. the closing tag of </rules> before the closing tag of </ruleset>
>>>>
>>>> I hope that helps.
>>>>
>>>> Evan.
>>>>
>>>> On 2/15/20 12:43 AM, Nitish Saboo wrote:
>>>>
>>>> Hi,
>>>>
>>>> After debugging further into the issue looks like there was a fix for
>>>> patterndb rule clash in syslog-ng-3.8 and this is the commit-id
>>>> '12cd960c8f47260b0b0d4154b096994d66fe345'
>>>> for the fix. And for this reason I am getting the following error for
>>>> same default.xml in syslog-ng-3.25.1 version and not in syslog-ng3.6.2 and
>>>> syslog-ng3.7.1.
>>>>
>>>> 2020-02-13T10:47:29.631090] Error parsing pattern database file;
>>>> filename='/home/nsaboo/abc/default.xml',
>>>> error='/home/nsaboo/abc/default.xml:17274:22: Joining rulesets with
>>>> mismatching program name sets, program=proxysg'.
>>>>
>>>> Snippet from default.xml
>>>> ==========================
>>>>
>>>> <ruleset id="f582419b3baa42d4a57e42b89704e38c" description="">
>>>> <pattern>proxysg</pattern>
>>>> <rules>
>>>> <rule id="f582419b3baa42d4a57e42b89704e38c">
>>>> <patterns>
>>>> <pattern>foo</pattern>
>>>> </patterns>
>>>>
>>>> <ruleset id="8d633c824e844a559088d803464e507a" description="">
>>>> <pattern>ProxySG</pattern>
>>>> <pattern>proxysg</pattern>
>>>> <rules>
>>>> <rule id="bb169f917216467985cc16e28015f5fa">
>>>> <patterns>
>>>> <pattern>bar</pattern>
>>>> </patterns>
>>>>
>>>> I am not able to understand the error message clearly.
>>>>
>>>> 1) Can someone please help me understand the issue here ?
>>>>
>>>> 2) Is the issue seen because a ruleset has multiple programs in it or
>>>> is it because the same program 'proxysg' is being used in different
>>>> rulesets ?
>>>>
>>>> 3) From the above snippet of default.xml, what changes can I make into
>>>> default.xml to avoid the error ?
>>>>
>>>> 4) Is there a workaround for this issue ?
>>>>
>>>> Thanks,
>>>> Nitish
>>>>
>>>> On Fri, Feb 14, 2020 at 2:40 PM Nitish Saboo <nitish.saboo55 at gmail.com>
>>>> wrote:
>>>>
>>>> Hi Attila,
>>>>
>>>> Thanks for your response.
>>>>
>>>> And what about the following error:
>>>>
>>>> 2020-02-13T10:47:29.631090] Error parsing pattern database file;
>>>> filename='/home/nsaboo/abc/default.xml',
>>>> error='/home/nsaboo/abc/default.xml:17274:22: Joining rulesets with
>>>> mismatching program name sets, program=proxysg'.
>>>>
>>>> The same default.xml file was getting loaded correctly in
>>>> syslog-ng-3.6.2 and syslog-ng-3.7.1 but getting following error while
>>>> loading same default.xml in syslog-ng3.25.1
>>>>
>>>> I came across a similar issue on githib '
>>>> https://github.com/syslog-ng/syslog-ng/issues/2763
>>>> <https://nam05.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fsyslog-ng%2Fsyslog-ng%2Fissues%2F2763&data=02%7C01%7CAttila.Szakacs%40oneidentity.com%7C7984ff7f5a4241fdae2208d7b36a9848%7C91c369b51c9e439c989c1867ec606603%7C0%7C0%7C637175142727691628&sdata=WdmKYNdboeB7Y26uuDXhmR225uPqIfX6x4U3Sff2qbY%3D&reserved=0>'
>>>> .I see the issue is still in open state.Is there a workaround for
>>>> this issue?
>>>>
>>>> Thanks,
>>>> Nitish
>>>>
>>>> On Fri, Feb 14, 2020 at 1:12 PM Attila Szakacs (aszakacs) <
>>>> Attila.Szakacs at oneidentity.com> wrote:
>>>>
>>>> Hi!
>>>>
>>>> WARNING: due to a bug in versions before syslog-ng 3.8numeric
>>>> comparison operators like '!=' in filter expressions were evaluated as
>>>> string operators. This is fixed in syslog-ng 3.8. As we are operating in
>>>> compatibility mode, syslog-ng will exhibit the buggy behaviour as previous
>>>> versions until you bump the @version value in your configuration file;
>>>>
>>>> ^^^ This refers to the syslog-ng.conf file version.
>>>>
>>>> The correct way to resolve it, and fix the buggy behavior of != and ==,
>>>> should be to change the != operators between strings to neq in your filters.
>>>>
>>>> Regards,
>>>> Attila
>>>> ------------------------------
>>>> *From:* syslog-ng <syslog-ng-bounces at lists.balabit.hu> on behalf of
>>>> Nitish Saboo <nitish.saboo55 at gmail.com>
>>>> *Sent:* Thursday, February 13, 2020 12:17 PM
>>>> *To:* Syslog-ng users' and developers' mailing list <
>>>> syslog-ng at lists.balabit.hu>
>>>> *Subject:* [syslog-ng] Warnings and error while loading default.xml in
>>>> syslog-ng-3.25.1
>>>>
>>>> Hi,
>>>>
>>>> I am using syslog-ng version 3.25.1.Getting following Warnings and
>>>> error while initialising syslog-ng engine:
>>>>
>>>> [2020-02-13T10:47:29.627899] WARNING: due to a bug in versions before
>>>> syslog-ng 3.8numeric comparison operators like '!=' in filter expressions
>>>> were evaluated as string operators. This is fixed in syslog-ng 3.8. As we
>>>> are operating in compatibility mode, syslog-ng will exhibit the buggy
>>>> behaviour as previous versions until you bump the @version value in your
>>>> configuration file;
>>>> [2020-02-13T10:47:29.627968] WARNING: due to a bug in versions before
>>>> syslog-ng 3.8numeric comparison operators like '!=' in filter expressions
>>>> were evaluated as string operators. This is fixed in syslog-ng 3.8. As we
>>>> are operating in compatibility mode, syslog-ng will exhibit the buggy
>>>> behaviour as previous versions until you bump the @version value in your
>>>> configuration file;
>>>> [2020-02-13T10:47:29.628059] WARNING: due to a bug in versions before
>>>> syslog-ng 3.8numeric comparison operators like '!=' in filter expressions
>>>> were evaluated as string operators. This is fixed in syslog-ng 3.8. As we
>>>> are operating in compatibility mode, syslog-ng will exhibit the buggy
>>>> behaviour as previous versions until you bump the @version value in your
>>>> configuration file;
>>>> [2020-02-13T10:47:29.631090] Error parsing pattern database file;
>>>> filename='/opt/tap-parsing/patterns/default.xml',
>>>> error='/opt/tap-parsing/patterns/default.xml:17274:22: Joining rulesets
>>>> with mismatching program name sets, program=proxysg'
>>>>
>>>>
>>>> 1)For the following warnings, to which version I have to bump up the
>>>> configuration file ?
>>>>
>>>> 2020-02-13T10:47:29.627899] WARNING: due to a bug in versions before
>>>> syslog-ng 3.8numeric comparison operators like '!=' in filter expressions
>>>> were evaluated as string operators. This is fixed in syslog-ng 3.8. As we
>>>> are operating in compatibility mode, syslog-ng will exhibit the buggy
>>>> behaviour as previous versions until you bump the @version value in your
>>>> configuration file;
>>>> [2020-02-13T10:47:29.627968] WARNING: due to a bug in versions before
>>>> syslog-ng 3.8numeric comparison operators like '!=' in filter expressions
>>>> were evaluated as string operators. This is fixed in syslog-ng 3.8. As we
>>>> are operating in compatibility mode, syslog-ng will exhibit the buggy
>>>> behaviour as previous versions until you bump the @version value in your
>>>> configuration file;
>>>> [2020-02-13T10:47:29.628059] WARNING: due to a bug in versions before
>>>> syslog-ng 3.8numeric comparison operators like '!=' in filter expressions
>>>> were evaluated as string operators. This is fixed in syslog-ng 3.8. As we
>>>> are operating in compatibility mode, syslog-ng will exhibit the buggy
>>>> behaviour as previous versions until you bump the @version value in your
>>>> configuration file;
>>>>
>>>> Currrently the configuration version is the following:
>>>>
>>>> configuration = cfg_new(0x0302)
>>>>
>>>> Do I have to change it to '0x0319' as defined in 'lib/versioning.h' ?
>>>>
>>>> 2)The same default.xml file was getting loaded correctly in
>>>> syslog-ng-3.6.2 and syslog-ng-3.7.1 but getting following error while
>>>> loading same default.xml in syslog-ng3.25.1
>>>>
>>>>
>>>> 2020-02-13T10:47:29.631090] Error parsing pattern database file;
>>>> filename='/home/nsaboo/abc/default.xml',
>>>> error='/home/nsaboo/abc/default.xml:17274:22: Joining rulesets with
>>>> mismatching program name sets, program=proxysg'.
>>>>
>>>> What can be the reason for this error ?
>>>>
>>>>
>>>> Thanks,
>>>>
>>>>
>>
>> ______________________________________________________________________________
>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>> Documentation:
>> http://www.balabit.com/support/documentation/?product=syslog-ng
>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20200221/b93d0208/attachment-0001.html>
More information about the syslog-ng
mailing list