Hey there, We have a number of systems that we're trying to deploy syslog-ng on, with puppet, which means we're dealing with multiple versions out there -- so we're trying to stay compatible with 3.4. On a newer version (3.6), we get the following errors on startup: (snip) [2015-03-06T20:47:33.065077] WARNING: Configuration file format is too old, syslog-ng is running in compatibility mode Please update it to use the syslog-ng 3.6 format at your time of convinience, compatibility mode can operate less efficiently in some cases. To upgrade the configuration, please review the warnings about incompatible changes printed by syslog-ng, and once completed change the @version header at the top of the configuration file.; [2015-03-06T20:47:33.096433] WARNING: Starting with syslog-ng 3.6, the system() source performs JSON parsing of messages starting with the '@cim:' prefix. No additional action is needed; [2015-03-06T20:47:33.096495] WARNING: Your configuration uses a newly introduced reserved word as identifier, please use a different name or enclose it in quotes before upgrading; keyword='channel', config-version='3.3', version='3.4', filename='source confgen system', line='1:1' Error parsing source, source plugin channel not found in source confgen system at line 1, column 1: included from /usr/local/etc/syslog-ng.conf line 27, column 9 channel ^^^^^^^ (snip) I've found if I comment out the "system()" source it makes this error go away, but the requisiste lines are exactly what are referenced in the manual: source s_local { system(); internal(); }; I suppose what's most confusing here, is syslog-ng is trying to tell me exactly where it finds the error, in the word "channel" saying "here, change this, this is the bad word I'm tripping over", but I don't find that word anywhere -- nor does it say *which* included file it's tripping over. Maybe it's telling me that's the word it *needs* to find, but that's less clear. I'm not finding anything in the mailing lists for this. Finally, I should probably ask -- since syslog-ng has the ability to tag your config files with a given version, is there a way to say "use this config if you're on 3.4 and this version if you're on 3.6", so I can stay compatible with the various versions I've got deployed. One other little nit: "Convenience" -Dan Mahoney
"Dan" == Dan Mahoney <dmahoney@isc.org> writes:
Dan> [2015-03-06T20:47:33.096495] WARNING: Your configuration uses a newly Dan> introduced reserved word as identifier, please use a different name or Dan> enclose it in quotes before upgrading; keyword='channel', Dan> config-version='3.3', version='3.4', filename='source confgen system', Dan> line='1:1' Dan> Error parsing source, source plugin channel not found in source confgen Dan> system at line 1, column 1: Dan> included from Dan> /usr/local/etc/syslog-ng.conf line 27, column 9 Dan> channel Dan> ^^^^^^^ That is expanded from system(). If you change your @version from 3.3 to 3.4, the config will work. Alternatively, you can replace system() with something that does not expand to a block using 3.4+ features. With syslog-ng 3.6, comes a system-expand tool (modules/system-source/system-expand) that can help you with that. Dan> Finally, I should probably ask -- since syslog-ng has the ability to tag Dan> your config files with a given version, is there a way to say "use this Dan> config if you're on 3.4 and this version if you're on 3.6", so I can stay Dan> compatible with the various versions I've got deployed. There are a few ways you can do that: 1) Using an environment variable and @include @include "/etc/syslog-ng/`SYSLOG_NG_VERSION`/main.conf" Assuming SYSLOG_NG_VERSION is either 3.4 or 3.6, you'd have your config in /etc/syslog-ng/3.4/main.conf or /etc/syslog-ng/3.6/main.conf. All you have to do, is set the appropriate environment variable. You can teach your init script to do that: export SYSLOG_NG_VERSION="$(syslog-ng -V | head -n 1 | cut -d" " -f2 | cut -d. -f1,2)" 2) Using puppet, a template and @include Instead of an environment variable, you can use a puppet template to put the appropriate version include in the config file. 3) Using puppet, an @include snippet and more @includes Instead of a template, you can also use puppet to copy either a 3.4 config, or a 3.6 config to the right place, depending on host. A variant of this is to copy both, and generate (or copy) a file that has the syslog-ng version. So, if you copy /etc/syslog-ng/conf.d/version.conf, which has: @define sng-version "3.4" And in the main config you do @include "/etc/syslog-ng/`sng-version`/main.conf", that works too. I would likely copy the appropriate config with puppet, that seems to be the cleanest. Though, the first option of using an environment variable is the most flexible, and requires the least amount of maintenance. -- |8]
In addition what Gergely just said, this seems to be a genuine bug in the system() source as it unconditionally uses newer features, even though the version you specified is older than that. Too bad. the code for the system source is pretty inconvinient a fragile to modify, but anyway I wrote an untested patch that should solve this issue. However while trying I noticed it is not enough, SCL is pretty incompatible with early @version tags. With my 3.7 tree, we are using stuff like "template-function" declarations, which is again unsupported in old config versions. If stuff like python blocks appear in SCL include files, then the entire @version stuff would become futile if the config in question uses SCL. I'm starting to think that new keywords shouldn't be considered identifiers even with old config versions, as that's the only thing preventing your configuration to work. In that case if someone used a keyword as an identifier would have a conflict, but that's probably less likely than an inoperable construct in SCL, and I don't want to put too much ifdef mockery into SCL itself. Would you like to try this patch: $ git diff diff --git a/lib/cfg-lexer.c b/lib/cfg-lexer.c index 80d36f2..d0fd30c 100644 --- a/lib/cfg-lexer.c +++ b/lib/cfg-lexer.c @@ -195,7 +195,7 @@ cfg_lexer_lookup_keyword(CfgLexer *self, YYSTYPE *yylval, YYLTYPE *yylloc, const if (token[j] == 0 && keywords[i].kw_name[j] == 0) { /* match */ - if (cfg_is_config_version_older(configuration, keywords[i].kw_req_version)) + if (0 && cfg_is_config_version_older(configuration, keywords[i].kw_req_version)) { msg_warning("WARNING: Your configuration uses a newly introduced reserved word as identifier, please use a different name or enclose it in quotes before upgrading", evt_tag_str("keyword", keywords[i].kw_name), Google may clobber the patch itself, but it simply disables the "too new keyword for this config version" check and lets syslog-ng process any new keywords (such as channel generated by system()). The entire "too new keyword" used was added becauses SLES 11 used "null" as a destination name, which became reserved in 3.1-3.2 timeframe causing syntax errors in established configs. We might just add one such exception for the "null" keyword and be done with the generic check. Any opinions? -- Bazsi On Sat, Mar 7, 2015 at 12:49 AM, Dan Mahoney <dmahoney@isc.org> wrote:
Hey there,
We have a number of systems that we're trying to deploy syslog-ng on, with puppet, which means we're dealing with multiple versions out there -- so we're trying to stay compatible with 3.4.
On a newer version (3.6), we get the following errors on startup:
(snip)
[2015-03-06T20:47:33.065077] WARNING: Configuration file format is too old, syslog-ng is running in compatibility mode Please update it to use the syslog-ng 3.6 format at your time of convinience, compatibility mode can operate less efficiently in some cases. To upgrade the configuration, please review the warnings about incompatible changes printed by syslog-ng, and once completed change the @version header at the top of the configuration file.; [2015-03-06T20:47:33.096433] WARNING: Starting with syslog-ng 3.6, the system() source performs JSON parsing of messages starting with the '@cim:' prefix. No additional action is needed; [2015-03-06T20:47:33.096495] WARNING: Your configuration uses a newly introduced reserved word as identifier, please use a different name or enclose it in quotes before upgrading; keyword='channel', config-version='3.3', version='3.4', filename='source confgen system', line='1:1' Error parsing source, source plugin channel not found in source confgen system at line 1, column 1: included from /usr/local/etc/syslog-ng.conf line 27, column 9
channel ^^^^^^^
(snip)
I've found if I comment out the "system()" source it makes this error go away, but the requisiste lines are exactly what are referenced in the manual:
source s_local { system(); internal(); };
I suppose what's most confusing here, is syslog-ng is trying to tell me exactly where it finds the error, in the word "channel" saying "here, change this, this is the bad word I'm tripping over", but I don't find that word anywhere -- nor does it say *which* included file it's tripping over. Maybe it's telling me that's the word it *needs* to find, but that's less clear.
I'm not finding anything in the mailing lists for this.
Finally, I should probably ask -- since syslog-ng has the ability to tag your config files with a given version, is there a way to say "use this config if you're on 3.4 and this version if you're on 3.6", so I can stay compatible with the various versions I've got deployed.
One other little nit: "Convenience"
-Dan Mahoney
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
Hi, Well... if this is because of only SLES11 then I agree with your idea: we should have only one exception for that "null"... L. On Mon, Mar 9, 2015 at 9:09 AM, Scheidler, Balázs <balazs.scheidler@balabit.com> wrote:
In addition what Gergely just said, this seems to be a genuine bug in the system() source as it unconditionally uses newer features, even though the version you specified is older than that.
Too bad. the code for the system source is pretty inconvinient a fragile to modify, but anyway I wrote an untested patch that should solve this issue. However while trying I noticed it is not enough, SCL is pretty incompatible with early @version tags. With my 3.7 tree, we are using stuff like "template-function" declarations, which is again unsupported in old config versions. If stuff like python blocks appear in SCL include files, then the entire @version stuff would become futile if the config in question uses SCL.
I'm starting to think that new keywords shouldn't be considered identifiers even with old config versions, as that's the only thing preventing your configuration to work. In that case if someone used a keyword as an identifier would have a conflict, but that's probably less likely than an inoperable construct in SCL, and I don't want to put too much ifdef mockery into SCL itself.
Would you like to try this patch:
$ git diff diff --git a/lib/cfg-lexer.c b/lib/cfg-lexer.c index 80d36f2..d0fd30c 100644 --- a/lib/cfg-lexer.c +++ b/lib/cfg-lexer.c @@ -195,7 +195,7 @@ cfg_lexer_lookup_keyword(CfgLexer *self, YYSTYPE *yylval, YYLTYPE *yylloc, const if (token[j] == 0 && keywords[i].kw_name[j] == 0) { /* match */ - if (cfg_is_config_version_older(configuration, keywords[i].kw_req_version)) + if (0 && cfg_is_config_version_older(configuration, keywords[i].kw_req_version)) { msg_warning("WARNING: Your configuration uses a newly introduced reserved word as identifier, please use a different name or enclose it in quotes before upgrading", evt_tag_str("keyword", keywords[i].kw_name),
Google may clobber the patch itself, but it simply disables the "too new keyword for this config version" check and lets syslog-ng process any new keywords (such as channel generated by system()).
The entire "too new keyword" used was added becauses SLES 11 used "null" as a destination name, which became reserved in 3.1-3.2 timeframe causing syntax errors in established configs. We might just add one such exception for the "null" keyword and be done with the generic check.
Any opinions?
-- Bazsi
On Sat, Mar 7, 2015 at 12:49 AM, Dan Mahoney <dmahoney@isc.org> wrote:
Hey there,
We have a number of systems that we're trying to deploy syslog-ng on, with puppet, which means we're dealing with multiple versions out there -- so we're trying to stay compatible with 3.4.
On a newer version (3.6), we get the following errors on startup:
(snip)
[2015-03-06T20:47:33.065077] WARNING: Configuration file format is too old, syslog-ng is running in compatibility mode Please update it to use the syslog-ng 3.6 format at your time of convinience, compatibility mode can operate less efficiently in some cases. To upgrade the configuration, please review the warnings about incompatible changes printed by syslog-ng, and once completed change the @version header at the top of the configuration file.; [2015-03-06T20:47:33.096433] WARNING: Starting with syslog-ng 3.6, the system() source performs JSON parsing of messages starting with the '@cim:' prefix. No additional action is needed; [2015-03-06T20:47:33.096495] WARNING: Your configuration uses a newly introduced reserved word as identifier, please use a different name or enclose it in quotes before upgrading; keyword='channel', config-version='3.3', version='3.4', filename='source confgen system', line='1:1' Error parsing source, source plugin channel not found in source confgen system at line 1, column 1: included from /usr/local/etc/syslog-ng.conf line 27, column 9
channel ^^^^^^^
(snip)
I've found if I comment out the "system()" source it makes this error go away, but the requisiste lines are exactly what are referenced in the manual:
source s_local { system(); internal(); };
I suppose what's most confusing here, is syslog-ng is trying to tell me exactly where it finds the error, in the word "channel" saying "here, change this, this is the bad word I'm tripping over", but I don't find that word anywhere -- nor does it say *which* included file it's tripping over. Maybe it's telling me that's the word it *needs* to find, but that's less clear.
I'm not finding anything in the mailing lists for this.
Finally, I should probably ask -- since syslog-ng has the ability to tag your config files with a given version, is there a way to say "use this config if you're on 3.4 and this version if you're on 3.6", so I can stay compatible with the various versions I've got deployed.
One other little nit: "Convenience"
-Dan Mahoney
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
participants (4)
-
Budai, László
-
Dan Mahoney
-
Gergely Nagy
-
Scheidler, Balázs