Problem with program_override in upgrade from 3.0.8 to 3.2.4
Hello there I've been using syslog-ng for a long time, no problems so far, till today ... :p I'm using Open Source Edition, upgrading from 3.0.8 to 3.2.4, installing from the .run file to /opt Today I wanted to update our syslog-ng's to the latest version and found that for some reason, when I override a program via program_override, the PROGRAM macro is empty when I send it to another loghost. program_override seems to be working, as locally writen files show so I did upgrade both syslog-ng, the client and the logserver When I switched back to the old version I found everything began working again Here's the conflicting config in the client: #CLIENT CONFIG source s_apache_access { file("/var/log/apache2/access.log" program_override ("apache_access")); }; destination d_logserver01 { tcp("logserver01"); }; destination d_tmp { file("/var/log/tmp.log" template("$HOST $PROGRAM $MESSAGE\n")); }; log { source(s_apache_error); destination(d_logserver01); destination(d_tmp); flags(final); }; And here's on the server: #LOGSERVER CONFIG filter f_tmp { host(web0[1-2]); }; filter f_apache_error { program(apache_error); }; destination d_apache_error { file("/array/logs/${S_YEAR}/${S_MONTH}/${S_DAY}/${HOST}/apache2/error.log" template("$ISODATE $HOST ${PROGRAM}: $MESSAGE\n") ); }; destination d_tmp { file("/array/logs/${S_YEAR}/${S_MONTH}/${S_DAY}/${HOST}/tmp.log" template("$HOST $PROGRAM $MSG\n")); }; log { source(s_tcp); filter(f_apache_error); destination(d_apache_error); flags(final); }; log { source(s_tcp); filter(f_tmp); destination(d_tmp); flags(final); }; After the update those are the results of tmp.log On client: web01 apache_error Aug 02 20:53:06 2011] [error] [client 192.168.0.5] client denied by server configuration: /var/www/asdadsasdas On server: web01 Aug 02 20:53:06 2011] [error] [client 192.168.50.65] client denied by server configuration: /var/www/asdadsasdas As you can see, the program_name has dissapeared I suspect is a bug, but I'm not sure as there might be some change in syslog-ng behaviour that I did miss :? Anybody could lend me a hand? thanks in advance :) -- Javi Polo Administrador de Sistemas Tel 93 734 97 70 Fax 93 734 97 71 jpolo@wtransnet.com
On Tue, 2011-08-02 at 20:55 +0200, Javi Polo wrote:
Hello there
I've been using syslog-ng for a long time, no problems so far, till today ... :p
I'm using Open Source Edition, upgrading from 3.0.8 to 3.2.4, installing from the .run file to /opt
Today I wanted to update our syslog-ng's to the latest version and found that for some reason, when I override a program via program_override, the PROGRAM macro is empty when I send it to another loghost. program_override seems to be working, as locally writen files show so
I did upgrade both syslog-ng, the client and the logserver When I switched back to the old version I found everything began working again
Here's the conflicting config in the client: #CLIENT CONFIG source s_apache_access { file("/var/log/apache2/access.log" program_override ("apache_access")); };
destination d_logserver01 { tcp("logserver01"); }; destination d_tmp { file("/var/log/tmp.log" template("$HOST $PROGRAM $MESSAGE\n")); };
log { source(s_apache_error); destination(d_logserver01); destination(d_tmp); flags(final); };
And here's on the server: #LOGSERVER CONFIG filter f_tmp { host(web0[1-2]); }; filter f_apache_error { program(apache_error); };
destination d_apache_error { file("/array/logs/${S_YEAR}/${S_MONTH}/${S_DAY}/${HOST}/apache2/error.log" template("$ISODATE $HOST ${PROGRAM}: $MESSAGE\n") ); }; destination d_tmp { file("/array/logs/${S_YEAR}/${S_MONTH}/${S_DAY}/${HOST}/tmp.log" template("$HOST $PROGRAM $MSG\n")); };
log { source(s_tcp); filter(f_apache_error); destination(d_apache_error); flags(final); };
log { source(s_tcp); filter(f_tmp); destination(d_tmp); flags(final); };
After the update those are the results of tmp.log On client: web01 apache_error Aug 02 20:53:06 2011] [error] [client 192.168.0.5] client denied by server configuration: /var/www/asdadsasdas
On server: web01 Aug 02 20:53:06 2011] [error] [client 192.168.50.65] client denied by server configuration: /var/www/asdadsasdas
As you can see, the program_name has dissapeared I suspect is a bug, but I'm not sure as there might be some change in syslog-ng behaviour that I did miss :?
flags(dont-store-legacy-msghdr) I wanted to explain why, but I'm in a hurry, probably in an upcoming post. (the flag might be called no-store-..., can't remember exactly) -- Bazsi
El 08/06/2011 06:56 PM, Balazs Scheidler escribió:
As you can see, the program_name has dissapeared I suspect is a bug, but I'm not sure as there might be some change in syslog-ng behaviour that I did miss :? flags(dont-store-legacy-msghdr)
I wanted to explain why, but I'm in a hurry, probably in an upcoming post. (the flag might be called no-store-..., can't remember exactly)
Thanks! I found an explanation there http://bazsi.blogs.balabit.com/2010/05/small-incompatible-change-for-3-1/ However, it still doesnt work As the flag 'dont-store-legacy-msghdr' is set on the 'source' statement in the logserver host, it seems it doesnt affect the syslog-ng generating the log, which stripes the program from the log-line I tried to put 'dont-store-legacy-msghdr' flag in the originating host, under 'destination' statement, but syslog-ng complains that the flag is invalid there ... I have switched back to syslog-ng 3.0 till I can resolve this issue -- Javi Polo Administrador de Sistemas Tel 93 734 97 70 Fax 93 734 97 71 jpolo@wtransnet.com
On Mon, 2011-08-08 at 17:04 +0200, Javi Polo wrote:
El 08/06/2011 06:56 PM, Balazs Scheidler escribió:
As you can see, the program_name has dissapeared I suspect is a bug, but I'm not sure as there might be some change in syslog-ng behaviour that I did miss :? flags(dont-store-legacy-msghdr)
I wanted to explain why, but I'm in a hurry, probably in an upcoming post. (the flag might be called no-store-..., can't remember exactly)
Thanks! I found an explanation there http://bazsi.blogs.balabit.com/2010/05/small-incompatible-change-for-3-1/
However, it still doesnt work As the flag 'dont-store-legacy-msghdr' is set on the 'source' statement in the logserver host, it seems it doesnt affect the syslog-ng generating the log, which stripes the program from the log-line
I tried to put 'dont-store-legacy-msghdr' flag in the originating host, under 'destination' statement, but syslog-ng complains that the flag is invalid there ...
I have switched back to syslog-ng 3.0 till I can resolve this issue
you need the flag on the host that has program-override(), program-override() will not work if store-legacy-msghdr is enabled, unless the destination template is customized too. If the client syslog-ng has program-override(), then you need the flag on that host, on its source statement. -- Bazsi
On Wed, 2011-08-17 at 11:09 +0200, Balazs Scheidler wrote:
On Mon, 2011-08-08 at 17:04 +0200, Javi Polo wrote:
El 08/06/2011 06:56 PM, Balazs Scheidler escribió:
As you can see, the program_name has dissapeared I suspect is a bug, but I'm not sure as there might be some change in syslog-ng behaviour that I did miss :? flags(dont-store-legacy-msghdr)
I wanted to explain why, but I'm in a hurry, probably in an upcoming post. (the flag might be called no-store-..., can't remember exactly)
Thanks! I found an explanation there http://bazsi.blogs.balabit.com/2010/05/small-incompatible-change-for-3-1/
However, it still doesnt work As the flag 'dont-store-legacy-msghdr' is set on the 'source' statement in the logserver host, it seems it doesnt affect the syslog-ng generating the log, which stripes the program from the log-line
I tried to put 'dont-store-legacy-msghdr' flag in the originating host, under 'destination' statement, but syslog-ng complains that the flag is invalid there ...
I have switched back to syslog-ng 3.0 till I can resolve this issue
you need the flag on the host that has program-override(), program-override() will not work if store-legacy-msghdr is enabled, unless the destination template is customized too.
If the client syslog-ng has program-override(), then you need the flag on that host, on its source statement.
I've pushed this to 3.3, which should be backported to 3.3 one day: commit c76483fcd6c033fdf9b46d1c18753d466362ea88 Author: Balazs Scheidler <bazsi@balabit.hu> Date: Mon Aug 22 09:28:48 2011 +0200 program-override(): disable the effect of store-legacy-msghdr flag During 3.1, the flag "store-legacy-msghdr" has become default, which means that the $PROGRAM[$PID] portion of the original syslog message was restored to the same format it was received as, instead of being reconstructed from the parsed values. However, when specifying program-override(), this behaviour was not restored, e.g. even if the value of the $PROGRAM name-value pair has changed, the originally stored format was used in files. This was certainly not very intuitive. This patch changes that, if $PROGRAM is set in any way (e.g. program-override(), rewrite rule, parser) the effect of store-legacy-msghdr is disabled. That can be restored by using an explict $LEGACY_MSGHDR macro in the destination template. Cc: syslog-ng-stable@balabit.hu Signed-off-by: Balazs Scheidler <bazsi@balabit.hu> -- Bazsi
participants (2)
-
Balazs Scheidler
-
Javi Polo