1.9.3 bug proposal : resolving $HOSTS in destination
We have an issue on syslog-ng 1.9.3 on this following case : when using the destination : destination d_tag {file("/var/log/hosts/$HOST/$FACILITY $YEAR-$MONTH-$DAY.$LEVEL");}; the following error appeared : Error opening file for writing; error='No such file or directory (2)', filename='localhost/var/log/hosts//mail 2005-02-28.notice' The resolved filename looks bugged. Here are the revelant debugging infos : Incoming log entry; line='<21>Feb 28 15:24:46 logger: test' Initializing destination file writer; filename='/var/log/messages', template='/var/log/messages' Initializing destination file writer; filename='localhost/var/log/hosts//mail 2005-02-28.notice', template='/var/log/hosts/$HOST/$FACILITY $YEAR-$MONTH-$DAY.$LEVEL' Error opening file for writing; error='No such file or directory (2)', filename='localhost/var/log/hosts//mail 2005-02-28.notice' This bug produce rapidly a 99% CPU load, mainly because all the sources are redirected to this "destination". syslog-ng 1.6.4 don't produce the bug. Attached is syslog-ng.conf that produce the bug. (ASCII \r\n format, sorry). -- Gilles DEMARTY ExaProtect Technology Tél: +33 (0) 4 26 23 25 25 Fax: +33 (0) 4 26 29 99 09 email(at)exaprotect.com www.exaprotect.com - "Be alert with your alerts" options { sync(0); create_dirs(on); time_reopen (10); log_fifo_size(1000); long_hostnames (off); keep_hostname (yes); use_dns (off); use_fqdn (off); }; source s_sys { pipe ("/proc/kmsg" log_prefix("kernel: ")); unix-stream ("/dev/log"); internal(); }; source src { udp(ip(0.0.0.0)); }; destination d_tag { file("/var/log/hosts/$HOST/$FACILITY $YEAR-$MONTH-$DAY.$LEVEL"); }; destination d_dest {file("/var/log/messages"); }; log { source(s_sys); source(src); destination(d_dest); }; log { source(s_sys); source(src); destination(d_tag); };
On Mon, Feb 28, 2005 at 02:31:00PM +0100, Gilles Demarty wrote:
Here are the revelant debugging infos :
Incoming log entry; line='<21>Feb 28 15:24:46 logger: test'
You haven't said what would you have it do when there's no hostname in the incoming message (as is the case here). Would you have it fall back to using the IP of the remote system? Maybe that's not a bad idea. Anyways, the FAQ states that creating files based on hostnames in syslog messages is bad: http://www.campin.net/syslog-ng/faq.html#bad_filenames ...and I happen to agree with it (oh wait - I wrote that! ;). -- Nate Real programmers never work 9 to 5. If any real programmers are around at 9 am, it's because they were up all night.
Nate Campi wrote:
On Mon, Feb 28, 2005 at 02:31:00PM +0100, Gilles Demarty wrote:
Here are the revelant debugging infos :
Incoming log entry; line='<21>Feb 28 15:24:46 logger: test'
You haven't said what would you have it do when there's no hostname in the incoming message (as is the case here). Would you have it fall back to using the IP of the remote system? Maybe that's not a bad idea.
The issue is not that the hostname is empty( well that's kind of issue too, but not the one I point out ;) ) , but that the resolve of the string is not perfomed correctly : filename='localhost/var/log/hosts//mail 2005-02-28.notice' ^^^^^^^^^ I got the same, if I remove the $HOST/ in the destination string. The resulting filename is : filename='mail/var/log/hosts/ 2005-02-28.notice' ^^^^ I know, the title would have been : "[syslog-ng]1.9.3 bug proposal : resolving $... in destination" instead of $HOST, but well, that's less understanding.
Anyways, the FAQ states that creating files based on hostnames in syslog messages is bad:
http://www.campin.net/syslog-ng/faq.html#bad_filenames
...and I happen to agree with it (oh wait - I wrote that! ;).
-- Gilles DEMARTY ExaProtect Technology Tél: +33 (0) 4 26 23 25 25 Fax: +33 (0) 4 26 29 99 09 email(at)exaprotect.com www.exaprotect.com - "Be alert with your alerts"
On Mon, 2005-02-28 at 15:46 +0100, Gilles Demarty wrote:
Nate Campi wrote:
On Mon, Feb 28, 2005 at 02:31:00PM +0100, Gilles Demarty wrote:
Here are the revelant debugging infos :
Incoming log entry; line='<21>Feb 28 15:24:46 logger: test'
You haven't said what would you have it do when there's no hostname in the incoming message (as is the case here). Would you have it fall back to using the IP of the remote system? Maybe that's not a bad idea.
The issue is not that the hostname is empty( well that's kind of issue too, but not the one I point out ;) ) , but that the resolve of the string is not perfomed correctly : filename='localhost/var/log/hosts//mail 2005-02-28.notice' ^^^^^^^^^
I got the same, if I remove the $HOST/ in the destination string. The resulting filename is : filename='mail/var/log/hosts/ 2005-02-28.notice' ^^^^
I know, the title would have been : "[syslog-ng]1.9.3 bug proposal : resolving $... in destination" instead of $HOST, but well, that's less understanding.
thanks for the report. I'll look into it. -- Bazsi
On Mon, 2005-02-28 at 15:46 +0100, Gilles Demarty wrote:
Nate Campi wrote:
On Mon, Feb 28, 2005 at 02:31:00PM +0100, Gilles Demarty wrote:
Here are the revelant debugging infos :
Incoming log entry; line='<21>Feb 28 15:24:46 logger: test'
You haven't said what would you have it do when there's no hostname in the incoming message (as is the case here). Would you have it fall back to using the IP of the remote system? Maybe that's not a bad idea.
The issue is not that the hostname is empty( well that's kind of issue too, but not the one I point out ;) ) , but that the resolve of the string is not perfomed correctly : filename='localhost/var/log/hosts//mail 2005-02-28.notice' ^^^^^^^^^
I got the same, if I remove the $HOST/ in the destination string. The resulting filename is : filename='mail/var/log/hosts/ 2005-02-28.notice' ^^^^
I know, the title would have been : "[syslog-ng]1.9.3 bug proposal : resolving $... in destination" instead of $HOST, but well, that's less understanding.
The bug is genuine and I've found it thanks for your report, this patch fixes it: --- orig/src/templates.c +++ mod/src/templates.c @@ -55,15 +55,6 @@ { if (*p == '$') { - if (last_macro != M_NONE) - { - e = g_new0(LogTemplateElem, 1); - e->macro = last_macro; - e->text = last_text; - self->compiled_template = g_list_prepend(self->compiled_template, e); - last_macro = M_NONE; - last_text = NULL; - } p++; /* macro reference */ if (*p >= '0' && *p <= '9') @@ -90,7 +81,16 @@ } last_macro = log_macro_lookup(start, p - start); } - + if (last_macro != M_NONE) + { + e = g_new0(LogTemplateElem, 1); + e->macro = last_macro; + e->text = last_text; + self->compiled_template = g_list_prepend(self->compiled_template, e); + last_macro = M_NONE; + last_text = NULL; + } + } else { @@ -122,6 +122,10 @@ for (p = self->compiled_template; p; p = g_list_next(p)) { e = (LogTemplateElem *) p->data; + if (e->text) + { + g_string_append(result, e->text->str); + } if (e->macro != M_NONE) { log_macro_expand(result, e->macro, @@ -130,10 +134,6 @@ (self->flags & LT_TZ_SET) ? self->zone_offset : timezone, lm); } - if (e->text) - { - g_string_append(result, e->text->str); - } } } -- Bazsi
On Mon, 28 Feb 2005 06:29:38 -0800, Nate Campi <nate@campin.net> wrote:
You haven't said what would you have it do when there's no hostname in the incoming message (as is the case here). Would you have it fall back to using the IP of the remote system? Maybe that's not a bad idea.
Anyways, the FAQ states that creating files based on hostnames in syslog messages is bad:
http://www.campin.net/syslog-ng/faq.html#bad_filenames
...and I happen to agree with it (oh wait - I wrote that! ;).
On the subject of log destination filenames by hostname or IP address, I would like to see an internal IP lookup table in syslog-ng, used to substitute for the IP address. Currently I use DNS lookups against a local (tinydns) nameserver, one which does not know about the internet, is only populated with my local log source names and IP reverse DNS. In my opinion it'd be considerably more secure and efficient to keep the IP->name lookup table internally to syslog-ng. Perhaps adding macros for $SRCIP and $SRCHOST expanding to the IP from which the packet was received by syslog-ng, and the hostname from an internal lookup table? Kevin Kadow
Just populate your hosts file. On Tue, 1 Mar 2005 13:57:13 -0600, Kevin <kkadow@gmail.com> wrote:
On Mon, 28 Feb 2005 06:29:38 -0800, Nate Campi <nate@campin.net> wrote:
You haven't said what would you have it do when there's no hostname in the incoming message (as is the case here). Would you have it fall back to using the IP of the remote system? Maybe that's not a bad idea.
Anyways, the FAQ states that creating files based on hostnames in syslog messages is bad:
http://www.campin.net/syslog-ng/faq.html#bad_filenames
...and I happen to agree with it (oh wait - I wrote that! ;).
On the subject of log destination filenames by hostname or IP address, I would like to see an internal IP lookup table in syslog-ng, used to substitute for the IP address.
Currently I use DNS lookups against a local (tinydns) nameserver, one which does not know about the internet, is only populated with my local log source names and IP reverse DNS. In my opinion it'd be considerably more secure and efficient to keep the IP->name lookup table internally to syslog-ng.
Perhaps adding macros for $SRCIP and $SRCHOST expanding to the IP from which the packet was received by syslog-ng, and the hostname from an internal lookup table?
participants (5)
-
Balazs Scheidler
-
Gilles Demarty
-
Jay Guerette
-
Kevin
-
Nate Campi