Re: [syslog-ng]replacing part of prog name with hostname
I thing you confused it a little... According to my last message (and a similar thread I created recently) The problem with the hostname resolving of Solaris is fixed with using: keep_hostname(no) But, I would really like to understand what's going on in there. Is my assumption correct? Noam --- On Thu 01/02, Balazs Scheidler wrote:From: Balazs Scheidler [mailto: bazsi@balabit.hu]To: syslog-ng@lists.balabit.huDate: Thu, 2 Jan 2003 10:43:45 +0100Subject: Re: [syslog-ng]replacing part of prog name with hostnameOn Tue, Dec 31, 2002 at 02:05:34PM -0800, Nate Campi wrote:> I have syslog-ng 1.5.24 on solaris 8, reading from /etc/.syslog_door and> I have a log entry like this:> > Dec 31 13:48:15 larry 6.0[8704]: [ID 702911 local0.warning] [0] Can't> stat file in FlushFile [news/PointCast]: No such file or directory> > ...but the program name was sent from the app was: "ctlds 6.0[8704]:"> and syslog-ng replaced the first part of the messed up program name with> the host's name. > > In the next version, can syslog-ng "learn" that it don't get a hostname> from solaris ever and that the entire text coming in is actually the log> message? I'm losing information this way. I wonder how many other apps> split up the program name and lose data - most people would never know> as end users.the problem is ctlds sends a space in the program name tag, thus syslog-nginterprets 'ctlds' as hostname and '6.0' as program name. askeep_hostname() is set to no it rewrites originating host name.Try setting keep_hostname() to yes, it will not touch the hostname then.-- BazsiPGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1_______________________________________________syslog-ng maillist - syslog-ng@lists.balabit.huhttps://lists.balabit.hu/mailman/listinfo/syslog-ngFrequently asked questions at http://www.campin.net/syslog-ng/faq.html _______________________________________________ Join Excite! - http://www.excite.com The most personalized portal on the Web!
On Thu, Jan 02, 2003 at 03:28:49PM -0500, Noam Meltzer wrote:
I thing you confused it a little... According to my last message (and a similar thread I created recently) The problem with the hostname resolving of Solaris is fixed with using:
keep_hostname(no)
But, I would really like to understand what's going on in there. Is my assumption correct?
No. syslog-ng parses the incoming message, but the format of messages is _very_ vague. Depending on the sender the message itself can have many form. The problem here was the sender program contains a space, and Solaris syslogd does not add originating hostname to its local messages (unless it relays the message) Thus it is not possible to decide whether the message received contains 'hostname' & 'program' or a single 'program' but with a space in it. keep_hostname() is not a solution, just a workaround, so syslog-ng itself does not rewrite the hostname. The filter expression host('^hostname$') would still use the part before the space (e.g. the program name). The solution is to fix the sender program, no better workaround exists in syslog-ng. Nate, the problem does not apply to local messages only, it happens to cases when Solaris sends these messages via UDP. It is not a solution to simply assume that there is no hostname for local messages -- Bazsi PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1
On Fri, Jan 03, 2003 at 10:48:51AM +0100, Balazs Scheidler wrote:
The solution is to fix the sender program, no better workaround exists in syslog-ng.
I tried that. I spent months trying to explain it to their developers. I almost got one of them to understand, at which point he said they don't really care. *Sigh*, we've paid hundreds of thousand of dollars to them too. I gave up and wrote my own syslog relay daemon to rewrite the messages as I needed.
Nate, the problem does not apply to local messages only, it happens to cases when Solaris sends these messages via UDP. It is not a solution to simply assume that there is no hostname for local messages
I know that it applies to both locally-read messages and ones read from over the network. What I want is for syslog-ng to know that when it's reading from /dev/log or /etc/.syslog_door on solaris to *then* do the behavior I'm speaking of. Over the network it can never know what the remote OS is, but when reading from the socket it certainly knows. -- Nate Campi http://www.campin.net "The real question is not whether machines think but whether men do. The mystery which surrounds a thinking machine already surrounds a thinking man." - B. F. Skinner, Contingencies of Reinforcement
There's one thing I've left out of all this, which is important. Syslog-ng loses the ctld string when logging only locally. It seems to think it's getting a hostname from the local socket, and rewrites it automatically. Solaris syslogd has this: Jan 3 10:57:13 larry cmd 6.0[8704]: [ID 702911 local0.error] [0] MetaTemplate info contains no cache key cmd is another prog that does the same thing (same software suite as ctld). Syslog-ng will do this: Dec 31 13:48:11 larry 6.0[8704]: [ID 702911 local0.warning] [0] Bad request (400): Malformed template area So on the local host, solaris syslogd realized that it's only getting a message, not a hostname and logged (locally) correctly. You see the program has the same pid, even over several days. The only difference is what syslog daemon is running. Now suppose I set "keep_hostnames(no)" on syslog-ng? I'd imagine I'll now get Dec 31 13:48:11 cmd 6.0[8704]: [ID 702911 local0.warning] [0] Bad request (400): Malformed template area ...written to my local logfile. That's not good either - especially when solaris syslogd figured out how to properly write this. Of course when solaris syslogd sends this over the network it'll look like this: <PRI>Dec 31 13:48:11 cmd 6.0[8704]: [ID 702911 local0.warning] [0] Bad request (400): Malformed template area ...and any decent self-respecting syslog daemon will think cmd is the hostname *all over again*. My syslog daemon knows about this and fixes it, but locally I *still* can't run syslog-ng and get filtering or TCP transport because it misinterprets the hostnames where syslogd does not. Does this all make sense now? -- Nate Campi http://www.campin.net "The will to win is not nearly as important as the will to prepare to win." - BOBBY KNIGHT
Nate Campi writes:
On Fri, Jan 03, 2003 at 10:48:51AM +0100, Balazs Scheidler wrote:
The solution is to fix the sender program, no better workaround exists in syslog-ng.
I tried that. I spent months trying to explain it to their developers. I almost got one of them to understand, at which point he said they don't really care.
How about a feature in syslog-ng like this: add_hostname(host1, host2, host3 [, ...]) Which would cause syslog-ng to insert a hostname or IP number for all packets received from the hosts requested. That would cover packets from Solaris hosts received via UDP - the add_hostname() option should also be available on individual syslog sources, so you could correct messages received from the local host via the syslog door or whatever that funky thing is Solaris uses.
On Fri, Jan 03, 2003 at 03:21:25PM -0500, Ed Ravin wrote:
Nate Campi writes:
On Fri, Jan 03, 2003 at 10:48:51AM +0100, Balazs Scheidler wrote:
The solution is to fix the sender program, no better workaround exists in syslog-ng.
I tried that. I spent months trying to explain it to their developers. I almost got one of them to understand, at which point he said they don't really care.
How about a feature in syslog-ng like this:
add_hostname(host1, host2, host3 [, ...])
Which would cause syslog-ng to insert a hostname or IP number for all packets received from the hosts requested. That would cover packets from Solaris hosts received via UDP - the add_hostname() option should also be available on individual syslog sources, so you could correct messages received from the local host via the syslog door or whatever that funky thing is Solaris uses.
Exactly what I suggested, except I called mine "bad_hostnames". I think this feature could work. -- Nate Campi http://www.campin.net "What a good thing Adam had. When he said a good thing, he knew nobody had said it before." - Samuel Clemens
On solaris 2.6 (I build on solaris 2.6 since that's the lowest version I run and therefore binaries built on it tend to work on later versions) I get this after applying the patch: libol-0.3.6/src -D_GNU_SOURCE -c affile.c affile.c: In function `do_init_affile_source': affile.c:165: structure has no member named `bad_hostname' affile.c:165: too many arguments to function `make_log_reader' make[3]: *** [affile.o] Error 1 make[3]: Leaving directory `/usr/share/src/syslog-ng-1.5.24/src' make[2]: *** [all-recursive] Error 1 make[2]: Leaving directory `/usr/share/src/syslog-ng-1.5.24/src' make[1]: *** [all-recursive-am] Error 2 make[1]: Leaving directory `/usr/share/src/syslog-ng-1.5.24/src' make: *** [all-recursive] Error 1 syslog-ng 1.5.24 and libol 0.3.6, gcc version 2.95. Any ideas? -- Nate Campi http://www.campin.net "E-mail - When it absolutely, positively has to get lost at the speed of light." - Jerry Vuoso.
On Mon, Jan 06, 2003 at 09:13:24AM -0800, Nate Campi wrote:
On solaris 2.6 (I build on solaris 2.6 since that's the lowest version I run and therefore binaries built on it tend to work on later versions) I get this after applying the patch:
libol-0.3.6/src -D_GNU_SOURCE -c affile.c affile.c: In function `do_init_affile_source': affile.c:165: structure has no member named `bad_hostname' affile.c:165: too many arguments to function `make_log_reader' make[3]: *** [affile.o] Error 1 make[3]: Leaving directory `/usr/share/src/syslog-ng-1.5.24/src' make[2]: *** [all-recursive] Error 1 make[2]: Leaving directory `/usr/share/src/syslog-ng-1.5.24/src' make[1]: *** [all-recursive-am] Error 2 make[1]: Leaving directory `/usr/share/src/syslog-ng-1.5.24/src' make: *** [all-recursive] Error 1
syslog-ng 1.5.24 and libol 0.3.6, gcc version 2.95. Any ideas?
this time you need to regenerate .x files as I have changed class definitions in my patch. I can also send you a patched .tar.gz in private if you want. -- Bazsi PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1
On Mon, Jan 06, 2003 at 07:00:28PM +0100, Balazs Scheidler wrote:
On Mon, Jan 06, 2003 at 09:13:24AM -0800, Nate Campi wrote:
this time you need to regenerate .x files as I have changed class definitions in my patch. I can also send you a patched .tar.gz in private if you want.
Hmm, that would be cool, since I don't want to bother with scheme for a one-time shot. Thanks in advance. -- Nate Campi http://www.campin.net There are two ways to write error-free programs. Only the third one works. -Anon.
On Mon, Jan 06, 2003 at 07:00:28PM +0100, Balazs Scheidler wrote:
On Mon, Jan 06, 2003 at 09:13:24AM -0800, Nate Campi wrote:
On solaris 2.6 (I build on solaris 2.6 since that's the lowest version I run and therefore binaries built on it tend to work on later versions) I get this after applying the patch:
libol-0.3.6/src -D_GNU_SOURCE -c affile.c affile.c: In function `do_init_affile_source': affile.c:165: structure has no member named `bad_hostname' affile.c:165: too many arguments to function `make_log_reader' make[3]: *** [affile.o] Error 1 make[3]: Leaving directory `/usr/share/src/syslog-ng-1.5.24/src' make[2]: *** [all-recursive] Error 1 make[2]: Leaving directory `/usr/share/src/syslog-ng-1.5.24/src' make[1]: *** [all-recursive-am] Error 2 make[1]: Leaving directory `/usr/share/src/syslog-ng-1.5.24/src' make: *** [all-recursive] Error 1
syslog-ng 1.5.24 and libol 0.3.6, gcc version 2.95. Any ideas?
this time you need to regenerate .x files as I have changed class definitions in my patch. I can also send you a patched .tar.gz in private if you want.
I installed scsh on this host, and it gets further, but doesn't compile. make[1]: Entering directory `/usr/share/src/syslog-ng-1.5.24/src' make all-recursive make[2]: Entering directory `/usr/share/src/syslog-ng-1.5.24/src' Making all in . make[3]: Entering directory `/usr/share/src/syslog-ng-1.5.24/src' gcc -DHAVE_CONFIG_H -I. -I/usr/share/src/syslog-ng-1.5.24/src -I. -g -O2 -Wall -I/usr/share/src/libol-0.3.6/src -D_GNU_SOURCE -c afstreams.c afstreams.c: In function `io_stream_get': afstreams.c:175: `self' undeclared (first use in this function) afstreams.c:175: (Each undeclared identifier is reported only once afstreams.c:175: for each function it appears in.) make[3]: *** [afstreams.o] Error 1 make[3]: Leaving directory `/usr/share/src/syslog-ng-1.5.24/src' make[2]: *** [all-recursive] Error 1 make[2]: Leaving directory `/usr/share/src/syslog-ng-1.5.24/src' make[1]: *** [all-recursive-am] Error 2 make[1]: Leaving directory `/usr/share/src/syslog-ng-1.5.24/src' make: *** [all-recursive] Error 1 The offending line is "regcomp(&self->bad_hostname, "^$", REG_NOSUB | REG_EXTENDED);" in this exerpt: { NEW(stream_fd, f); init_file(backend, &f->super, fd, NULL); f->super.prepare = prepare_stream_fd; f->super.read = stream_read_callback; f->super.want_read = 1; f->pipe = pipe; if (hostname_re == NULL) regcomp(&self->bad_hostname, "^$", REG_NOSUB | REG_EXTENDED); else regcomp(&self->bad_hostname, hostname_re, REG_NOSUB | REG_EXTENDED); return &f->super; } What should I do now? TIA -- Nate Campi http://www.campin.net ignorami: n: The BOFH art of folding problem lusers into representational shapes.
On Mon, Jan 06, 2003 at 04:03:25PM -0800, Nate Campi wrote:
On Mon, Jan 06, 2003 at 07:00:28PM +0100, Balazs Scheidler wrote:
On Mon, Jan 06, 2003 at 09:13:24AM -0800, Nate Campi wrote:
On solaris 2.6 (I build on solaris 2.6 since that's the lowest version I run and therefore binaries built on it tend to work on later versions) I get this after applying the patch:
libol-0.3.6/src -D_GNU_SOURCE -c affile.c affile.c: In function `do_init_affile_source': affile.c:165: structure has no member named `bad_hostname' affile.c:165: too many arguments to function `make_log_reader' make[3]: *** [affile.o] Error 1 make[3]: Leaving directory `/usr/share/src/syslog-ng-1.5.24/src' make[2]: *** [all-recursive] Error 1 make[2]: Leaving directory `/usr/share/src/syslog-ng-1.5.24/src' make[1]: *** [all-recursive-am] Error 2 make[1]: Leaving directory `/usr/share/src/syslog-ng-1.5.24/src' make: *** [all-recursive] Error 1
syslog-ng 1.5.24 and libol 0.3.6, gcc version 2.95. Any ideas?
this time you need to regenerate .x files as I have changed class definitions in my patch. I can also send you a patched .tar.gz in private if you want.
I installed scsh on this host, and it gets further, but doesn't compile.
make[1]: Entering directory `/usr/share/src/syslog-ng-1.5.24/src' make all-recursive make[2]: Entering directory `/usr/share/src/syslog-ng-1.5.24/src' Making all in . make[3]: Entering directory `/usr/share/src/syslog-ng-1.5.24/src' gcc -DHAVE_CONFIG_H -I. -I/usr/share/src/syslog-ng-1.5.24/src -I. -g -O2 -Wall -I/usr/share/src/libol-0.3.6/src -D_GNU_SOURCE -c afstreams.c afstreams.c: In function `io_stream_get': afstreams.c:175: `self' undeclared (first use in this function) afstreams.c:175: (Each undeclared identifier is reported only once afstreams.c:175: for each function it appears in.) make[3]: *** [afstreams.o] Error 1 make[3]: Leaving directory `/usr/share/src/syslog-ng-1.5.24/src' make[2]: *** [all-recursive] Error 1 make[2]: Leaving directory `/usr/share/src/syslog-ng-1.5.24/src' make[1]: *** [all-recursive-am] Error 2 make[1]: Leaving directory `/usr/share/src/syslog-ng-1.5.24/src' make: *** [all-recursive] Error 1
The offending line is "regcomp(&self->bad_hostname, "^$", REG_NOSUB | REG_EXTENDED);" in this exerpt:
{ NEW(stream_fd, f);
init_file(backend, &f->super, fd, NULL); f->super.prepare = prepare_stream_fd; f->super.read = stream_read_callback; f->super.want_read = 1; f->pipe = pipe; if (hostname_re == NULL) regcomp(&self->bad_hostname, "^$", REG_NOSUB | REG_EXTENDED); else regcomp(&self->bad_hostname, hostname_re, REG_NOSUB | REG_EXTENDED);
return &f->super; }
change 'self' to 'f'. -- Bazsi PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1
participants (4)
-
Balazs Scheidler
-
Ed Ravin
-
Nate Campi
-
Noam Meltzer