Re: Re: [syslog-ng]syslog-ng mistreating data as part of the hos tname ?
On Wed, Jan 16, 2002 at 07:45:26PM +0100, Nate Campi wrote:
On Wed, Jan 16, 2002 at 05:14:11PM +0100, Hildenbrand, Patrick wrote:
some more info.
tracing the output of the SSR, the packet does not contain the hostname at the proper place but only the timestamp. So the output looks like (translated into ascii): <174>Jan 13 04:02:12 %ACL_LOG-I-DENY, ACL [280] on "rtfa" UDP 192.168.1.2:4721 -> 14.9.1.3:53
this is the ascii converted hexdump of the package, there is only a single space between date and message.
[ ... ]
Linux syslog sends messages like this:
<123>named[123]: another error from BIND, you should use djbdns my debugging showed the prio beeing only two digits (ex. <23>) but yes and the difference is, there is even no timestamp, why this case is explicitly covered in the RFC, while the case of the cabeltron ssr is not. This is probably also why syslog-ng does not have a problem with this case, it gets logged as: Jan 17 08:11:21 hwdf0006/hwdf0006 PAM_pwdb[22525]: (sshd) session close so syslog-ng correctly adds time and hostname to the log entry. The format of linux (and others) is: <Prio>message The format of the SSR is: <Prio>datetime<space>message The format the RFC suggests is: <Prio>datetime{space}hostname{space}message
It is up to the relay/collector to input the complete header. this is my understanding from the rfc too, but how do I get this using syslog-ng ?
I missed the beggining of this thread, is "%ACL_LOG-I-DENY" getting set as your hostname? I had the same problem with solaris logs when the "TAG" field had a space in it, so syslog-ng (correctly) thinks the first part of the process name (in the "TAG") was the hostname. I wrote a syslog proxy to overcome this, since I can't ask syslog-ng to stop following standards. as said above, there is one space between date and message but according to the standard, there is no standard on how devices do send their messages Perhaps syslog-ng can have a configuration setting where if it receives a certain string in the hostname field, you can set keep-hostname to no for just that message. That would save the day for me, but I don't know how hard it would be to implement that. Well I would vote for getting a setting just the other way around, so for being able to configure something like options( addhostname(<pattern>) which would lead to the hostname being added if the pattern is matched and the pattern then being treated as part of the message. For me this just would be a '^%' as every message of the ssr is prepended with the percent sign ;-)
still do not know how much work it would be but ..... BTW: I do not know how the linux syslog does it, but it does not have this problem. Maybe because '%' is not a valid char for hostnames. linux syslog is a pretty standard syslog I'd guess, though you could argue, wether this is correct according to the standard. Linux syslog displays the message above as: Jan 13 04:02:12 1.2.3.4 %ACL_LOG-I-DENY, ACL [280] on "rtfa" UDP 192.168.1.2:4721 -> 14.9.1.3:53 so it automagically adds the ip address to the message. Again maybe because of the '%' and ',' signs inside the string at the <hostname> position.
-- Nate Campi <http://www.campin.net> GnuPG key: 0xC17AEF79
Kind regards, Patrick Hildenbrand
Patrick Hildenbrand Operations & Technology SAP Hosting AG & Co. KG Raiffeisenring 45 68789 St. Leon-Rot, Germany T +49/6227/7-66410 F +49/6227/7-66301 E patrick.hildenbrand@sap.com http://www.saphosting.com
On Thu, Jan 17, 2002 at 08:48:23AM +0100, Hildenbrand, Patrick wrote:
On Wed, Jan 16, 2002 at 07:45:26PM +0100, Nate Campi wrote:
On Wed, Jan 16, 2002 at 05:14:11PM +0100, Hildenbrand, Patrick wrote:
<123>named[123]: another error from BIND, you should use djbdns
my debugging showed the prio beeing only two digits (ex. <23>) but yes and the difference is,
The PRI is between 0 (one digit) and 192 (3 digits), no leading 0's unless it is the number zero. This variance is normal.
there is even no timestamp, why this case is explicitly covered in the RFC, while the case of the cabeltron ssr is not. This is probably also why syslog-ng does not have a problem with this case, it gets logged as: Jan 17 08:11:21 hwdf0006/hwdf0006 PAM_pwdb[22525]: (sshd) session close so syslog-ng correctly adds time and hostname to the log entry. The format of linux (and others) is: <Prio>message The format of the SSR is: <Prio>datetime<space>message
Solaris does it the same way. No hostname is ever sent, but the rest of the HEADER is sent.
The format the RFC suggests is: <Prio>datetime{space}hostname{space}message
It is up to the relay/collector to input the complete header.
this is my understanding from the rfc too, but how do I get this using syslog-ng ?
You cannot. I had to write a proxy that inserts the hostname before the message. syslog-ng is following the RFC (before it was even written ;), it cannot know whether a string is a hostname or the beginning of the message. It has to assume that if the rest of the HEADER is there, the hostname must be correct.
I missed the beggining of this thread, is "%ACL_LOG-I-DENY" getting set as your hostname? I had the same problem with solaris logs when the "TAG" field had a space in it, so syslog-ng (correctly) thinks the first part of the process name (in the "TAG") was the hostname. I wrote a syslog proxy to overcome this, since I can't ask syslog-ng to stop following standards.
as said above, there is one space between date and message but according to the standard, there is no standard on how devices do send their messages
Perhaps syslog-ng can have a configuration setting where if it receives a certain string in the hostname field, you can set keep-hostname to no for just that message. That would save the day for me, but I don't know how hard it would be to implement that.
Well I would vote for getting a setting just the other way around, so for being able to configure something like options( addhostname(<pattern>) which would lead to the hostname being added if the pattern is matched and the pattern then being treated as part of the message. For me this just would be a '^%' as every message of the ssr is prepended with the percent sign ;-)
D'oh! That's actually what I meant, I sent off the message too fast and didn't feel like replying to my own message with a correction.
BTW: I do not know how the linux syslog does it, but it does not have this problem. Maybe because '%' is not a valid char for hostnames. linux syslog is a pretty standard syslog I'd guess, though you could argue, wether this is correct according to the standard. Linux syslog displays the message above as: Jan 13 04:02:12 1.2.3.4 %ACL_LOG-I-DENY, ACL [280] on "rtfa" UDP 192.168.1.2:4721 -> 14.9.1.3:53 so it automagically adds the ip address to the message. Again maybe because of the '%' and ',' signs inside the string at the <hostname> position.
Balazs, could syslog-ng do the same here please? You haven't reponded to any of my message lately asking for input on feature requests. Maybe I'm so out of line you just don't want to respond :( -- Nate Campi http://www.campin.net GnuPG key: 0xC17AEF79 If you tell them, they never listen. If they listen, they never learn. If they learn, they never remember. If they remember, they never obey.
On Fri, Jan 18, 2002 at 10:35:28AM -0800, Nate Campi wrote:
On Thu, Jan 17, 2002 at 08:48:23AM +0100, Hildenbrand, Patrick wrote:
On Wed, Jan 16, 2002 at 07:45:26PM +0100, Nate Campi wrote:
On Wed, Jan 16, 2002 at 05:14:11PM +0100, Hildenbrand, Patrick wrote:
<123>named[123]: another error from BIND, you should use djbdns
my debugging showed the prio beeing only two digits (ex. <23>) but yes and the difference is,
The PRI is between 0 (one digit) and 192 (3 digits), no leading 0's unless it is the number zero. This variance is normal.
there is even no timestamp, why this case is explicitly covered in the RFC, while the case of the cabeltron ssr is not. This is probably also why syslog-ng does not have a problem with this case, it gets logged as: Jan 17 08:11:21 hwdf0006/hwdf0006 PAM_pwdb[22525]: (sshd) session close so syslog-ng correctly adds time and hostname to the log entry. The format of linux (and others) is: <Prio>message The format of the SSR is: <Prio>datetime<space>message
Solaris does it the same way. No hostname is ever sent, but the rest of the HEADER is sent.
The format the RFC suggests is: <Prio>datetime{space}hostname{space}message
It is up to the relay/collector to input the complete header.
this is my understanding from the rfc too, but how do I get this using syslog-ng ?
You cannot. I had to write a proxy that inserts the hostname before the message. syslog-ng is following the RFC (before it was even written ;), it cannot know whether a string is a hostname or the beginning of the message. It has to assume that if the rest of the HEADER is there, the hostname must be correct.
I missed the beggining of this thread, is "%ACL_LOG-I-DENY" getting set as your hostname? I had the same problem with solaris logs when the "TAG" field had a space in it, so syslog-ng (correctly) thinks the first part of the process name (in the "TAG") was the hostname. I wrote a syslog proxy to overcome this, since I can't ask syslog-ng to stop following standards.
as said above, there is one space between date and message but according to the standard, there is no standard on how devices do send their messages
Perhaps syslog-ng can have a configuration setting where if it receives a certain string in the hostname field, you can set keep-hostname to no for just that message. That would save the day for me, but I don't know how hard it would be to implement that.
Well I would vote for getting a setting just the other way around, so for being able to configure something like options( addhostname(<pattern>) which would lead to the hostname being added if the pattern is matched and the pattern then being treated as part of the message. For me this just would be a '^%' as every message of the ssr is prepended with the percent sign ;-)
D'oh! That's actually what I meant, I sent off the message too fast and didn't feel like replying to my own message with a correction.
BTW: I do not know how the linux syslog does it, but it does not have this problem. Maybe because '%' is not a valid char for hostnames. linux syslog is a pretty standard syslog I'd guess, though you could argue, wether this is correct according to the standard. Linux syslog displays the message above as: Jan 13 04:02:12 1.2.3.4 %ACL_LOG-I-DENY, ACL [280] on "rtfa" UDP 192.168.1.2:4721 -> 14.9.1.3:53 so it automagically adds the ip address to the message. Again maybe because of the '%' and ',' signs inside the string at the <hostname> position.
Balazs, could syslog-ng do the same here please? You haven't reponded to any of my message lately asking for input on feature requests. Maybe I'm so out of line you just don't want to respond :(
It's nothing personal. I'm more -or less following the discussion, but I simply have no time. I'm loaded with Zorp related tasks, and syslog-ng is currently at a low priorirty. The few minutes I spend on syslog-ng is abolsutely in my free time. I have a bugreport about leaking file descriptors in file sources, and thats the first thing I want to fix. Message parsing is as follows: (it's in the file log.c, function parse_log_msg) * parse priority if present (decided based on the first chracter) * skip white space if present * check if we have something that looks like date * if we have date: * skip white space after date * check if we have an AIX "forwarded for" or a "last message repeated" message * if nothing is special, we look for the first word up to ' ' or ':' or '[' * if the first word isended by a space, it is taken as hostname, otherwise as programname * if we don't have date: * the first word up to ' ' or '[' or ':' or '/' or ',' or '<' is taken as programname So the message is above is parsed as follows: date=Jan 13 04:02:12 hostname=%ACL_LOG-I-DENY, msg=ACL [280] on "rtfa" UDP 192.168.1.2:4721 -> 14.9.1.3:53 if keep-hostname is on, syslog-ng will not add a hostname, but will keep %ACL_LOG-I-DENY. if keep-hostname is off, syslog-ng will substitute the hostname (or IP address) of the host it received the message from instead of %ACL_LOG-I-DENY,. chain-hostnames may vary this behaviour a bit. I think an option "missing-hostname" on a per source basis might be useful. Or some new filters which would modify message parts. -- Bazsi PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1
On Sat, Jan 19, 2002 at 10:06:27PM +0100, Balazs Scheidler wrote:
Message parsing is as follows: (it's in the file log.c, function parse_log_msg)
* parse priority if present (decided based on the first chracter) * skip white space if present * check if we have something that looks like date * if we have date: * skip white space after date * check if we have an AIX "forwarded for" or a "last message repeated" message * if nothing is special, we look for the first word up to ' ' or ':' or '[' * if the first word isended by a space, it is taken as hostname, otherwise as programname
Perhaps we could make this: * if the first word is ended by a space *and doesn't match the conf file param bad_hostnames(),* it is taken as hostname, otherwise as programname. *If the hostname matches bad_hostnames(), the result of gethostbyaddr() is inserted as the hostname, with the original value of the hostname field pushed into the message contents.* So bad_hostnames() could be a list of string that are known program names that show up as hostnames in our logs. I have software that sends the prog name like this "ctld 5.0.6:" - so syslog-ng thinks "ctld" is the hostname (solaris system so network syslog packets are transmitted without the hostname field). If I could tell syslog-ng that "ctld" is *not* a hostname, and this would be great. bad_hostnames("ctld"; "ctlds"; "cmd";);
* if we don't have date: * the first word up to ' ' or '[' or ':' or '/' or ',' or '<' is taken as programname
Thanks for all your hard work on syslog-ng, Balazs. -- Nate Campi http://www.campin.net GnuPG key: 0xC17AEF79 "#define QUESTION ((bb) || !(bb))" - Shakespeare
participants (3)
-
Balazs Scheidler
-
Hildenbrand, Patrick
-
Nate Campi