Bug: Numeric digits stripped from source hostname and time mangled.
[Apologies if this has been reported before, but I couldn't find any mention of it. Also, my apologies if this shows up twice. I sent it yesterday, but after 24 hours, it hasn't hit the list yet, so I'm resending.] It appears that Syslog-NG 2.0.x (tested with 2.0.9) will mangle originating source hostname when it begins numerically hostnames, and this also causes errors with date/time handling. Verified with Syslog-NG 2.0.9 on Red Hat Enterprise Linux 3 To duplicate and display the bug (destination(d_log_expanded) and template added to more easily display what's going on): [root@logbox syslog-ng]# cat syslog-ng.conf options { sync(0); chain_hostnames(yes); }; source s_net { tcp(); }; destination d_log { file("/tmp/test.log") ; }; destination d_log_expanded { file("/tmp/test-long.log" template("Date: $DATE, Sender Unixtime: $S_UNIXTIME, Receiver Unixtime: $R_UNIXTIME, Host: $FULLHOST Message: $MSGONLY\n") template_escape(no)) ; }; log { source(s_net); destination(d_log); }; log { source(s_net); destination(d_log_expanded); }; [root@logbox tmp]# cat /tmp/problem-log <0d>Mar 31 04:41:57 1234-xxxx/192.168.1.1 TestLog: This is a test log that includes the relevant bits of the original log, before Syslog-NG mangles them: "Mar 31 04:41:57 1234-xxxx/192.168.1.1 TestLog: This is a test log. . ." [root@logbox tmp]# cat < /tmp/problem-log | nc localhost 514 [root@logbox tmp]# tail -1 *.log ==> test.log <== Dec 31 17:59:59 -xxxx/192.168.1.1/localhost TestLog: This is a test log that includes the relevant bits of the original log, before Syslog-NG mangles them: "Mar 31 04:41:57 1234-xxxx/192.168.1.1 TestLog: This is a test log. . ." ==> test-long.log <== Date: Dec 31 17:59:59, Sender Unixtime: -1, Receiver Unixtime: 1207676046, Host: -xxxx/192.168.1.1/localhost Message: This is a test log that includes the relevant bits of the original log, before Syslog-NG mangles them: "Mar 31 04:41:57 1234-xxxx/192.168.1.1 TestLog: This is a test log. . ." You can see from the above that the '1234' is being stripped from the originating hostname '1234-xxxx', and the sender timestamp is then being mangled to -1. It appears to only pull the first 4 numbers from the source hostname (assuming there are more numbers, such as '123456-xxxx'), and it only happens when the hostname starts numerically. If you add a non-numeric character to the beginning part of the hostname, it is handled correctly (such as 'z1234-xxxx' or '12z34-xxxx'). -- Christopher Cashell ____________________________________ Christopher Cashell Systems & Network Security Engineer phone: 402.361.3065 fax: 402.361.3165 e-mail: christophercashell@solutionary.com Solutionary, Inc. www.Solutionary.com Making Security Manageable _____________________________________ Confidentiality Notice The content of this communication, along with any attachments, is covered by federal and state law governing electronic communications and may contain confidential and legally privileged information. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution, use or copying of the information contained herein is strictly prohibited. If you have received this communication in error, please immediately contact us by telephone at (402) 361-3000 or e-mail security@solutionary.com. Thank you. Copyright 2000-2007, Solutionary, Inc. All rights reserved. ActiveGuard, eV3, Solutionary and the Solutionary logo are registered marks of Solutionary, Inc. SecurCompass is a service mark of Solutionary, Inc.
On Wed, 2008-04-09 at 10:58 -0500, Christopher Cashell wrote:
[Apologies if this has been reported before, but I couldn't find any mention of it. Also, my apologies if this shows up twice. I sent it yesterday, but after 24 hours, it hasn't hit the list yet, so I'm resending.]
It appears that Syslog-NG 2.0.x (tested with 2.0.9) will mangle originating source hostname when it begins numerically hostnames, and this also causes errors with date/time handling.
Verified with Syslog-NG 2.0.9 on Red Hat Enterprise Linux 3
To duplicate and display the bug (destination(d_log_expanded) and template added to more easily display what's going on):
[root@logbox syslog-ng]# cat syslog-ng.conf options { sync(0); chain_hostnames(yes); };
source s_net { tcp(); }; destination d_log { file("/tmp/test.log") ; }; destination d_log_expanded { file("/tmp/test-long.log" template("Date: $DATE, Sender Unixtime: $S_UNIXTIME, Receiver Unixtime: $R_UNIXTIME, Host: $FULLHOST Message: $MSGONLY\n") template_escape(no)) ; };
log { source(s_net); destination(d_log); }; log { source(s_net); destination(d_log_expanded); };
[root@logbox tmp]# cat /tmp/problem-log <0d>Mar 31 04:41:57 1234-xxxx/192.168.1.1 TestLog: This is a test log that includes the relevant bits of the original log, before Syslog-NG mangles them: "Mar 31 04:41:57 1234-xxxx/192.168.1.1 TestLog: This is a test log. . ."
[root@logbox tmp]# cat < /tmp/problem-log | nc localhost 514 [root@logbox tmp]# tail -1 *.log ==> test.log <== Dec 31 17:59:59 -xxxx/192.168.1.1/localhost TestLog: This is a test log that includes the relevant bits of the original log, before Syslog-NG mangles them: "Mar 31 04:41:57 1234-xxxx/192.168.1.1 TestLog: This is a test log. . ."
==> test-long.log <== Date: Dec 31 17:59:59, Sender Unixtime: -1, Receiver Unixtime: 1207676046, Host: -xxxx/192.168.1.1/localhost Message: This is a test log that includes the relevant bits of the original log, before Syslog-NG mangles them: "Mar 31 04:41:57 1234-xxxx/192.168.1.1 TestLog: This is a test log. . ."
You can see from the above that the '1234' is being stripped from the originating hostname '1234-xxxx', and the sender timestamp is then being mangled to -1. It appears to only pull the first 4 numbers from the source hostname (assuming there are more numbers, such as '123456-xxxx'), and it only happens when the hostname starts numerically. If you add a non-numeric character to the beginning part of the hostname, it is handled correctly (such as 'z1234-xxxx' or '12z34-xxxx').
So much about heuristics, the reason for this is that syslog-ng tries to parse multiple date/time formats and one of them has a year right after the original BSD timestamp. So syslog-ng assumes that the message given is from year 1234, which cannot be represented by unix timestamps (range 1970 - 2038), and that's the reason for the garbled timestamp. Can you check if this patch fixes it: commit 476a72e1d927d3404e22866bd9310c6013938d1b Author: Balazs Scheidler <bazsi@balabit.hu> Date: Wed Apr 9 21:30:50 2008 +0200 made LinkSys timestamp parsing stricter Only take the year value as year if the timestamp is terminated with a space. diff --git a/src/logmsg.c b/src/logmsg.c index 3861f40..9310846 100644 --- a/src/logmsg.c +++ b/src/logmsg.c @@ -296,8 +296,8 @@ log_msg_parse(LogMessage *self, gchar *data, gint length, guint flags, regex_t * self->stamp.time.tv_usec = 0; } - else if (left >= 20 && src[3] == ' ' && src[6] == ' ' && src[9] == ':' && src[12] == ':' && src[15] == ' ' && - isdigit(src[16]) && isdigit(src[17]) && isdigit(src[18]) && isdigit(src[19])) + else if (left >= 21 && src[3] == ' ' && src[6] == ' ' && src[9] == ':' && src[12] == ':' && src[15] == ' ' && + isdigit(src[16]) && isdigit(src[17]) && isdigit(src[18]) && isdigit(src[19]) && isspace(src[20])) { /* LinkSys timestamp, expected format: MMM DD HH:MM:SS YYYY */ -- Bazsi
Balazs Scheidler did thus speak on 4/9/2008 2:31 PM:
So much about heuristics, the reason for this is that syslog-ng tries to parse multiple date/time formats and one of them has a year right after the original BSD timestamp.
So syslog-ng assumes that the message given is from year 1234, which cannot be represented by unix timestamps (range 1970 - 2038), and that's the reason for the garbled timestamp.
Can you check if this patch fixes it:
commit 476a72e1d927d3404e22866bd9310c6013938d1b Author: Balazs Scheidler <bazsi@balabit.hu> Date: Wed Apr 9 21:30:50 2008 +0200
[Snip: Patch] I have applied the patch and verified that all of the previously failing test cases I had are now working correctly. Thank you! -- Christopher Cashell
Christopher Cashell wrote:
[Apologies if this has been reported before, but I couldn't find any mention of it. Also, my apologies if this shows up twice. I sent it yesterday, but after 24 hours, it hasn't hit the list yet, so I'm resending.]
It appears that Syslog-NG 2.0.x (tested with 2.0.9) will mangle originating source hostname when it begins numerically hostnames, and this also causes errors with date/time handling.
The RFC specifies that host names can not start with a digit. If you have control over these host names, you should change them. If not, then you have to rely on applications handling invalid data, and thankfully syslog-ng is fairly good at accommodating vendors shortcomings. Evan.
Evan Rempel did thus speak on 4/9/2008 2:39 PM:
Christopher Cashell wrote:
It appears that Syslog-NG 2.0.x (tested with 2.0.9) will mangle originating source hostname when it begins numerically hostnames, and this also causes errors with date/time handling.
The RFC specifies that host names can not start with a digit. If you have control over these host names, you should change them. If not, then you have to rely on applications handling invalid data, and thankfully syslog-ng is fairly good at accommodating vendors shortcomings.
That was actually amended in a later RFC (RFC1123): 2. GENERAL ISSUES This section contains general requirements that may be applicable to all application-layer protocols. 2.1 Host Names and Numbers The syntax of a legal Internet host name was specified in RFC-952 [DNS:4]. One aspect of host name syntax is hereby changed: the restriction on the first character is relaxed to allow either a letter or a digit. Host software MUST support this more liberal syntax. Host software MUST handle host names of up to 63 characters and SHOULD handle host names of up to 255 characters. Whenever a user inputs the identity of an Internet host, it SHOULD be possible to enter either (1) a host domain name or (2) an IP address in dotted-decimal ("#.#.#.#") form. The host SHOULD check the string syntactically for a dotted-decimal number before looking it up in the Domain Name System. Personally, I still think it's a good idea to avoid them, but in this case, it wasn't my call, and they are legal hostnames.
Evan.
-- Christopher Cashell
participants (3)
-
Balazs Scheidler
-
Christopher Cashell
-
Evan Rempel