RE: [syslog-ng] problems with Cisco WiFi controller syslog messages
Bazsi, So I finally got around to the upgrade this morning (now on 2.0.4), only one month later... Is it possible you can help with these obviously non-standard Cisco syslog timestamps? JDP --------------------------------- Jason D Poley Network Tech GS ITS Network County of Santa Barbara 805.568.2680 jpoley@co.santa-barbara.ca.us -----Original Message----- From: Poley, Jason Sent: Thursday, June 14, 2007 7:57 AM To: 'Syslog-ng users' and developers' mailing list' Subject: RE: [syslog-ng] problems with Cisco WiFi controller syslog messages Thanks so much for the reply. So, I will upgrade to the latest version and hope for a fix from you I suppose. A question along those lines then... What is the implication of upgrading from 1.6.9 to 2.0.x? Are there any problems or changes that will affect my current logs? I suppose I should mention that I dump these to a mysql database and report against them with php-syslog-ng. I sure don't want to blow up the whole system. JDP --------------------------------- Jason D Poley Network Tech GS ITS Network County of Santa Barbara 805.568.2680 jpoley@co.santa-barbara.ca.us
-----Original Message----- From: syslog-ng-bounces@lists.balabit.hu [mailto:syslog-ng- bounces@lists.balabit.hu] On Behalf Of Balazs Scheidler Sent: Thursday, June 14, 2007 3:43 AM To: Syslog-ng users' and developers' mailing list Subject: Re: [syslog-ng] problems with Cisco WiFi controller syslog messages
On Wed, 2007-06-13 at 07:02 -0700, Poley, Jason wrote:
We have upgraded our Cisco WiFi controller and now its syslog messages contain milliseconds which syslog-ng does not know how to handle.
I am running version 1.6.9 of syslog-ng on RedHat version 3.
TCP dump of first 96 bytes... 06:57:07.584716 IP (tos 0x0, ttl 59, id 0, offset 0, flags [DF], proto 17, length: 248) 161.213.8.243.32768 > 161.213.4.226.syslog: UDP, length 220 0x0000: 4500 00f8 0000 4000 3b11 ed75 a1d5 08f3 E.....@.;..u.... 0x0010: a1d5 04e2 8000 0202 00e4 660c 3c31 3238 ..........f.<128 0x0020: 3e20 4a75 6e20 3133 2030 363a 3536 3a31 .Jun.13.06:56:1 0x0030: 362e 3732 3820 6170 665f 726f 6775 655f 6.728.apf_rogue_ 0x0040: 6465 7465 6374 2e63 3a35 3735 2041 5046 detect.c:575.APF 0x0050: 2d31 -1
Is this behavior different in a later version of syslog-ng and should I upgrade?
syslog-ng 2.0.x supports milliseconds in timestamps, however it uses ISO8601 timestamps for that purpose. As I see the snipped quoted here uses a BSD timestamps with milliseconds added.
Gee.. At least they could have added year information too.
So, upgrading to 2.0.x will not solve your problems, but there's a chance that I can change this there.
-- Bazsi
_______________________________________________ syslog-ng maillist - syslog-ng@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html
On Thu, 2007-07-19 at 11:09 -0700, Poley, Jason wrote:
Bazsi,
So I finally got around to the upgrade this morning (now on 2.0.4), only one month later...
Is it possible you can help with these obviously non-standard Cisco syslog timestamps?
The attached patch should do it. Please report back if it indeed worked, I could only test it with a small unit test program. While doing this I've also found a minor issue with fraction support on ISO timestamps, an integer overflow could have caused an invalid value, if second fraction was specified with more than 4 digits, because the formula: usec = frac * 1000000 / div could overflow, whenever frac is over 4294 (2^32/10e6). After using proper parentheses, the issue is gone. diff --git a/src/logmsg.c b/src/logmsg.c index 4ea6fb0..2ab8ae1 100644 --- a/src/logmsg.c +++ b/src/logmsg.c @@ -229,13 +229,13 @@ log_msg_parse(LogMessage *self, gchar *data, gint length, guint flags, regex_t * /* process second fractions */ p++; - while (isdigit(*p)) + while (div < 10e6 && isdigit(*p)) { frac = 10 * frac + (*p) - '0'; div = div * 10; p++; } - self->stamp.time.tv_usec = frac * 1000000 / div; + self->stamp.time.tv_usec = frac * (1000000 / div); } if (p && (*p == '+' || *p == '-') && strlen(p) == 6 && isdigit(*(p+1)) && isdigit(*(p+2)) && *(p+3) == ':' && isdigit(*(p+4)) && isdigit(*(p+5))) @@ -298,14 +298,35 @@ log_msg_parse(LogMessage *self, gchar *data, gint length, guint flags, regex_t * { /* RFC 3164 timestamp, expected format: MMM DD HH:MM:SS ... */ struct tm tm, *nowtm; + glong usec = 0; /* Just read the buffer data into a textual datestamp. */ + g_string_assign_len(&self->date, src, 15); src += 15; left -= 15; + if (left > 0 && src[0] == '.') + { + gulong frac = 0; + gint div = 1; + gint i = 1; + + /* gee, funny Cisco extension, BSD timestamp with fraction of second support */ + + while (i < left && div < 10e6 && isdigit(src[i])) + { + frac = 10 * frac + (src[i]) - '0'; + div = div * 10; + i++; + } + usec = frac * (1000000 / div); + left -= i; + src += i; + } + /* And also make struct time timestamp for the msg */ nowtm = localtime(&now); @@ -318,7 +339,7 @@ log_msg_parse(LogMessage *self, gchar *data, gint length, guint flags, regex_t * /* NOTE: no timezone information in the message, assume it is local time */ self->stamp.time.tv_sec = mktime(&tm); - self->stamp.time.tv_usec = 0; + self->stamp.time.tv_usec = usec; self->stamp.zone_offset = get_local_timezone_ofs(self->stamp.time.tv_sec); /* assume local timezone */ } -- Bazsi
Well, I seem to be doing something wrong with patch... I created a patch file from the enclosed update, starting from the 'diff' line down to the end (no including your sig of course). I then run patch and see the following output... -------------<paste>----------- [jpoley@ITS-syslog src]$ patch -p2 -b -i patchfile patching file logmsg.c Hunk #1 succeeded at 229 with fuzz 1. missing header for unified diff at line 56 of patch can't find file to patch at input line 56 Perhaps you used the wrong -p or --strip option? The text leading up to this was: -------------------------- | { | /* RFC 3164 timestamp, expected format: MMM DD HH:MM:SS ... */ | struct tm tm, *nowtm; |+ glong usec = 0; | | /* Just read the buffer data into a textual | datestamp. */ |+ | | g_string_assign_len(&self->date, src, 15); | src += 15; | left -= 15; | |+ if (left > 0 && src[0] == '.') |+ { |+ gulong frac = 0; |+ gint div = 1; |+ gint i = 1; |+ |+ /* gee, funny Cisco extension, BSD timestamp with fraction of |+ second support */ |+ |+ while (i < left && div < 10e6 && isdigit(src[i])) |+ { |+ frac = 10 * frac + (src[i]) - '0'; |+ div = div * 10; |+ i++; |+ } |+ usec = frac * (1000000 / div); |+ left -= i; |+ src += i; |+ } |+ | /* And also make struct time timestamp for the msg */ | | nowtm = localtime(&now); -------------------------- File to patch: logmsg.c patching file logmsg.c Hunk #1 succeeded at 313 with fuzz 1 (offset -26 lines). ------------<end paste>------------ I then ran diff to see if it worked correctly and it seems that only one line changed? --------<paste>--------- [jpoley@ITS-syslog src]$ diff logmsg.c logmsg.c.orig 295c295 < self->stamp.time.tv_usec = usec; ---
self->stamp.time.tv_usec = 0;
[jpoley@ITS-syslog src]$ --------<end paste>--------- What am I doing wrong? JDP --------------------------------- Jason D Poley Network Tech GS ITS Network County of Santa Barbara 805.568.2680 jpoley@co.santa-barbara.ca.us -----Original Message----- From: Balazs Scheidler [mailto:bazsi@balabit.hu] Sent: Thursday, July 19, 2007 2:25 PM To: Syslog-ng users' and developers' mailing list Cc: Poley, Jason Subject: RE: [syslog-ng] problems with Cisco WiFi controller syslog messages On Thu, 2007-07-19 at 11:09 -0700, Poley, Jason wrote:
Bazsi,
So I finally got around to the upgrade this morning (now on 2.0.4), only one month later...
Is it possible you can help with these obviously non-standard Cisco syslog timestamps?
The attached patch should do it. Please report back if it indeed worked, I could only test it with a small unit test program. While doing this I've also found a minor issue with fraction support on ISO timestamps, an integer overflow could have caused an invalid value, if second fraction was specified with more than 4 digits, because the formula: usec = frac * 1000000 / div could overflow, whenever frac is over 4294 (2^32/10e6). After using proper parentheses, the issue is gone. diff --git a/src/logmsg.c b/src/logmsg.c index 4ea6fb0..2ab8ae1 100644 --- a/src/logmsg.c +++ b/src/logmsg.c @@ -229,13 +229,13 @@ log_msg_parse(LogMessage *self, gchar *data, gint length, guint flags, regex_t * /* process second fractions */ p++; - while (isdigit(*p)) + while (div < 10e6 && isdigit(*p)) { frac = 10 * frac + (*p) - '0'; div = div * 10; p++; } - self->stamp.time.tv_usec = frac * 1000000 / div; + self->stamp.time.tv_usec = frac * (1000000 / div); } if (p && (*p == '+' || *p == '-') && strlen(p) == 6 && isdigit(*(p+1)) && isdigit(*(p+2)) && *(p+3) == ':' && isdigit(*(p+4)) && isdigit(*(p+5))) @@ -298,14 +298,35 @@ log_msg_parse(LogMessage *self, gchar *data, gint length, guint flags, regex_t * { /* RFC 3164 timestamp, expected format: MMM DD HH:MM:SS ... */ struct tm tm, *nowtm; + glong usec = 0; /* Just read the buffer data into a textual datestamp. */ + g_string_assign_len(&self->date, src, 15); src += 15; left -= 15; + if (left > 0 && src[0] == '.') + { + gulong frac = 0; + gint div = 1; + gint i = 1; + + /* gee, funny Cisco extension, BSD timestamp with fraction of second support */ + + while (i < left && div < 10e6 && isdigit(src[i])) + { + frac = 10 * frac + (src[i]) - '0'; + div = div * 10; + i++; + } + usec = frac * (1000000 / div); + left -= i; + src += i; + } + /* And also make struct time timestamp for the msg */ nowtm = localtime(&now); @@ -318,7 +339,7 @@ log_msg_parse(LogMessage *self, gchar *data, gint length, guint flags, regex_t * /* NOTE: no timezone information in the message, assume it is local time */ self->stamp.time.tv_sec = mktime(&tm); - self->stamp.time.tv_usec = 0; + self->stamp.time.tv_usec = usec; self->stamp.zone_offset = get_local_timezone_ofs(self->stamp.time.tv_sec); /* assume local timezone */ } -- Bazsi
On Fri, 2007-07-20 at 09:32 -0700, Poley, Jason wrote:
Well, I seem to be doing something wrong with patch...
I created a patch file from the enclosed update, starting from the 'diff' line down to the end (no including your sig of course). I then run patch and see the following output...
Try a current snapshot, that should already contain the patch. -- Bazsi
Guys, I think this is a small problem, but need some help solving it. I want to append 'yum' to each line of the yum.log file as currently it gets transferred to the remote logging server, but ends up in /var/log/messages as the filter to move it to /var/log/yum.log doesn't work. the filter is currently set to filter f_yum { program(yum); }; but I can't see anywhere where the program name is also transmitted. Can I append 'yum' to the initial transfer so that I can change the filter to filter f_yum { match(yum); }; Regards, -------- Andy Loughran www.zrmt.com m: 07921076319 ----- Original Message ----- From: "Balazs Scheidler" <bazsi@balabit.hu> To: "Jason Poley" <jpoley@co.santa-barbara.ca.us> Cc: "Syslog-ng users' and developers' mailing list" <syslog-ng@lists.balabit.hu> Sent: 20 July 2007 17:43:18 o'clock (GMT) Europe/London Subject: RE: [syslog-ng] problems with Cisco WiFi controller syslog messages On Fri, 2007-07-20 at 09:32 -0700, Poley, Jason wrote:
Well, I seem to be doing something wrong with patch...
I created a patch file from the enclosed update, starting from the 'diff' line down to the end (no including your sig of course). I then run patch and see the following output...
Try a current snapshot, that should already contain the patch. -- Bazsi _______________________________________________ syslog-ng maillist - syslog-ng@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html
Worked like a charm! Thanks for the assist. Cheers, JDP ________________________________ From: Balazs Scheidler [mailto:bazsi@balabit.hu] Sent: Fri 7/20/2007 9:43 AM To: Poley, Jason Cc: Syslog-ng users' and developers' mailing list Subject: RE: [syslog-ng] problems with Cisco WiFi controller syslog messages On Fri, 2007-07-20 at 09:32 -0700, Poley, Jason wrote:
Well, I seem to be doing something wrong with patch...
I created a patch file from the enclosed update, starting from the 'diff' line down to the end (no including your sig of course). I then run patch and see the following output...
Try a current snapshot, that should already contain the patch. -- Bazsi
Hi, I added the following entry to /etc/syslog.conf on a linux machine *.* @loghost On the central syslog-ng (server) , I only see the messages file under /var/log/syslog-ng/$HOST Is there a way to REALLY get everything happening on the remote Linux machine. Thanks From: syslog-ng-bounces@lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Poley, Jason Sent: Monday, July 23, 2007 6:58 AM To: Balazs Scheidler Cc: Syslog-ng users' and developers' mailing list Subject: RE: [syslog-ng] problems with Cisco WiFi controller syslog messages Worked like a charm! Thanks for the assist. Cheers, JDP ________________________________ From: Balazs Scheidler [mailto:bazsi@balabit.hu] Sent: Fri 7/20/2007 9:43 AM To: Poley, Jason Cc: Syslog-ng users' and developers' mailing list Subject: RE: [syslog-ng] problems with Cisco WiFi controller syslog messages On Fri, 2007-07-20 at 09:32 -0700, Poley, Jason wrote:
Well, I seem to be doing something wrong with patch...
I created a patch file from the enclosed update, starting from the 'diff' line down to the end (no including your sig of course). I then run patch and see the following output...
Try a current snapshot, that should already contain the patch. -- Bazsi
participants (4)
-
Andy Loughran
-
Balazs Scheidler
-
Poley, Jason
-
Tamer Tayea