Specific log messages have wrong hostname
All of the syslog messages sent by our Solaris servers that contain output from the service processor are getting the wrong hostname assigned to them - the log messages get filed under the hostname of the receiving syslog-ng server. They all share the same process name "SC Alert". The packets look correctly formed so I am assuming it is the space in the process name. Any way to fix this without dropping the messages completely using something like bad_hostname? We are running 2.1.11a Enterprise Edition. Here is the relevant part of my syslog-ng.conf: options { sync (0); create_dirs (yes); keep_hostname (yes); check_hostname (yes); chain_hostnames (no); bad_hostname ("\t"); normalize_hostnames (yes); mark_freq (10); }; source src_net { udp(port(514)); }; destination d_hosts { file("/var/log/systems/$HOST_FROM/$FACILITY/$YEAR/$MONTH/$FACILITY\-$YEAR$MONTH$DAY" perm(0644) dir_perm(0755) create_dirs(yes)); ... log { source(src_net); destination(d_hosts); destination(d_splunk); flags(fallback); }; And here is an example packet from snoop. ETHER: ----- Ether Header ----- ETHER: ETHER: Packet 36 arrived at 12:09:18.28026 ETHER: Packet size = 169 bytes ETHER: Destination = 0:3:ba:71:22:65, ETHER: Source = 0:21:28:4:ec:b7, ETHER: Ethertype = 0800 (IP) ETHER: IP: ----- IP Header ----- IP: IP: Version = 4 IP: Header length = 20 bytes IP: Type of service = 0x00 IP: xxx. .... = 0 (precedence) IP: ...0 .... = normal delay IP: .... 0... = normal throughput IP: .... .0.. = normal reliability IP: .... ..0. = not ECN capable transport IP: .... ...0 = no ECN congestion experienced IP: Total length = 155 bytes IP: Identification = 25576 IP: Flags = 0x4 IP: .1.. .... = do not fragment IP: ..0. .... = last fragment IP: Fragment offset = 0 bytes IP: Time to live = 255 seconds/hops IP: Protocol = 17 (UDP) IP: Header checksum = f267 IP: Source address = 10.244.236.183, vienna IP: Destination address = 10.241.34.101, auctor.backup.es.gwu.edu IP: No options IP: UDP: ----- UDP Header ----- UDP: UDP: Source port = 32832 UDP: Destination port = 514 (SYSLOG) UDP: Length = 135 UDP: Checksum = F5D8 UDP: SYSLOG: ----- SYSLOG: ----- SYSLOG: SYSLOG: Priority: <29> (daemon.notice) SYSLOG: "<29>Mar 12 12:09:18 SC Alert: [ID 362536 daemon.notice] Audi" SYSLOG: -- Christopher Caldwell Senior Engineer, Technology Operations and Engineering The George Washington University caldwell @ gwu . edu | +1 202.994.4674 (w) | +1 202.409.0878 (c) PGP key ID: 0x0A0EC46C "Quis custodiet ipsos custodes?" - Juvenal
Hi, I am unsure how to fix this on version 2.x (which does not mean that it cannot be done), but versions 3.0.x have an option called host-override for such cases. Regards, Robert On Friday, March 12, 2010 19:02 CET, Christopher Jon Caldwell <caldwell@gwu.edu> wrote:
All of the syslog messages sent by our Solaris servers that contain output from the service processor are getting the wrong hostname assigned to them - the log messages get filed under the hostname of the receiving syslog-ng server. They all share the same process name "SC Alert". The packets look correctly formed so I am assuming it is the space in the process name. Any way to fix this without dropping the messages completely using something like bad_hostname? We are running 2.1.11a Enterprise Edition.
Here is the relevant part of my syslog-ng.conf:
options { sync (0); create_dirs (yes); keep_hostname (yes); check_hostname (yes); chain_hostnames (no); bad_hostname ("\t"); normalize_hostnames (yes); mark_freq (10); };
source src_net { udp(port(514)); };
destination d_hosts { file("/var/log/systems/$HOST_FROM/$FACILITY/$YEAR/$MONTH/$FACILITY\-$YEAR$MONTH$DAY" perm(0644) dir_perm(0755) create_dirs(yes));
...
log { source(src_net); destination(d_hosts); destination(d_splunk); flags(fallback); };
And here is an example packet from snoop.
ETHER: ----- Ether Header ----- ETHER: ETHER: Packet 36 arrived at 12:09:18.28026 ETHER: Packet size = 169 bytes ETHER: Destination = 0:3:ba:71:22:65, ETHER: Source = 0:21:28:4:ec:b7, ETHER: Ethertype = 0800 (IP) ETHER: IP: ----- IP Header ----- IP: IP: Version = 4 IP: Header length = 20 bytes IP: Type of service = 0x00 IP: xxx. .... = 0 (precedence) IP: ...0 .... = normal delay IP: .... 0... = normal throughput IP: .... .0.. = normal reliability IP: .... ..0. = not ECN capable transport IP: .... ...0 = no ECN congestion experienced IP: Total length = 155 bytes IP: Identification = 25576 IP: Flags = 0x4 IP: .1.. .... = do not fragment IP: ..0. .... = last fragment IP: Fragment offset = 0 bytes IP: Time to live = 255 seconds/hops IP: Protocol = 17 (UDP) IP: Header checksum = f267 IP: Source address = 10.244.236.183, vienna IP: Destination address = 10.241.34.101, auctor.backup.es.gwu.edu IP: No options IP: UDP: ----- UDP Header ----- UDP: UDP: Source port = 32832 UDP: Destination port = 514 (SYSLOG) UDP: Length = 135 UDP: Checksum = F5D8 UDP: SYSLOG: ----- SYSLOG: ----- SYSLOG: SYSLOG: Priority: <29> (daemon.notice) SYSLOG: "<29>Mar 12 12:09:18 SC Alert: [ID 362536 daemon.notice] Audi" SYSLOG:
-- Christopher Caldwell
Senior Engineer, Technology Operations and Engineering The George Washington University caldwell @ gwu . edu | +1 202.994.4674 (w) | +1 202.409.0878 (c) PGP key ID: 0x0A0EC46C
"Quis custodiet ipsos custodes?" - Juvenal
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html
On Fri, 2010-03-12 at 13:02 -0500, Christopher Jon Caldwell wrote:
All of the syslog messages sent by our Solaris servers that contain output from the service processor are getting the wrong hostname assigned to them - the log messages get filed under the hostname of the receiving syslog-ng server. They all share the same process name "SC Alert". The packets look correctly formed so I am assuming it is the space in the process name. Any way to fix this without dropping the messages completely using something like bad_hostname? We are running 2.1.11a Enterprise Edition.
bad_hostname() was invented for this purpose. Or the 3.0.x versions provide rewrite functionality that lets you fix things like this. -- Bazsi
Another option could be to use $HOST_FROM. At our site, we used the hostname as part of the directory path, and we were seeing hostnames that were quite strange appaering from time to time until we switched to using $HOST_FROM in the destination path. The problem is that a number of "syslog" messages are not formatted correctly, so the parser pulls out incorrect portions of the log message interpreting them as the host name. The downside is that we end up with directories by IP address instead of hostname, but the upside is we are no longer dependent on how every application formats their log messages. (Caveat: If you forward messages more than once, you would get the address of the sending server, not the originating system.) Jim -----Original Message----- From: syslog-ng-bounces@lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Balazs Scheidler Sent: Thursday, March 18, 2010 6:17 AM To: Syslog-ng users' and developers' mailing list Subject: Re: [syslog-ng] Specific log messages have wrong hostname On Fri, 2010-03-12 at 13:02 -0500, Christopher Jon Caldwell wrote:
All of the syslog messages sent by our Solaris servers that contain output from the service processor are getting the wrong hostname assigned to them - the log messages get filed under the hostname of the receiving syslog-ng server. They all share the same process name "SC Alert". The packets look correctly formed so I am assuming it is the space in the process name. Any way to fix this without dropping the messages completely using something like bad_hostname? We are running 2.1.11a Enterprise Edition.
bad_hostname() was invented for this purpose. Or the 3.0.x versions provide rewrite functionality that lets you fix things like this. -- Bazsi ____________________________________________________________________________ __ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html
participants (4)
-
Balazs Scheidler
-
Christopher Jon Caldwell
-
Fekete Róbert
-
Jim Hendrick