hostname incorrect after upgrade to 3.1.2
I have used syslog-ng for many years and have been a faithful reader of the mailing list, but do not believe I have ever had occasion to post. Thank you for an excellent product and for the great support you have provided. I have always been able to solve problems by reading the archives...until now. I just completed an upgrade from source on several (6) systems from syslog-ng 3.1.1 (OSE) to 3.1.2 and then 3.1.3. All of them went smoothly, without error. However, after the upgrade to 3.1.2, one of the systems began reporting the hostname as "localhost" rather than the correct host name. The problem persisted when I upgraded to 3.1.3. When I roll it back to 3.1.1 it reports the hostname correctly. All of the systmes are fedora. Some are fc13, some are fc14. The system that failed is fc14. It is forwarding its logs to a central logger via tcp. I have checked the following: hosts file is OK. Contains only 127.0.0.1 entry as localhost dig -x returns the correct hostname hostname returns the correct hostname tcpdump confirms that syslog-ng is reporting "localhost" I was unable to correct the problem by changing global options. Global options are: options { flush_lines (0); time_reopen (10); log_fifo_size (1000); long_hostnames (off); use_dns (yes); use_fqdn (no); create_dirs (no); check_hostname(yes); chain_hostnames(no); keep_hostname (no); stats_freq (43200); mark_freq (1200); keep_timestamp (no); # send_time_zone (-04:00); }; I have googled but have not found anything that seemed relavant. Not sure how to procede to correct this situation. Any ideas would be appreciated. I would be happy to post additional data that will help.
Hi, On Mon, 2010-11-29 at 08:48 -0500, w3euu wrote:
I have used syslog-ng for many years and have been a faithful reader of the mailing list, but do not believe I have ever had occasion to post. Thank you for an excellent product and for the great support you have provided. I have always been able to solve problems by reading the archives...until now.
Thanks :)
I just completed an upgrade from source on several (6) systems from syslog-ng 3.1.1 (OSE) to 3.1.2 and then 3.1.3. All of them went smoothly, without error. However, after the upgrade to 3.1.2, one of the systems began reporting the hostname as "localhost" rather than the correct host name. The problem persisted when I upgraded to 3.1.3. When I roll it back to 3.1.1 it reports the hostname correctly.
I assume you see "localhost" on the central log server, right?
All of the systmes are fedora. Some are fc13, some are fc14. The system that failed is fc14. It is forwarding its logs to a central logger via tcp.
I have checked the following: hosts file is OK. Contains only 127.0.0.1 entry as localhost dig -x returns the correct hostname hostname returns the correct hostname tcpdump confirms that syslog-ng is reporting "localhost"
Hmm.. is this tcpdump running on the traffic between the client and the central server? Because of keep_hostname(no) setting in your server's settings, it doesn't matter what is on the wire.
I was unable to correct the problem by changing global options. Global options are:
options { flush_lines (0); time_reopen (10); log_fifo_size (1000); long_hostnames (off); use_dns (yes); use_fqdn (no); create_dirs (no); check_hostname(yes); chain_hostnames(no); keep_hostname (no); stats_freq (43200); mark_freq (1200); keep_timestamp (no); # send_time_zone (-04:00); };
I have googled but have not found anything that seemed relavant.
Not sure how to procede to correct this situation. Any ideas would be appreciated. I would be happy to post additional data that will help.
Since you are using keep_hostname(no), this means that the syslog-ng server will always use the resolved name as the hostname, rather than the one provided by the client. Can you show what tcpdump has shown on the wire traffic? Also, can you strace syslog-ng for a short while and see if syslog-ng is trying to resolve hostnames? (although syslog-ng will definitely cache hostnames, so that will only work for the first occasion as syslog-ng receives a message from the offending system). The only related issue I seem to remember is the problem with non-zero-terminating hostnames, but AFAIR it only affected chained_hostnames(yes) (equal to long_hostnames) which you don't use. -- Bazsi
I just completed an upgrade from source on several (6) systems from syslog-ng 3.1.1 (OSE) to 3.1.2 and then 3.1.3. All of them went smoothly, without error. However, after the upgrade to 3.1.2, one of the systems began reporting the hostname as "localhost" rather than the correct host name. The problem persisted when I upgraded to 3.1.3. When I roll it back to 3.1.1 it reports the hostname correctly.
I assume you see "localhost" on the central log server, right? I see "localhost" on the central server and also on the tcp link going from the failed (client) server to the central server -- as observed on the offending system.
All of the systmes are fedora. Some are fc13, some are fc14. The system that failed is fc14. It is forwarding its logs to a central logger via tcp.
tcpdump is running on the client machine, and monitoring the traffic leaving that machine, before it gets to the central server. Hmm.. is this tcpdump running on the traffic between the client and the central server? Because of keep_hostname(no) setting in your server's settings, it doesn't matter what is on the wire. Tcpdump is running on the client. The problem seems to be there, not on the central server. Since you are using keep_hostname(no), this means that the syslog-ng server will always use the resolved name as the hostname, rather than the one provided by the client. I am 95% sure I tried keep_hostname(yes) on the client earlier, and it made no difference. I can try again if you think that's the problem. Can you show what tcpdump has shown on the wire traffic? Also, can you strace syslog-ng for a short while and see if syslog-ng is trying to resolve hostnames? (although syslog-ng will definitely cache hostnames, so that will only work for the first occasion as syslog-ng receives a message from the offending system). I have attached tcpdump and strace outputs. Both tcpdump and strace are running on the "offending" system. It is running syslog-ng version 3.1.3. Not sure that the strace is what you want as I am new to that program. Let me know in greater detail what you need if this isn't it.
On Mon, 2010-11-29 at 15:51 -0500, w3euu wrote:
I just completed an upgrade from source on several (6) systems from syslog-ng 3.1.1 (OSE) to 3.1.2 and then 3.1.3. All of them went smoothly, without error. However, after the upgrade to 3.1.2, one of the systems began reporting the hostname as "localhost" rather than the correct host name. The problem persisted when I upgraded to 3.1.3. When I roll it back to 3.1.1 it reports the hostname correctly.
I assume you see "localhost" on the central log server, right?
I see "localhost" on the central server and also on the tcp link going from the failed (client) server to the central server -- as observed on the offending system.
All of the systmes are fedora. Some are fc13, some are fc14. The system that failed is fc14. It is forwarding its logs to a central logger via tcp.
tcpdump is running on the client machine, and monitoring the traffic leaving that machine, before it gets to the central server.
Hmm.. is this tcpdump running on the traffic between the client and the central server? Because of keep_hostname(no) setting in your server's settings, it doesn't matter what is on the wire.
Tcpdump is running on the client. The problem seems to be there, not on the central server.
Since you are using keep_hostname(no), this means that the syslog-ng server will always use the resolved name as the hostname, rather than the one provided by the client.
I am 95% sure I tried keep_hostname(yes) on the client earlier, and it made no difference. I can try again if you think that's the problem.
Can you show what tcpdump has shown on the wire traffic? Also, can you strace syslog-ng for a short while and see if syslog-ng is trying to resolve hostnames? (although syslog-ng will definitely cache hostnames, so that will only work for the first occasion as syslog-ng receives a message from the offending system).
I have attached tcpdump and strace outputs. Both tcpdump and strace are running on the "offending" system. It is running syslog-ng version 3.1.3. Not sure that the strace is what you want as I am new to that program. Let me know in greater detail what you need if this isn't it.
strace is not what I wanted, you should have followed the child process like this: strace -s 256 -o /tmp/strace.dmp -f /sbin/syslog-ng See this chapter in the documentation: http://www.balabit.com/sites/default/files/documents/syslog-ng-ose-v3.2-guid... (you may need to replace the path of syslog-ng as the documentation assumes the paths used by our binary installer) But anyway, I have an idea: do you have the hostname set on the client host properly? syslog-ng uses the gethostname() call to determine the hostname. And if that is not fully qualified, it'll attempt to do the same with reverse DNS (e.g. most probably from /etc/hosts). Here's the pseudo code that does the hostname lookup (original in the function reset_cached_hostname() in misc.c): local_hostname_fqdn = gethostname(local_hostname_fqdn); if (local_hostname_fqdn has no '.' character in it (e.g. not fully qualified)) { /* not fully qualified, resolve it using DNS or /etc/hosts */ local_hostname_fqdn = gethostbyname(local_hostname_fqdn); } e.g. it'll try to resolve the name returned by gethostname() (e.g. the contents of /etc/hostname on most distros) using /etc/hosts. -- Bazsi
OK. I've attached another strace. This one looks better. Sorry for the fumbled attempt earlier And thank you for the hint. Let me quickly review the situation. I have upgraded 6 systems from syslog-ng (OSE) 3.1.1 to 3.1.2 and then to 3.1.3. All went well. Five of the systms work great (including the central logging machine.) On the sixth system (hostname = bridge) syslog-ng doesn't pick up the system's hostname but instead syslog-ng reports the hostname as "locallhost." I proved that the error is occuring on "bridge" by running tcpdump on bridge. The tcpdump output provided yesterday shows the hostname "localhost" in the tcp output stream from bridge. Bridge reports its own hostname correctly using the hostname command, its reverse dns reports its hostname correctly and there is nothing in the hosts file that should create this problem. Rolling only bridge (ie not the other 5 systems) back to syslog-ng 3.1.1 fixes the problem. Tcpdump output shows the correct hostname (bridge) reported under 3.1.1. I changed nothing, including the conf file, on bridge except for the rollback to 3.1.1. So it would seem that something has changed between syslog-ng 3.1.1 and syslog-ng 3.1.2. Could be a misconfiguration on "bridge" but I can't find anything, and didn't change anything. Could be on syslog-ng. (Has a default changed? Has a peculiar bug apperaed?) Anyhow, hope you can help. If not, I guess my only option is to roll bridge back to 3.1.1 and use it that way. Thanks again. Lee -----Original Message----- From: Balazs Scheidler [mailto:bazsi@balabit.hu] Sent: Tuesday, November 30, 2010 7:38 AM To: Syslog-ng users' and developers' mailing list Subject: Re: [syslog-ng] hostname incorrect after upgrade to 3.1.2 On Mon, 2010-11-29 at 15:51 -0500, w3euu wrote:
I just completed an upgrade from source on several (6) systems from syslog-ng 3.1.1 (OSE) to 3.1.2 and then 3.1.3. All of them went smoothly, without error. However, after the upgrade to 3.1.2, one of the systems began reporting the hostname as "localhost" rather than the correct host name. The problem persisted when I upgraded to 3.1.3. When I roll it back to 3.1.1 it reports the hostname correctly.
I assume you see "localhost" on the central log server, right?
I see "localhost" on the central server and also on the tcp link going from the failed (client) server to the central server -- as observed on the offending system.
All of the systmes are fedora. Some are fc13, some are fc14. The
system
that failed is fc14. It is forwarding its logs to a central logger via tcp.
tcpdump is running on the client machine, and monitoring the traffic leaving that machine, before it gets to the central server.
Hmm.. is this tcpdump running on the traffic between the client and the central server? Because of keep_hostname(no) setting in your server's settings, it doesn't matter what is on the wire.
Tcpdump is running on the client. The problem seems to be there, not on the central server.
Since you are using keep_hostname(no), this means that the syslog-ng server will always use the resolved name as the hostname, rather than the one provided by the client.
I am 95% sure I tried keep_hostname(yes) on the client earlier, and it made no difference. I can try again if you think that's the problem.
Can you show what tcpdump has shown on the wire traffic? Also, can you strace syslog-ng for a short while and see if syslog-ng is trying to resolve hostnames? (although syslog-ng will definitely cache hostnames, so that will only work for the first occasion as syslog-ng receives a message from the offending system).
I have attached tcpdump and strace outputs. Both tcpdump and strace are running on the "offending" system. It is running syslog-ng version 3.1.3. Not sure that the strace is what you want as I am new to that program. Let me know in greater detail what you need if this isn't it.
strace is not what I wanted, you should have followed the child process like this: strace -s 256 -o /tmp/strace.dmp -f /sbin/syslog-ng See this chapter in the documentation: http://www.balabit.com/sites/default/files/documents/syslog-ng-ose-v3.2-guid e-admin-en.html/syslog-ng-strace-truss-tusc.html (you may need to replace the path of syslog-ng as the documentation assumes the paths used by our binary installer) But anyway, I have an idea: do you have the hostname set on the client host properly? syslog-ng uses the gethostname() call to determine the hostname. And if that is not fully qualified, it'll attempt to do the same with reverse DNS (e.g. most probably from /etc/hosts). Here's the pseudo code that does the hostname lookup (original in the function reset_cached_hostname() in misc.c): local_hostname_fqdn = gethostname(local_hostname_fqdn); if (local_hostname_fqdn has no '.' character in it (e.g. not fully qualified)) { /* not fully qualified, resolve it using DNS or /etc/hosts */ local_hostname_fqdn = gethostbyname(local_hostname_fqdn); } e.g. it'll try to resolve the name returned by gethostname() (e.g. the contents of /etc/hostname on most distros) using /etc/hosts. -- Bazsi ____________________________________________________________________________ __ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html
On Tue, 2010-11-30 at 11:55 -0500, w3euu wrote:
OK. I've attached another strace. This one looks better. Sorry for the fumbled attempt earlier And thank you for the hint.
Let me quickly review the situation.
I have upgraded 6 systems from syslog-ng (OSE) 3.1.1 to 3.1.2 and then to 3.1.3. All went well. Five of the systms work great (including the central logging machine.) On the sixth system (hostname = bridge) syslog-ng doesn't pick up the system's hostname but instead syslog-ng reports the hostname as "locallhost." I proved that the error is occuring on "bridge" by running tcpdump on bridge. The tcpdump output provided yesterday shows the hostname "localhost" in the tcp output stream from bridge. Bridge reports its own hostname correctly using the hostname command, its reverse dns reports its hostname correctly and there is nothing in the hosts file that should create this problem.
The problem is that "bridge" as a hostname is listed for the 127.0.0.1 address and there the fully-qualified-name is "localhost.localdomain". There was a change in syslog-ng, to use DNS and /etc/hosts to determine the local hostname, since /etc/hostname usually has no FQDN. So, the solution is to add an entry to /etc/hosts with the local IP address and the FQDN of bridge and the hostname. -- Bazsi
That did it! Thank you. Somehow "bridge" got included in the hosts file on the 127.0.0.1 line. I must have done it, but don't know why I would have ever done such a thing. Simply removing bridge from the line and restarting syslog-ng corected the problem. I looked right at it earlier and didn't see it. Thanks again for your help and your patience. Lee -----Original Message----- From: Balazs Scheidler [mailto:bazsi@balabit.hu] Sent: Wednesday, December 01, 2010 7:16 AM To: Syslog-ng users' and developers' mailing list Subject: Re: [syslog-ng] hostname incorrect after upgrade to 3.1.2 On Tue, 2010-11-30 at 11:55 -0500, w3euu wrote:
OK. I've attached another strace. This one looks better. Sorry for the fumbled attempt earlier And thank you for the hint.
Let me quickly review the situation.
I have upgraded 6 systems from syslog-ng (OSE) 3.1.1 to 3.1.2 and then to 3.1.3. All went well. Five of the systms work great (including the central logging machine.) On the sixth system (hostname = bridge) syslog-ng doesn't pick up the system's hostname but instead syslog-ng reports the hostname as "locallhost." I proved that the error is occuring on "bridge" by running tcpdump on bridge. The tcpdump output provided yesterday shows the hostname "localhost" in the tcp output stream from bridge. Bridge reports its own hostname correctly using the hostname command, its reverse dns reports its hostname correctly and there is nothing in the hosts file that should create this problem.
The problem is that "bridge" as a hostname is listed for the 127.0.0.1 address and there the fully-qualified-name is "localhost.localdomain". There was a change in syslog-ng, to use DNS and /etc/hosts to determine the local hostname, since /etc/hostname usually has no FQDN. So, the solution is to add an entry to /etc/hosts with the local IP address and the FQDN of bridge and the hostname. -- Bazsi ____________________________________________________________________________ __ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html
participants (2)
-
Balazs Scheidler
-
w3euu