I'm running 2.0.0, and have eight remote servers logging to a central server. Seven of those servers are running fine; the eighth keeps getting log messages like this: Jun 26 10:41:33 kyushu.denmantire.com syslog-ng[6829]: syslog-ng starting up; version='2.0.0' Jun 26 10:41:33 kyushu.denmantire.com syslog-ng: syslog-ng startup succeeded Jun 26 10:41:33 kyushu.denmantire.com syslog-ng[6829]: EOF occurred while idle;fd='5' Jun 26 10:41:33 kyushu.denmantire.com syslog-ng[6829]: Connection broken; time_reopen='60' My first assumption was a firewall problem, but tcpdump says that data's getting there: 10:42:49.013423 IP kyushu-vpn-cli.denmantire.com.37759 > buran.denmantire.com.5142: S 1168830611:1168830611(0) win 5840 <mss 1460,sackOK,timestamp 316509070 0,nop,wscale 2> 10:42:49.014768 IP buran.denmantire.com.5142 > kyushu-vpn-cli.denmantire.com.37759: S 845996771:845996771(0) ack 1168830612 win 5792 <mss 1460,sackOK,timestamp 39334539 316509070,nop,wscale 7> Any ideas what could be causing the connection to drop - but only on this server? Thanks much, -- Tim Boyer Director IT and Engineering Projects Denman Tire Corporation (330) 675-4249
On Tue, 2007-06-26 at 10:45 -0400, Tim Boyer wrote:
I'm running 2.0.0, and have eight remote servers logging to a central server. Seven of those servers are running fine; the eighth keeps getting log messages like this:
Jun 26 10:41:33 kyushu.denmantire.com syslog-ng[6829]: syslog-ng starting up; version='2.0.0' Jun 26 10:41:33 kyushu.denmantire.com syslog-ng: syslog-ng startup succeeded Jun 26 10:41:33 kyushu.denmantire.com syslog-ng[6829]: EOF occurred while idle;fd='5' Jun 26 10:41:33 kyushu.denmantire.com syslog-ng[6829]: Connection broken; time_reopen='60'
My first assumption was a firewall problem, but tcpdump says that data's getting there:
10:42:49.013423 IP kyushu-vpn-cli.denmantire.com.37759 > buran.denmantire.com.5142: S 1168830611:1168830611(0) win 5840 <mss 1460,sackOK,timestamp 316509070 0,nop,wscale 2> 10:42:49.014768 IP buran.denmantire.com.5142 > kyushu-vpn-cli.denmantire.com.37759: S 845996771:845996771(0) ack 1168830612 win 5792 <mss 1460,sackOK,timestamp 39334539 316509070,nop,wscale 7>
Any ideas what could be causing the connection to drop - but only on this server?
The "EOF" occurred while idle means that syslog-ng sensed incoming data on a simplex channel, this should only happen if the remote end is closing the channel. Please start tcpdump on the given connection and check what kind of packets go through when the connection is broken. You should see a FIN packet or a packet data has data payload. This should never happen. -- Bazsi
On Tue, 2007-06-26 at 10:45 -0400, Tim Boyer wrote:
I'm running 2.0.0, and have eight remote servers logging to a central server. Seven of those servers are running fine; the eighth keeps getting log messages like this:
Jun 26 10:41:33 kyushu.denmantire.com syslog-ng[6829]: syslog-ng starting up; version='2.0.0' Jun 26 10:41:33 kyushu.denmantire.com syslog-ng: syslog-ng startup succeeded Jun 26 10:41:33 kyushu.denmantire.com syslog-ng[6829]: EOF occurred while idle;fd='5' Jun 26 10:41:33 kyushu.denmantire.com syslog-ng[6829]: Connection broken; time_reopen='60'
My first assumption was a firewall problem, but tcpdump says that data's getting there:
10:42:49.013423 IP kyushu-vpn-cli.denmantire.com.37759 > buran.denmantire.com.5142: S 1168830611:1168830611(0) win 5840 <mss 1460,sackOK,timestamp 316509070 0,nop,wscale 2> 10:42:49.014768 IP buran.denmantire.com.5142 > kyushu-vpn-cli.denmantire.com.37759: S 845996771:845996771(0) ack 1168830612 win 5792 <mss 1460,sackOK,timestamp 39334539 316509070,nop,wscale 7>
Any ideas what could be causing the connection to drop - but only on this server?
The "EOF" occurred while idle means that syslog-ng sensed incoming data on a simplex channel, this should only happen if the remote end is closing the channel.
Please start tcpdump on the given connection and check what kind of packets go through when the connection is broken.
You should see a FIN packet or a packet data has data payload. This should never happen.
-- Bazsi
Not seeing it: [root@buran tmp]# tcpdump port 5142 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth1, link-type EN10MB (Ethernet), capture size 96 bytes 19:15:46.504529 IP kyushu-vpn-cli.denmantire.com.38378 > buran.denmantire.com.5142: S 3593561021:3593561021(0) win 5840 <mss 1460,sackOK,timestamp 347293143 0,nop,wscale 2> 19:15:46.506014 IP buran.denmantire.com.5142 > kyushu-vpn-cli.denmantire.com.38378: S 3279541618:3279541618(0) ack 3593561022 win 5792 <mss 1460,sackOK,timestamp 47028610 347293143,nop,wscale 7> 19:15:46.720099 IP kyushu-vpn-cli.denmantire.com.38378 > buran.denmantire.com.5142: . ack 1 win 1460 <nop,nop,timestamp 347293280 47028610> 19:15:46.720119 IP kyushu-vpn-cli.denmantire.com.38378 > buran.denmantire.com.5142: P 1:101(100) ack 1 win 1460 <nop,nop,timestamp 347293280 47028610> 19:15:46.720128 IP buran.denmantire.com.5142 > kyushu-vpn-cli.denmantire.com.38378: . ack 101 win 46 <nop,nop,timestamp 47028664 347293280> 19:15:46.720505 IP buran.denmantire.com.5142 > kyushu-vpn-cli.denmantire.com.38378: R 1:1(0) ack 101 win 46 <nop,nop,timestamp 47028664 347293280> 19:15:46.785157 IP kyushu-vpn-cli.denmantire.com.38378 > buran.denmantire.com.5142: P 101:183(82) ack 1 win 1460 <nop,nop,timestamp 347293422 47028664> 19:15:46.785204 IP buran.denmantire.com.5142 > kyushu-vpn-cli.denmantire.com.38378: R 3279541619:3279541619(0) win 0 Jun 26 19:15:46 kyushu.denmantire.com syslog-ng: syslog-ng startup succeeded Jun 26 19:15:46 kyushu.denmantire.com syslog-ng[20057]: EOF occurred while idle; fd='5' Jun 26 19:15:46 kyushu.denmantire.com syslog-ng[20057]: Connection broken; time_reopen='60' -- tim --
I started seeing this kind of behaviour on my syslog-ng clients when I updated my syslog-ng server to 2.0.4 and tracked it down to the newely added support of TCPWrappers. There was no clue on the client machines since the rejection occured on the syslog-ng server. Just adding my $0.02 so that nothing was overlooked. Evan. Tim Boyer wrote:
On Tue, 2007-06-26 at 10:45 -0400, Tim Boyer wrote:
I'm running 2.0.0, and have eight remote servers logging to
a central
server. Seven of those servers are running fine; the
eighth keeps getting
log messages like this:
Jun 26 10:41:33 kyushu.denmantire.com syslog-ng[6829]:
syslog-ng starting
up; version='2.0.0' Jun 26 10:41:33 kyushu.denmantire.com syslog-ng: syslog-ng
startup succeeded
Jun 26 10:41:33 kyushu.denmantire.com syslog-ng[6829]: EOF
occurred while
idle;fd='5' Jun 26 10:41:33 kyushu.denmantire.com syslog-ng[6829]:
Connection broken;
time_reopen='60'
My first assumption was a firewall problem, but tcpdump
says that data's
getting there:
10:42:49.013423 IP kyushu-vpn-cli.denmantire.com.37759 > buran.denmantire.com.5142: S 1168830611:1168830611(0) win 5840 <mss 1460,sackOK,timestamp 316509070 0,nop,wscale 2> 10:42:49.014768 IP buran.denmantire.com.5142 > kyushu-vpn-cli.denmantire.com.37759: S
845996771:845996771(0) ack 1168830612
win 5792 <mss 1460,sackOK,timestamp 39334539 316509070,nop,wscale 7>
Any ideas what could be causing the connection to drop -
but only on this
server?
The "EOF" occurred while idle means that syslog-ng sensed incoming data on a simplex channel, this should only happen if the remote end is closing the channel.
Please start tcpdump on the given connection and check what kind of packets go through when the connection is broken.
You should see a FIN packet or a packet data has data payload. This should never happen.
-- Bazsi
Not seeing it:
[root@buran tmp]# tcpdump port 5142 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth1, link-type EN10MB (Ethernet), capture size 96 bytes 19:15:46.504529 IP kyushu-vpn-cli.denmantire.com.38378 > buran.denmantire.com.5142: S 3593561021:3593561021(0) win 5840 <mss 1460,sackOK,timestamp 347293143 0,nop,wscale 2> 19:15:46.506014 IP buran.denmantire.com.5142 > kyushu-vpn-cli.denmantire.com.38378: S 3279541618:3279541618(0) ack 3593561022 win 5792 <mss 1460,sackOK,timestamp 47028610 347293143,nop,wscale 7> 19:15:46.720099 IP kyushu-vpn-cli.denmantire.com.38378 > buran.denmantire.com.5142: . ack 1 win 1460 <nop,nop,timestamp 347293280 47028610> 19:15:46.720119 IP kyushu-vpn-cli.denmantire.com.38378 > buran.denmantire.com.5142: P 1:101(100) ack 1 win 1460 <nop,nop,timestamp 347293280 47028610> 19:15:46.720128 IP buran.denmantire.com.5142 > kyushu-vpn-cli.denmantire.com.38378: . ack 101 win 46 <nop,nop,timestamp 47028664 347293280> 19:15:46.720505 IP buran.denmantire.com.5142 > kyushu-vpn-cli.denmantire.com.38378: R 1:1(0) ack 101 win 46 <nop,nop,timestamp 47028664 347293280> 19:15:46.785157 IP kyushu-vpn-cli.denmantire.com.38378 > buran.denmantire.com.5142: P 101:183(82) ack 1 win 1460 <nop,nop,timestamp 347293422 47028664> 19:15:46.785204 IP buran.denmantire.com.5142 > kyushu-vpn-cli.denmantire.com.38378: R 3279541619:3279541619(0) win 0
Jun 26 19:15:46 kyushu.denmantire.com syslog-ng: syslog-ng startup succeeded Jun 26 19:15:46 kyushu.denmantire.com syslog-ng[20057]: EOF occurred while idle; fd='5' Jun 26 19:15:46 kyushu.denmantire.com syslog-ng[20057]: Connection broken; time_reopen='60'
-- tim --
_______________________________________________ syslog-ng maillist - syslog-ng@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html
I started seeing this kind of behaviour on my syslog-ng clients when I updated my syslog-ng server to 2.0.4 and tracked it down to the newely added support of TCPWrappers. There was no clue on the client machines since the rejection occured on the syslog-ng server.
Just adding my $0.02 so that nothing was overlooked.
Evan.
Evan - What did you have to do to get it going again? -- tim --
I added a line to the /etc/hosts.allow file on the syslog-ng server. syslog-ng:.uvic.ca:ALLOW where the .uvic.ca domain is the scope of the client syslog-ng machines. You setting will be different. As a quick test, you could add the line syslog-ng:ALL:ALLOW If things start to work then you know this is the cause and you can lock down the connection a little better. You could also compile syslog-ng without tcp wrappers support, but that would have to be installed onto the syslong-ng server. Evan. Tim Boyer wrote:
I started seeing this kind of behaviour on my syslog-ng clients when I updated my syslog-ng server to 2.0.4 and tracked it down to the newely added support of TCPWrappers. There was no clue on the client machines since the rejection occured on the syslog-ng server.
Just adding my $0.02 so that nothing was overlooked.
Evan.
Evan -
What did you have to do to get it going again?
-- tim --
_______________________________________________ syslog-ng maillist - syslog-ng@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html
That did it! Sonofagun. Weird thing is, it's only happening on the one client. But that fixed it. Thanks _very_ much... -- tim --
I added a line to the /etc/hosts.allow file on the syslog-ng server.
syslog-ng:.uvic.ca:ALLOW
where the .uvic.ca domain is the scope of the client syslog-ng machines. You setting will be different.
As a quick test, you could add the line
syslog-ng:ALL:ALLOW
If things start to work then you know this is the cause and you can lock down the connection a little better.
You could also compile syslog-ng without tcp wrappers support, but that would have to be installed onto the syslong-ng server.
Evan.
Tim Boyer wrote:
I started seeing this kind of behaviour on my syslog-ng clients when I updated my syslog-ng server to 2.0.4 and tracked it down to the newely added support of TCPWrappers. There was no clue on the client machines since the rejection occured on the syslog-ng server.
Just adding my $0.02 so that nothing was overlooked.
Evan.
Evan -
What did you have to do to get it going again?
-- tim --
_______________________________________________ syslog-ng maillist - syslog-ng@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html
On Tue, 2007-06-26 at 19:19 -0400, Tim Boyer wrote:
On Tue, 2007-06-26 at 10:45 -0400, Tim Boyer wrote:
I'm running 2.0.0, and have eight remote servers logging to a central server. Seven of those servers are running fine; the eighth keeps getting log messages like this:
Jun 26 10:41:33 kyushu.denmantire.com syslog-ng[6829]: syslog-ng starting up; version='2.0.0' Jun 26 10:41:33 kyushu.denmantire.com syslog-ng: syslog-ng startup succeeded Jun 26 10:41:33 kyushu.denmantire.com syslog-ng[6829]: EOF occurred while idle;fd='5' Jun 26 10:41:33 kyushu.denmantire.com syslog-ng[6829]: Connection broken; time_reopen='60'
My first assumption was a firewall problem, but tcpdump says that data's getting there:
10:42:49.013423 IP kyushu-vpn-cli.denmantire.com.37759 > buran.denmantire.com.5142: S 1168830611:1168830611(0) win 5840 <mss 1460,sackOK,timestamp 316509070 0,nop,wscale 2> 10:42:49.014768 IP buran.denmantire.com.5142 > kyushu-vpn-cli.denmantire.com.37759: S 845996771:845996771(0) ack 1168830612 win 5792 <mss 1460,sackOK,timestamp 39334539 316509070,nop,wscale 7>
Any ideas what could be causing the connection to drop - but only on this server?
The "EOF" occurred while idle means that syslog-ng sensed incoming data on a simplex channel, this should only happen if the remote end is closing the channel.
Please start tcpdump on the given connection and check what kind of packets go through when the connection is broken.
You should see a FIN packet or a packet data has data payload. This should never happen.
-- Bazsi
Not seeing it:
[root@buran tmp]# tcpdump port 5142 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth1, link-type EN10MB (Ethernet), capture size 96 bytes 19:15:46.504529 IP kyushu-vpn-cli.denmantire.com.38378 > buran.denmantire.com.5142: S 3593561021:3593561021(0) win 5840 <mss 1460,sackOK,timestamp 347293143 0,nop,wscale 2> 19:15:46.506014 IP buran.denmantire.com.5142 > kyushu-vpn-cli.denmantire.com.38378: S 3279541618:3279541618(0) ack 3593561022 win 5792 <mss 1460,sackOK,timestamp 47028610 347293143,nop,wscale 7> 19:15:46.720099 IP kyushu-vpn-cli.denmantire.com.38378 > buran.denmantire.com.5142: . ack 1 win 1460 <nop,nop,timestamp 347293280 47028610> 19:15:46.720119 IP kyushu-vpn-cli.denmantire.com.38378 > buran.denmantire.com.5142: P 1:101(100) ack 1 win 1460 <nop,nop,timestamp 347293280 47028610> 19:15:46.720128 IP buran.denmantire.com.5142 > kyushu-vpn-cli.denmantire.com.38378: . ack 101 win 46 <nop,nop,timestamp 47028664 347293280> 19:15:46.720505 IP buran.denmantire.com.5142 > kyushu-vpn-cli.denmantire.com.38378: R 1:1(0) ack 101 win 46 <nop,nop,timestamp 47028664 347293280> 19:15:46.785157 IP kyushu-vpn-cli.denmantire.com.38378 > buran.denmantire.com.5142: P 101:183(82) ack 1 win 1460 <nop,nop,timestamp 347293422 47028664> 19:15:46.785204 IP buran.denmantire.com.5142 > kyushu-vpn-cli.denmantire.com.38378: R 3279541619:3279541619(0) win 0
Jun 26 19:15:46 kyushu.denmantire.com syslog-ng: syslog-ng startup succeeded Jun 26 19:15:46 kyushu.denmantire.com syslog-ng[20057]: EOF occurred while idle; fd='5' Jun 26 19:15:46 kyushu.denmantire.com syslog-ng[20057]: Connection broken; time_reopen='60'
the tcpdump indicates that the server drops connection right after it was established. Hmm.. tcpwrappers might be a good idea to check, as I see syslog-ng generates a log verbose log message in this case. (try running syslog-ng with -v on the server). I'll change this log level to have a higher severity. -- Bazsi
Hi, when I restart syslog-ng on a client, the loghost prints these lines: Jun 28 12:28:38 loghost syslog-ng[7068]: AF_INET client dropped connection from w.x.y.z, port 38171 Jun 28 12:28:39 loghost syslog-ng[7068]: AF_INET client connected from w.x.y.z, port 38040 Do I lose some information from the client? If yes, what can I do? thanks -- Pol Moal - Comité Réseau des Universités |--> http://www.cru.fr
On Thu, 2007-06-28 at 12:42 +0200, Pol Moal wrote:
Hi,
when I restart syslog-ng on a client, the loghost prints these lines:
Jun 28 12:28:38 loghost syslog-ng[7068]: AF_INET client dropped connection from w.x.y.z, port 38171
Jun 28 12:28:39 loghost syslog-ng[7068]: AF_INET client connected from w.x.y.z, port 38040
Do I lose some information from the client? If yes, what can I do?
If you restart syslog-ng, then 1) some messages might arrive while syslog-ng itself is stopped. These messages are lost. 2) the messages that had been been accumulated in syslog-ng's output buffers can also be lost. I don't really see solutions for item 1) above, item 2) is solved by Premium Edition's disk buffer feature. You should not periodically trigger a restart, use the 'reload' functionality instead, that's much better in this regard. -- Bazsi
When you say "use the reload functionality" are you refering to SIGHUP? Or, is there another method of triggering a reload. Thanks Anthony
Balazs Scheidler <bazsi@balabit.hu> 07/02/07 7:01 PM >>> On Thu, 2007- 06- 28 at 12:42 +0200, Pol Moal wrote: Hi,
when I restart syslog- ng on a client, the loghost prints these lines:
Jun 28 12:28:38 loghost syslog- ng[7068]: AF_INET client dropped connection from w.x.y.z, port 38171
Jun 28 12:28:39 loghost syslog- ng[7068]: AF_INET client connected from w.x.y.z, port 38040
Do I lose some information from the client? If yes, what can I do?
If you restart syslog- ng, then 1) some messages might arrive while syslog- ng itself is stopped. These messages are lost. 2) the messages that had been been accumulated in syslog- ng's output buffers can also be lost. I don't really see solutions for item 1) above, item 2) is solved by Premium Edition's disk buffer feature. You should not periodically trigger a restart, use the 'reload' functionality instead, that's much better in this regard. -- Bazsi _______________________________________________ syslog- ng maillist - syslog- ng@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog- ng Frequently asked questions at http://www.campin.net/syslog- ng/faq.html
On Tue, 2007-07-03 at 09:22 +1200, anthony lineham wrote:
When you say "use the reload functionality" are you refering to SIGHUP? Or, is there another method of triggering a reload.
Yeah, I meant SIGHUP. -- Bazsi
participants (5)
-
anthony lineham
-
Balazs Scheidler
-
Evan Rempel
-
Pol Moal
-
Tim Boyer