[syslog-ng]Syslog-ng 1.6.6 Redhat ES 3.0 - too many open files [solved]
syslog-ng@lists.balabit.hu
syslog-ng@lists.balabit.hu
Fri, 8 Apr 2005 10:14:23 -0400
This is a multipart message in MIME format.
--=_alternative 004E3C3085256FDD_=
Content-Type: text/plain; charset="US-ASCII"
Balazs,
It appears the patch is working. I've been running for over an hour with
no signs of "file handle leaks".
Thank you so much for the support. And Dave, I followed your suggestions
and upgraded the kernel as well (after I verified the patches worked).
For anyone else searching this list, here are the patches Balazs supplied
that fixed my various problems:
Fixed memory leak:
-----------------------------------------------------------------------------------------------------------------------------
diff -u -r1.25.4.6 afinet.c
--- afinet.c 5 Aug 2004 11:35:12 -0000 1.25.4.6
+++ afinet.c 28 Feb 2005 19:17:30 -0000
@@ -653,6 +653,7 @@
if (libnet_write(self->lnet_ctx) < 0) {
werror("Error sending raw frame, error:
%z", libnet_geterror(self->lnet_ctx));
}
+ ol_string_free(msg_line);
}
else {
fallback_socket:
--
Fixed intel/sparc endian issue for ports !=514:
-----------------------------------------------------------------------------------------------------------------------------
diff -u -r1.25.4.6 afinet.c
--- afinet.c 5 Aug 2004 11:35:12 -0000 1.25.4.6
+++ afinet.c 3 Mar 2005 12:26:47 -0000
@@ -617,11 +617,10 @@
ADDRESS2SOCKADDR(msg->saddr, sizeof(src_addr),
(struct sockaddr *) &src_addr);
ADDRESS2SOCKADDR(self->super.dest_addr,
sizeof(dst_addr), (struct sockaddr *) &dst_addr);
-
libnet_clear_packet(self->lnet_ctx);
- udp = libnet_build_udp(src_addr.sin_port,
- dst_addr.sin_port,
+ udp = libnet_build_udp(ntohs(src_addr.sin_port),
+ ntohs(dst_addr.sin_port),
LIBNET_UDP_H +
msg_line->length,
0,
msg_line->data,
Fixed "File handle leak" with UDP spoof:
-----------------------------------------------------------------------------------------------------------------------------
diff -u -r1.25.4.7 afinet.c
--- afinet.c 4 Mar 2005 16:06:38 -0000 1.25.4.7
+++ afinet.c 8 Apr 2005 09:00:15 -0000
@@ -520,7 +520,7 @@
!!(self->super.flags
& AFSOCKET_DGRAM));
}
#if ENABLE_SPOOF_SOURCE
- if (self->spoof_source) {
+ if (self->spoof_source && !self->lnet_ctx) {
UINT8 error[LIBNET_ERRBUF_SIZE];
self->lnet_ctx = libnet_init(LIBNET_RAW4, NULL, error);
if (!self->lnet_ctx) {
Balazs Scheidler <bazsi@balabit.hu>
Sent by: syslog-ng-admin@lists.balabit.hu
04/08/2005 05:03 AM
Please respond to
syslog-ng@lists.balabit.hu
To
syslog-ng@lists.balabit.hu
cc
Subject
Re: [syslog-ng]Syslog-ng 1.6.6 Redhat ES 3.0 - too many open files
On Thu, 2005-04-07 at 10:01 -0400, henry@shoelacecity.com wrote:
>
> Any further ideas on fixing/debuggering this?
>
As I see there is some reason which causes syslog-ng to reinit its
destination (maybe an ICMP port unreachable error returned by the
server?). Although that should be visible from the logs ("error
connecting to remote host", or "connection broken to ...")
Anyway, syslog-ng should not leak the libnet context in this case and it
does, can you check whether this patch works for you?
diff -u -r1.25.4.7 afinet.c
--- afinet.c 4 Mar 2005 16:06:38 -0000 1.25.4.7
+++ afinet.c 8 Apr 2005 09:00:15 -0000
@@ -520,7 +520,7 @@
!!(self->super.flags
& AFSOCKET_DGRAM));
}
#if ENABLE_SPOOF_SOURCE
- if (self->spoof_source) {
+ if (self->spoof_source && !self->lnet_ctx) {
UINT8 error[LIBNET_ERRBUF_SIZE];
self->lnet_ctx = libnet_init(LIBNET_RAW4, NULL, error);
if (!self->lnet_ctx) {
Please remember touching afinet.c.x after applying this patch if you
don't have scsh installed, then rebuild.
--
Bazsi
_______________________________________________
syslog-ng maillist - syslog-ng@lists.balabit.hu
https://lists.balabit.hu/mailman/listinfo/syslog-ng
Frequently asked questions at http://www.campin.net/syslog-ng/faq.html
--=_alternative 004E3C3085256FDD_=
Content-Type: text/html; charset="US-ASCII"
<br><font size=2 face="sans-serif">Balazs,</font>
<br>
<br><font size=2 face="sans-serif">It appears the patch is working. I've
been running for over an hour with no signs of "file handle
leaks". </font>
<br>
<br><font size=2 face="sans-serif">Thank you so much for the support. And
Dave, I followed your suggestions and upgraded the kernel as well (after
I verified the patches worked).</font>
<br>
<br><font size=2 face="sans-serif">For anyone else searching this list,
here are the patches Balazs supplied that fixed my various problems:</font>
<br>
<br>
<br>
<br>
<br><font size=2 face="sans-serif">Fixed memory leak:</font>
<br><font size=2 face="sans-serif">-----------------------------------------------------------------------------------------------------------------------------</font>
<br>
<br><font size=2 face="sans-serif">diff -u -r1.25.4.6 afinet.c</font>
<br><font size=2 face="sans-serif">--- afinet.c 5 Aug 2004
11:35:12 -0000 1.25.4.6</font>
<br><font size=2 face="sans-serif">+++ afinet.c 28 Feb 2005
19:17:30 -0000</font>
<br><font size=2 face="sans-serif">@@ -653,6 +653,7 @@</font>
<br><font size=2 face="sans-serif">
if (libnet_write(self->lnet_ctx)
< 0) {</font>
<br><font size=2 face="sans-serif">
werror("Error
sending raw frame, error: %z", libnet_geterror(self->lnet_ctx));</font>
<br><font size=2 face="sans-serif">
}</font>
<br><font size=2 face="sans-serif">+
ol_string_free(msg_line);</font>
<br><font size=2 face="sans-serif">
}</font>
<br><font size=2 face="sans-serif">
else {</font>
<br><font size=2 face="sans-serif">
fallback_socket:</font>
<br>
<br><font size=2 face="sans-serif">-- </font>
<br>
<br>
<br>
<br><font size=2 face="sans-serif">Fixed intel/sparc endian issue for ports
!=514:</font>
<br><font size=2 face="sans-serif">-----------------------------------------------------------------------------------------------------------------------------</font>
<br>
<br><font size=2 face="sans-serif">diff -u -r1.25.4.6 afinet.c</font>
<br><font size=2 face="sans-serif">--- afinet.c 5 Aug 2004
11:35:12 -0000 1.25.4.6</font>
<br><font size=2 face="sans-serif">+++ afinet.c 3 Mar 2005
12:26:47 -0000</font>
<br><font size=2 face="sans-serif">@@ -617,11 +617,10 @@</font>
<br><font size=2 face="sans-serif">
ADDRESS2SOCKADDR(msg->saddr,
sizeof(src_addr), (struct sockaddr *) &src_addr);</font>
<br><font size=2 face="sans-serif">
ADDRESS2SOCKADDR(self->super.dest_addr,
sizeof(dst_addr), (struct sockaddr *) &dst_addr);</font>
<br>
<br><font size=2 face="sans-serif">-</font>
<br><font size=2 face="sans-serif">
libnet_clear_packet(self->lnet_ctx);</font>
<br>
<br><font size=2 face="sans-serif">-
udp = libnet_build_udp(src_addr.sin_port,</font>
<br><font size=2 face="sans-serif">-
dst_addr.sin_port,</font>
<br><font size=2 face="sans-serif">+
udp = libnet_build_udp(ntohs(src_addr.sin_port),</font>
<br><font size=2 face="sans-serif">+
ntohs(dst_addr.sin_port),</font>
<br><font size=2 face="sans-serif">
LIBNET_UDP_H + msg_line->length,</font>
<br><font size=2 face="sans-serif">
0,</font>
<br><font size=2 face="sans-serif">
msg_line->data,</font>
<br><font size=2 face="sans-serif">
</font>
<br><font size=2 face="sans-serif">
</font>
<br><font size=2 face="sans-serif">
</font>
<br><font size=2 face="sans-serif">Fixed "File handle leak" with
UDP spoof:</font>
<br><font size=2 face="sans-serif">-----------------------------------------------------------------------------------------------------------------------------</font>
<br>
<br>
<br><font size=2 face="sans-serif">diff -u -r1.25.4.7 afinet.c</font>
<br><font size=2 face="sans-serif">--- afinet.c 4 Mar 2005
16:06:38 -0000 1.25.4.7</font>
<br><font size=2 face="sans-serif">+++ afinet.c 8 Apr 2005
09:00:15 -0000</font>
<br><font size=2 face="sans-serif">@@ -520,7 +520,7 @@</font>
<br><font size=2 face="sans-serif">
!!(self->super.flags
& AFSOCKET_DGRAM));</font>
<br><font size=2 face="sans-serif"> }</font>
<br><font size=2 face="sans-serif"> #if ENABLE_SPOOF_SOURCE</font>
<br><font size=2 face="sans-serif">- if (self->spoof_source)
{</font>
<br><font size=2 face="sans-serif">+ if (self->spoof_source
&& !self->lnet_ctx) {</font>
<br><font size=2 face="sans-serif">
UINT8 error[LIBNET_ERRBUF_SIZE];</font>
<br><font size=2 face="sans-serif">
self->lnet_ctx = libnet_init(LIBNET_RAW4, NULL, error);</font>
<br><font size=2 face="sans-serif">
if (!self->lnet_ctx) {</font>
<br><font size=2 face="sans-serif">
</font>
<br>
<br>
<br>
<br>
<br>
<table width=100%>
<tr valign=top>
<td width=40%><font size=1 face="sans-serif"><b>Balazs Scheidler <bazsi@balabit.hu></b>
</font>
<br><font size=1 face="sans-serif">Sent by: syslog-ng-admin@lists.balabit.hu</font>
<p><font size=1 face="sans-serif">04/08/2005 05:03 AM</font>
<table border>
<tr valign=top>
<td bgcolor=white>
<div align=center><font size=1 face="sans-serif">Please respond to<br>
syslog-ng@lists.balabit.hu</font></div></table>
<br>
<td width=59%>
<table width=100%>
<tr>
<td>
<div align=right><font size=1 face="sans-serif">To</font></div>
<td valign=top><font size=1 face="sans-serif">syslog-ng@lists.balabit.hu</font>
<tr>
<td>
<div align=right><font size=1 face="sans-serif">cc</font></div>
<td valign=top>
<tr>
<td>
<div align=right><font size=1 face="sans-serif">Subject</font></div>
<td valign=top><font size=1 face="sans-serif">Re: [syslog-ng]Syslog-ng
1.6.6 Redhat ES 3.0 - too many open files</font></table>
<br>
<table>
<tr valign=top>
<td>
<td></table>
<br></table>
<br>
<br>
<br><font size=2><tt>On Thu, 2005-04-07 at 10:01 -0400, henry@shoelacecity.com
wrote:<br>
> <br>
> Any further ideas on fixing/debuggering this? <br>
> <br>
<br>
As I see there is some reason which causes syslog-ng to reinit its<br>
destination (maybe an ICMP port unreachable error returned by the<br>
server?). Although that should be visible from the logs ("error<br>
connecting to remote host", or "connection broken to ...")<br>
<br>
Anyway, syslog-ng should not leak the libnet context in this case and it<br>
does, can you check whether this patch works for you?<br>
<br>
diff -u -r1.25.4.7 afinet.c<br>
--- afinet.c 4 Mar 2005 16:06:38 -0000
1.25.4.7<br>
+++ afinet.c 8 Apr 2005 09:00:15 -0000<br>
@@ -520,7 +520,7 @@<br>
!!(self->super.flags &
AFSOCKET_DGRAM));<br>
}<br>
#if ENABLE_SPOOF_SOURCE<br>
- if (self->spoof_source) {<br>
+ if (self->spoof_source && !self->lnet_ctx)
{<br>
UINT8 error[LIBNET_ERRBUF_SIZE];<br>
self->lnet_ctx
= libnet_init(LIBNET_RAW4, NULL, error);<br>
if (!self->lnet_ctx)
{<br>
<br>
Please remember touching afinet.c.x after applying this patch if you <br>
don't have scsh installed, then rebuild.<br>
<br>
-- <br>
Bazsi<br>
<br>
<br>
_______________________________________________<br>
syslog-ng maillist - syslog-ng@lists.balabit.hu<br>
https://lists.balabit.hu/mailman/listinfo/syslog-ng<br>
Frequently asked questions at http://www.campin.net/syslog-ng/faq.html<br>
<br>
</tt></font>
<br>
--=_alternative 004E3C3085256FDD_=--