[syslog-ng]Syslog-NG 1.6.6 memory leak when sending UDP logs
syslog-ng@lists.balabit.hu
syslog-ng@lists.balabit.hu
Mon, 28 Feb 2005 11:34:54 -0500
This is a multipart message in MIME format.
--=_alternative 005B309B85256FB6_=
Content-Type: text/plain; charset="US-ASCII"
No sooner did I write this that ElectricFence reported back (after 2 hours
of uptime):
"ElectricFence: Exiting: mprotect() failed: Cannot allocate memory"
The machine is a dual Proc P2 w/512 Mb RAM. syslog-ng initilizes with
10mb, and slowly consumes about a 1 mb per minute.
henry@shoelacecity.com
Sent by: syslog-ng-admin@lists.balabit.hu
02/28/2005 11:27 AM
Please respond to
syslog-ng@lists.balabit.hu
To
syslog-ng@lists.balabit.hu
cc
Subject
Re: [syslog-ng]Syslog-NG 1.6.6 memory leak when sending UDP logs
The installation is a fresh 1.6.6 installation on a clean install of
Redhat 3.0. It's been running fine for about 2 weeks logging the same
amount of traffic to local files only.
When I turned on the UDP destinations, it started chewing up memory. The
destinations are receiving logs entries properly, and the source address
is properly spoofed.
Versions of libs:
libnet 1.1.2
libol 0.3.14
Here's the syslog-ng config file - the only thing that changed to cause
this memory eating behavior is the 3 lines under "# Sets up local
destination for Cisco logs". It makes no difference if I am sending to a
single host, or multiple, the memory "leak" is still there.
I have it currently running under "ElectricFence" looking for malloc
problems - so far none, just memory consumption.
-------------------------------------------
options { long_hostnames(off); sync(0); };
source src { unix-stream("/dev/log"); internal(); };
source kernsrc { file("/proc/kmsg"); };
source net { udp(port(514)); };
# Sets up local desitation for Cisco logs
destination ciscolog { file("/var/log/cisco.log"); };
# Sets up forwarding to other hosts
destination loghostd { udp(loghostd spoof_source(yes)); };
destination loghosta { udp(loghosta spoof_source(yes)); };
destination scribe02 { udp(scribe02 spoof_source(yes)); };
# standard localhost logging stuff
destination authlog { file("/var/log/auth.log"); };
destination syslog { file("/var/log/syslog"); };
destination cron { file("/var/log/cron.log"); };
destination daemon { file("/var/log/daemon.log"); };
destination kern { file("/var/log/kern.log"); };
destination lpr { file("/var/log/lpr.log"); };
destination user { file("/var/log/user.log"); };
destination uucp { file("/var/log/uucp.log"); };
destination ppp { file("/var/log/ppp.log"); };
destination mail { file("/var/log/mail.log"); };
destination mailinfo { file("/var/log/mail.info"); };
destination mailwarn { file("/var/log/mail.warn"); };
destination mailerr { file("/var/log/mail.err"); };
destination local0 { file("/var/log/local0.log"); };
destination local1 { file("/var/log/local1.log"); };
destination all { file("/var/log/all.log"); };
destination allnet { file("/var/log/allnet.log"); };
destination newscrit { file("/var/log/news/news.crit"); };
destination newserr { file("/var/log/news/news.err"); };
destination newsnotice { file("/var/log/news/news.notice"); };
destination debug { file("/var/log/debug"); };
destination messages { file("/var/log/messages"); };
destination console { usertty("root"); };
destination console_all { file("/dev/tty12"); };
# Filters Cisco events - well, at least local7 and local4 events
filter f_cisco { facility(local7,local4);};
filter f_notcisco {not facility(local7, local4); };
filter f_auth { facility(auth); };
filter f_authpriv { facility(auth, authpriv); };
filter f_syslog { not facility(authpriv, mail) and not match(ppp.*LCP); };
filter f_cron { facility(cron); };
filter f_daemon { facility(daemon); };
filter f_kern { facility(kern); };
filter f_lpr { facility(lpr); };
filter f_mail { facility(mail) and not match (imapd); };
filter f_user { facility(user); };
filter f_uucp { facility(cron); };
filter f_ppp { program(ppp); };
filter f_news { facility(news); };
filter f_debug { not facility(auth, authpriv, news, mail) and not
match(ppp.*LCP); };
filter f_messages { level(info..warn)
and not facility(auth, authpriv, mail, news); };
filter f_emergency { level(emerg); };
filter f_info { level(info); };
filter f_notice { level(notice); };
filter f_warn { level(warn); };
filter f_crit { level(crit); };
filter f_err { level(err); };
# Reflects Cisco SYSLOG packets to remote hosts
log { source(net); filter(f_cisco); destination(loghostd); };
log { source(net); filter(f_cisco); destination(loghosta); };
log { source(net); filter(f_cisco); destination(scribe02); };
# Keeps a copy for ourself
log { source(net); filter(f_cisco); destination(ciscolog); };
log { source(src); filter(f_authpriv); destination(authlog); };
log { source(src); filter(f_syslog); destination(syslog); };
log { source(src); filter(f_cron); destination(cron); };
log { source(src); filter(f_daemon); destination(daemon); };
log { source(kernsrc); filter(f_kern); destination(kern); };
log { source(src); filter(f_lpr); destination(lpr); };
log { source(src); filter(f_mail); destination(mail); };
log { source(src); filter(f_user); destination(user); };
log { source(src); filter(f_uucp); destination(uucp); };
log { source(src); filter(f_mail); filter(f_info); destination(mailinfo);
};
log { source(src); filter(f_mail); filter(f_warn); destination(mailwarn);
};
log { source(src); filter(f_mail); filter(f_err); destination(mailerr); };
log { source(src); filter(f_news); filter(f_crit); destination(newscrit);
};
log { source(src); filter(f_news); filter(f_err); destination(newserr); };
log { source(src); filter(f_news); filter(f_notice);
destination(newsnotice); };
log { source(src); filter(f_debug); destination(debug); };
log { source(src); filter(f_messages); destination(messages); };
log { source(src); filter(f_emergency); destination(console); };
log { source(src); filter(f_ppp); destination(ppp); };
log { source(src); destination(console_all); };
log { source(src); destination(all); };
# Log all non cisco stuff to localfile for debugging
log { source(net); filter(f_notcisco); destination(allnet); };
Henry -
Balazs Scheidler <bazsi@balabit.hu>
Sent by: syslog-ng-admin@lists.balabit.hu
02/28/2005 11:11 AM
Please respond to
syslog-ng@lists.balabit.hu
To
syslog-ng@lists.balabit.hu
cc
Subject
Re: [syslog-ng]Syslog-NG 1.6.6 memory leak when sending UDP logs
On Mon, 2005-02-28 at 09:36 -0500, henry@shoelacecity.com wrote:
>
> I'm running 1.6.6 on Linux Redhat ES 3.0 logging Cisco router logs to
> local files and it's been running just fine for over a week.
> I just enabled forwarding for local4 and local7 (with spoof UDP) to
> 2 additional hosts via UDP and noticed, within a day, that it
> syslog-NG has consumed all available memory. It continues to log,
> however, the system starts killing processes off unless I restart
> syslog-ng.
If I understand you correctly you use syslog-ng to send logs to another
host via UDP. Is it a new installation or this installation has worked
so far and upgrading to 1.6.6 is what triggered the problem?
Are you using address spoofing (spoof-source(yes))? Can you post your
configuration file?
--
Bazsi
_______________________________________________
syslog-ng maillist - syslog-ng@lists.balabit.hu
https://lists.balabit.hu/mailman/listinfo/syslog-ng
Frequently asked questions at http://www.campin.net/syslog-ng/faq.html
--=_alternative 005B309B85256FB6_=
Content-Type: text/html; charset="US-ASCII"
<br><font size=2 face="sans-serif">No sooner did I write this that ElectricFence
reported back (after 2 hours of uptime):</font>
<br>
<br><font size=2 face="sans-serif">"ElectricFence: Exiting: mprotect()
failed: Cannot allocate memory"</font>
<br>
<br>
<br><font size=2 face="sans-serif">The machine is a dual Proc P2 w/512
Mb RAM. syslog-ng initilizes with 10mb, and slowly consumes about
a 1 mb per minute. </font>
<br>
<br>
<br>
<br>
<br>
<table width=100%>
<tr valign=top>
<td width=40%><font size=1 face="sans-serif"><b>henry@shoelacecity.com</b>
</font>
<br><font size=1 face="sans-serif">Sent by: syslog-ng-admin@lists.balabit.hu</font>
<p><font size=1 face="sans-serif">02/28/2005 11:27 AM</font>
<table border>
<tr valign=top>
<td bgcolor=white>
<div align=center><font size=1 face="sans-serif">Please respond to<br>
syslog-ng@lists.balabit.hu</font></div></table>
<br>
<td width=59%>
<table width=100%>
<tr>
<td>
<div align=right><font size=1 face="sans-serif">To</font></div>
<td valign=top><font size=1 face="sans-serif">syslog-ng@lists.balabit.hu</font>
<tr>
<td>
<div align=right><font size=1 face="sans-serif">cc</font></div>
<td valign=top>
<tr>
<td>
<div align=right><font size=1 face="sans-serif">Subject</font></div>
<td valign=top><font size=1 face="sans-serif">Re: [syslog-ng]Syslog-NG
1.6.6 memory leak when sending UDP logs</font></table>
<br>
<table>
<tr valign=top>
<td>
<td></table>
<br></table>
<br>
<br>
<br><font size=2 face="sans-serif"><br>
The installation is a fresh 1.6.6 installation on a clean install of Redhat
3.0. It's been running fine for about 2 weeks logging the
same amount of traffic to local files only.</font><font size=3> </font><font size=2 face="sans-serif"><br>
When I turned on the UDP destinations, it started chewing up memory. The
destinations are receiving logs entries properly, and the source address
is properly spoofed. </font><font size=3><br>
</font><font size=2 face="sans-serif"><br>
Versions of libs:</font><font size=3> <br>
</font><font size=2 face="sans-serif"><br>
libnet 1.1.2</font><font size=3> </font><font size=2 face="sans-serif"><br>
libol 0.3.14</font><font size=3> <br>
</font><font size=2 face="sans-serif"><br>
Here's the syslog-ng config file - the only thing that changed to cause
this memory eating behavior is the 3 lines under "# Sets up local
destination for Cisco logs". It makes no difference if I am sending
to a single host, or multiple, the memory "leak" is still there.</font><font size=3>
<br>
</font><font size=2 face="sans-serif"><br>
I have it currently running under "ElectricFence" looking
for malloc problems - so far none, just memory consumption. </font><font size=3><br>
</font><font size=2 face="sans-serif"><br>
-------------------------------------------</font><font size=3> </font><font size=2 face="sans-serif"><br>
options { long_hostnames(off); sync(0); };</font><font size=3> <br>
</font><font size=2 face="sans-serif"><br>
source src { unix-stream("/dev/log"); internal(); };</font><font size=3>
</font><font size=2 face="sans-serif"><br>
source kernsrc { file("/proc/kmsg"); };</font><font size=3> </font><font size=2 face="sans-serif"><br>
source net { udp(port(514)); };</font><font size=3> <br>
<br>
</font><font size=2 face="sans-serif"><br>
# Sets up local desitation for Cisco logs</font><font size=3> </font><font size=2 face="sans-serif"><br>
destination ciscolog { file("/var/log/cisco.log"); };</font><font size=3>
<br>
</font><font size=2 face="sans-serif"><br>
# Sets up forwarding to other hosts</font><font size=3> </font><font size=2 face="sans-serif"><br>
destination loghostd { udp(loghostd spoof_source(yes)); };</font><font size=3>
</font><font size=2 face="sans-serif"><br>
destination loghosta { udp(loghosta spoof_source(yes)); };</font><font size=3>
</font><font size=2 face="sans-serif"><br>
destination scribe02 { udp(scribe02 spoof_source(yes)); };</font><font size=3>
<br>
</font><font size=2 face="sans-serif"><br>
# standard localhost logging stuff</font><font size=3> </font><font size=2 face="sans-serif"><br>
destination authlog { file("/var/log/auth.log"); };</font><font size=3>
</font><font size=2 face="sans-serif"><br>
destination syslog { file("/var/log/syslog"); };</font><font size=3>
</font><font size=2 face="sans-serif"><br>
destination cron { file("/var/log/cron.log"); };</font><font size=3>
</font><font size=2 face="sans-serif"><br>
destination daemon { file("/var/log/daemon.log"); };</font><font size=3>
</font><font size=2 face="sans-serif"><br>
destination kern { file("/var/log/kern.log"); };</font><font size=3>
</font><font size=2 face="sans-serif"><br>
destination lpr { file("/var/log/lpr.log"); };</font><font size=3>
</font><font size=2 face="sans-serif"><br>
destination user { file("/var/log/user.log"); };</font><font size=3>
</font><font size=2 face="sans-serif"><br>
destination uucp { file("/var/log/uucp.log"); };</font><font size=3>
</font><font size=2 face="sans-serif"><br>
destination ppp { file("/var/log/ppp.log"); };</font><font size=3>
</font><font size=2 face="sans-serif"><br>
destination mail { file("/var/log/mail.log"); };</font><font size=3>
<br>
</font><font size=2 face="sans-serif"><br>
destination mailinfo { file("/var/log/mail.info"); };</font><font size=3>
</font><font size=2 face="sans-serif"><br>
destination mailwarn { file("/var/log/mail.warn"); };</font><font size=3>
</font><font size=2 face="sans-serif"><br>
destination mailerr { file("/var/log/mail.err"); };</font><font size=3>
</font><font size=2 face="sans-serif"><br>
destination local0 { file("/var/log/local0.log"); };</font><font size=3>
</font><font size=2 face="sans-serif"><br>
destination local1 { file("/var/log/local1.log"); };</font><font size=3>
<br>
</font><font size=2 face="sans-serif"><br>
destination all { file("/var/log/all.log"); };</font><font size=3>
</font><font size=2 face="sans-serif"><br>
destination allnet { file("/var/log/allnet.log"); };</font><font size=3>
<br>
</font><font size=2 face="sans-serif"><br>
destination newscrit { file("/var/log/news/news.crit"); };</font><font size=3>
</font><font size=2 face="sans-serif"><br>
destination newserr { file("/var/log/news/news.err"); };</font><font size=3>
</font><font size=2 face="sans-serif"><br>
destination newsnotice { file("/var/log/news/news.notice"); };</font><font size=3>
<br>
</font><font size=2 face="sans-serif"><br>
destination debug { file("/var/log/debug"); };</font><font size=3>
</font><font size=2 face="sans-serif"><br>
destination messages { file("/var/log/messages"); };</font><font size=3>
</font><font size=2 face="sans-serif"><br>
destination console { usertty("root"); };</font><font size=3>
</font><font size=2 face="sans-serif"><br>
destination console_all { file("/dev/tty12"); };</font><font size=3>
<br>
<br>
<br>
<br>
</font><font size=2 face="sans-serif"><br>
# Filters Cisco events - well, at least local7 and local4 events</font><font size=3>
</font><font size=2 face="sans-serif"><br>
filter f_cisco { facility(local7,local4);};</font><font size=3> </font><font size=2 face="sans-serif"><br>
filter f_notcisco {not facility(local7, local4); };</font><font size=3>
<br>
</font><font size=2 face="sans-serif"><br>
filter f_auth { facility(auth); };</font><font size=3> </font><font size=2 face="sans-serif"><br>
filter f_authpriv { facility(auth, authpriv); };</font><font size=3> </font><font size=2 face="sans-serif"><br>
filter f_syslog { not facility(authpriv, mail) and not match(ppp.*LCP);
};</font><font size=3> </font><font size=2 face="sans-serif"><br>
filter f_cron { facility(cron); };</font><font size=3> </font><font size=2 face="sans-serif"><br>
filter f_daemon { facility(daemon); };</font><font size=3> </font><font size=2 face="sans-serif"><br>
filter f_kern { facility(kern); };</font><font size=3> </font><font size=2 face="sans-serif"><br>
filter f_lpr { facility(lpr); };</font><font size=3> </font><font size=2 face="sans-serif"><br>
filter f_mail { facility(mail) and not match (imapd); };</font><font size=3>
</font><font size=2 face="sans-serif"><br>
filter f_user { facility(user); };</font><font size=3> </font><font size=2 face="sans-serif"><br>
filter f_uucp { facility(cron); };</font><font size=3> </font><font size=2 face="sans-serif"><br>
filter f_ppp { program(ppp); };</font><font size=3> </font><font size=2 face="sans-serif"><br>
filter f_news { facility(news); };</font><font size=3> </font><font size=2 face="sans-serif"><br>
filter f_debug { not facility(auth, authpriv, news, mail) and not match(ppp.*LCP);
};</font><font size=3> </font><font size=2 face="sans-serif"><br>
filter f_messages { level(info..warn) <br>
and not facility(auth, authpriv, mail, news);
};</font><font size=3> </font><font size=2 face="sans-serif"><br>
filter f_emergency { level(emerg); };</font><font size=3> <br>
</font><font size=2 face="sans-serif"><br>
filter f_info { level(info); };</font><font size=3> </font><font size=2 face="sans-serif"><br>
filter f_notice { level(notice); };</font><font size=3> </font><font size=2 face="sans-serif"><br>
filter f_warn { level(warn); };</font><font size=3> </font><font size=2 face="sans-serif"><br>
filter f_crit { level(crit); };</font><font size=3> </font><font size=2 face="sans-serif"><br>
filter f_err { level(err); };</font><font size=3> <br>
<br>
</font><font size=2 face="sans-serif"><br>
# Reflects Cisco SYSLOG packets to remote hosts</font><font size=3> </font><font size=2 face="sans-serif"><br>
log { source(net); filter(f_cisco); destination(loghostd); };</font><font size=3>
</font><font size=2 face="sans-serif"><br>
log { source(net); filter(f_cisco); destination(loghosta); };</font><font size=3>
</font><font size=2 face="sans-serif"><br>
log { source(net); filter(f_cisco); destination(scribe02); };</font><font size=3>
<br>
</font><font size=2 face="sans-serif"><br>
# Keeps a copy for ourself</font><font size=3> </font><font size=2 face="sans-serif"><br>
log { source(net); filter(f_cisco); destination(ciscolog); };</font><font size=3>
<br>
<br>
</font><font size=2 face="sans-serif"><br>
log { source(src); filter(f_authpriv); destination(authlog); };</font><font size=3>
</font><font size=2 face="sans-serif"><br>
log { source(src); filter(f_syslog); destination(syslog); };</font><font size=3>
</font><font size=2 face="sans-serif"><br>
log { source(src); filter(f_cron); destination(cron); };</font><font size=3>
</font><font size=2 face="sans-serif"><br>
log { source(src); filter(f_daemon); destination(daemon); };</font><font size=3>
</font><font size=2 face="sans-serif"><br>
log { source(kernsrc); filter(f_kern); destination(kern); };</font><font size=3>
</font><font size=2 face="sans-serif"><br>
log { source(src); filter(f_lpr); destination(lpr); };</font><font size=3>
</font><font size=2 face="sans-serif"><br>
log { source(src); filter(f_mail); destination(mail); };</font><font size=3>
</font><font size=2 face="sans-serif"><br>
log { source(src); filter(f_user); destination(user); };</font><font size=3>
</font><font size=2 face="sans-serif"><br>
log { source(src); filter(f_uucp); destination(uucp); };</font><font size=3>
</font><font size=2 face="sans-serif"><br>
log { source(src); filter(f_mail); filter(f_info); destination(mailinfo);
};</font><font size=3> </font><font size=2 face="sans-serif"><br>
log { source(src); filter(f_mail); filter(f_warn); destination(mailwarn);
};</font><font size=3> </font><font size=2 face="sans-serif"><br>
log { source(src); filter(f_mail); filter(f_err); destination(mailerr);
};</font><font size=3> </font><font size=2 face="sans-serif"><br>
log { source(src); filter(f_news); filter(f_crit); destination(newscrit);
};</font><font size=3> </font><font size=2 face="sans-serif"><br>
log { source(src); filter(f_news); filter(f_err); destination(newserr);
};</font><font size=3> </font><font size=2 face="sans-serif"><br>
log { source(src); filter(f_news); filter(f_notice); destination(newsnotice);
};</font><font size=3> </font><font size=2 face="sans-serif"><br>
log { source(src); filter(f_debug); destination(debug); };</font><font size=3>
</font><font size=2 face="sans-serif"><br>
log { source(src); filter(f_messages); destination(messages); };</font><font size=3>
</font><font size=2 face="sans-serif"><br>
log { source(src); filter(f_emergency); destination(console); };</font><font size=3>
</font><font size=2 face="sans-serif"><br>
log { source(src); filter(f_ppp); destination(ppp); };</font><font size=3>
</font><font size=2 face="sans-serif"><br>
log { source(src); destination(console_all); };</font><font size=3> <br>
</font><font size=2 face="sans-serif"><br>
log { source(src); destination(all); };</font><font size=3> <br>
</font><font size=2 face="sans-serif"><br>
# Log all non cisco stuff to localfile for debugging</font><font size=3>
</font><font size=2 face="sans-serif"><br>
log { source(net); filter(f_notcisco); destination(allnet); };</font><font size=3>
<br>
<br>
</font><font size=2 face="sans-serif"><br>
Henry -</font><font size=3> <br>
<br>
<br>
</font>
<table width=100%>
<tr valign=top>
<td width=36%><font size=1 face="sans-serif"><b>Balazs Scheidler <bazsi@balabit.hu></b>
<br>
Sent by: syslog-ng-admin@lists.balabit.hu</font><font size=3> </font>
<p><font size=1 face="sans-serif">02/28/2005 11:11 AM</font><font size=3>
</font>
<br>
<table border=4 width=100%>
<tr valign=top>
<td width=100% bgcolor=white>
<div align=center><font size=1 face="sans-serif">Please respond to<br>
syslog-ng@lists.balabit.hu</font></div></table>
<p>
<td width=63%>
<table width=100%>
<tr>
<td width=13%>
<div align=right><font size=1 face="sans-serif">To</font></div>
<td width=86% valign=top><font size=1 face="sans-serif">syslog-ng@lists.balabit.hu</font><font size=3>
</font>
<tr>
<td>
<div align=right><font size=1 face="sans-serif">cc</font></div>
<td valign=top>
<tr>
<td>
<div align=right><font size=1 face="sans-serif">Subject</font></div>
<td valign=top><font size=1 face="sans-serif">Re: [syslog-ng]Syslog-NG
1.6.6 memory leak when sending UDP logs</font></table>
<br>
<br>
<table width=100%>
<tr valign=top>
<td width=49%>
<td width=50%></table>
<br></table>
<br><font size=3><br>
<br>
</font><font size=2><tt><br>
On Mon, 2005-02-28 at 09:36 -0500, henry@shoelacecity.com wrote:<br>
> <br>
> I'm running 1.6.6 on Linux Redhat ES 3.0 logging Cisco router logs
to<br>
> local files and it's been running just fine for over a week. <br>
> I just enabled forwarding for local4 and local7 (with spoof
UDP) to<br>
> 2 additional hosts via UDP and noticed, within a day, that it<br>
> syslog-NG has consumed all available memory. It continues to
log,<br>
> however, the system starts killing processes off unless I restart<br>
> syslog-ng. <br>
<br>
If I understand you correctly you use syslog-ng to send logs to another<br>
host via UDP. Is it a new installation or this installation has worked<br>
so far and upgrading to 1.6.6 is what triggered the problem?<br>
<br>
Are you using address spoofing (spoof-source(yes))? Can you post your<br>
configuration file?<br>
<br>
-- <br>
Bazsi<br>
<br>
<br>
_______________________________________________<br>
syslog-ng maillist - syslog-ng@lists.balabit.hu<br>
https://lists.balabit.hu/mailman/listinfo/syslog-ng<br>
Frequently asked questions at http://www.campin.net/syslog-ng/faq.html<br>
</tt></font><font size=3><br>
</font>
<br>
--=_alternative 005B309B85256FB6_=--