Any further ideas on fixing/debuggering this? henry@shoelacecity.com Sent by: syslog-ng-admin@lists.balabit.hu 03/31/2005 11:56 AM Please respond to syslog-ng@lists.balabit.hu To syslog-ng@lists.balabit.hu cc Subject Re: [syslog-ng]Syslog-ng 1.6.6 Redhat ES 3.0 - too many open files I never kill HUP the process. I do a full kill, and then a clean start. I've been needing to do this daily to prevent the accumulated open sockets from causing problems. The sockets clear as soon as I kill syslog-ng(as they should), and start building slowly after a retarting it. One minor thing I left out is that it is an dual proc machine running the SMP kernel. Here's a snippet of the complete log: Mar 31 10:56:41 scribe01 syslog-ng[17624]: STATS: dropped 14307 Mar 31 10:56:58 scribe01 syslog-ng[17624]: Error initializing raw socket, spoof-source support disabled. (libnet_open_raw: SOCK_RAW allocation failed: Too many open files ) Mar 31 10:58:01 scribe01 syslog-ng[17624]: Error initializing raw socket, spoof-source support disabled. (libnet_open_raw: SOCK_RAW allocation failed: Too many open files ) Mar 31 10:59:03 scribe01 syslog-ng[17624]: Error initializing raw socket, spoof-source support disabled. (libnet_open_raw: SOCK_RAW allocation failed: Too many open files ) Mar 31 11:00:03 scribe01 syslog-ng[17624]: Error initializing raw socket, spoof-source support disabled. (libnet_open_raw: SOCK_RAW allocation failed: Too many open files ) Mar 31 11:01:03 scribe01 syslog-ng[17624]: Error initializing raw socket, spoof-source support disabled. (libnet_open_raw: SOCK_RAW allocation failed: Too many open files ) Mar 31 11:02:06 scribe01 syslog-ng[17624]: Error initializing raw socket, spoof-source support disabled. (libnet_open_raw: SOCK_RAW allocation failed: Too many open files ) Mar 31 11:03:06 scribe01 syslog-ng[17624]: Error initializing raw socket, spoof-source support disabled. (libnet_open_raw: SOCK_RAW allocation failed: Too many open files ) Mar 31 11:04:06 scribe01 syslog-ng[17624]: Error initializing raw socket, spoof-source support disabled. (libnet_open_raw: SOCK_RAW allocation failed: Too many open files ) Mar 31 11:05:07 scribe01 syslog-ng[17624]: Error initializing raw socket, spoof-source support disabled. (libnet_open_raw: SOCK_RAW allocation failed: Too many open files ) Mar 31 11:06:11 scribe01 syslog-ng[17624]: Error initializing raw socket, spoof-source support disabled. (libnet_open_raw: SOCK_RAW allocation failed: Too many open files ) Mar 31 11:06:41 scribe01 syslog-ng[17624]: STATS: dropped 15328 Mar 31 11:07:11 scribe01 syslog-ng[17624]: Error initializing raw socket, spoof-source support disabled. (libnet_open_raw: SOCK_RAW allocation failed: Too many open files ) Mar 31 11:08:13 scribe01 syslog-ng[17624]: Error initializing raw socket, spoof-source support disabled. (libnet_open_raw: SOCK_RAW allocation failed: Too many open files ) Mar 31 11:09:13 scribe01 syslog-ng[17624]: Error initializing raw socket, spoof-source support disabled. (libnet_open_raw: SOCK_RAW allocation failed: Too many open files ) Mar 31 11:10:13 scribe01 syslog-ng[17624]: Error initializing raw socket, spoof-source support disabled. (libnet_open_raw: SOCK_RAW allocation failed: Too many open files ) Mar 31 11:11:14 scribe01 syslog-ng[17624]: Error initializing raw socket, spoof-source support disabled. (libnet_open_raw: SOCK_RAW allocation failed: Too many open files ) Mar 31 11:12:15 scribe01 syslog-ng[17624]: Error initializing raw socket, spoof-source support disabled. (libnet_open_raw: SOCK_RAW allocation failed: Too many open files ) Balazs Scheidler <bazsi@balabit.hu> Sent by: syslog-ng-admin@lists.balabit.hu 03/30/2005 02:59 AM Please respond to syslog-ng@lists.balabit.hu To syslog-ng@lists.balabit.hu cc Subject Re: [syslog-ng]Syslog-ng 1.6.6 Redhat ES 3.0 - too many open files On Wed, 2005-03-30 at 09:51 +0200, Balazs Scheidler wrote:
On Tue, 2005-03-29 at 13:08 -0500, henry@shoelacecity.com wrote:
My syslog-ng.conf specifies only 4 real log file to write to, and one pipe(for mysql writing), and two UDP destinations (spoof enabled).
CPU utilization on the machine is less than 5%, and there's plenty of free memory.
Hmmm it is strange, you most certainly have an fd leak, even though the libnet context (ie. the raw socket) is initialized at destination initialization time and destroyed at deinitialization time, seemingly properly.
This is only done on initialization and configuration file reload. Are you HUP-ing syslog-ng very often? I can't see how so many raw sockets accumulated, assuming that one fd is leaked for each HUP.
Hmm.. I've now tried to reproduce the problem but without success, I created a spoof-source enabled UDP destination, sent a couple of messages then sent a HUP to syslog-ng, again a couple of messages, HUP and so on a couple of times. The end result was that I had a single raw socket opened. I'm still curious how many times you HUP syslog-ng in one week to have so many raw sockets accumulated. Either there's a problem in your libnet library, or something triggers a reinit within syslog-ng without tearing down the previous instance. But I can't reproduce that here. Another related question is that I see this in your logs that you sent: syslog-ng[15710]: STATS: dropped 12828 syslog-ng[15710]: Error initializing raw socket, spoof- source support disabled. (libnet_open_raw: SOCK_RAW allocation failed: Too many open files) syslog-ng[15710]: Error initializing raw socket, spoof- source support disabled. (libnet_open_raw: SOCK_RAW allocation failed: Too many open files) syslog-ng[15710]: Error initializing raw socket, spoof- source support disabled. (libnet_open_raw: SOCK_RAW allocation failed: Too many open files) Can you add the timestamps for these, just to see the interval between those? -- Bazsi _______________________________________________ syslog-ng maillist - syslog-ng@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html