[syslog-ng] Fixed number fd-limit doesn't scale

[Corinna Vinschen]

On Jan 19 11:42, Corinna Vinschen wrote:
> On Jan 19 10:55, Balazs Scheidler wrote:
> > On Sat, 2009-01-17 at 10:30 +0100, Corinna Vinschen wrote:
> > > --- gprocess.c.ORIG	2009-01-17 10:17:42.000000000 +0100
> > > +++ gprocess.c	2009-01-17 10:26:56.000000000 +0100
> > > @@ -484,10 +484,16 @@ g_process_change_limits(void)
> > >  {
> > >    struct rlimit limit;
> > >  
> > > -  limit.rlim_cur = limit.rlim_max = process_opts.fd_limit_min;
> > > -  
> > > -  if (setrlimit(RLIMIT_NOFILE, &limit) < 0)
> > > -    g_process_message("Error setting file number limit; limit='%d'; error='%s'", process_opts.fd_limit_min, g_strerror(errno));
> > > +  if (getrlimit (RLIMIT_NOFILE, &limit) == 0)
> > > +    {
> > > +      if (limit.rlim_max == RLIM_INFINITY || limit.rlim_max > process_opts.fd_limit_min)
> > > +      	limit.rlim_cur = limit.rlim_max = process_opts.fd_limit_min;
> > > +      else
> > > +      	limit.rlim_cur = limit.rlim_max;
> > > +      if (setrlimit(RLIMIT_NOFILE, &limit) == 0)
> > > +	return;
> > > +    }
> > > +  g_process_message("Error setting file number limit; limit='%d'; error='%s'", process_opts.fd_limit_min, g_strerror(errno));
> > >  }
> > >  
> > 
> > The whole point is to _increase_ the ulimit value, not to decrease it.
> > The problem is that people run into the default ulimit values in large
> > installations, e.g.
> > 
> > destination d_file { file("/var/log/$HOST/messages"); };
> > 
> > If you have a large number of different $HOST values, the default limit
> > of 1024 fds might prove small. The patch above, unless I miss something,
> > is not increasing the fd limit, it only decreases it.
> 
> The above patch sets limit.rlim_cur to limit.rlim_max and limit.rlim_max
> is the maximum a process can get anyway.  On my Linux box, the default
> values for rlim_cur is 1024 and rlim_max is 8192.  So the above code
> raises the current limit to the max of 8192.  That's what you want,
> right?

Btw., if you want to allow to decrease the rlim_max value, you just have
to tweak the above condition. which uses the process_opts.fd_limit_min
value as upper limit.

Other than that, IMHO it's not feasible to decrease the hard limit over
its standard value by *default*, allowing syslog-ng by default to waste
system resources.  That's just not safe.  If an installation has the
problem you're mentioning, it should set the value explicitly to a
higher value, instead of requiring to decrease the value in smaller
environments.  I really think that raising rlim_cur and rlim_max should
only be done when requested with the fd-limit setting.


Corinna

-- 
Corinna Vinschen
Cygwin Project Co-Leader
Red Hat