[syslog-ng] lsof shows open files which should be closed

[Balazs Scheidler]

On Wed, 2007-12-19 at 12:31 +0100, Nagy Daniel wrote:
> Hello,
> 
> Scenario: this is a log server which receives logs from variuos hosts
> via TCP. We use time macros, which create a new log file in every hour
> for each remote host. The problem is, that 'lsof -c syslog-ng'
> shows, that the logfiles of the previous hours are still open,
> which exhausts the number of file descriptors after a while.
> kill -HUP syslog-ng corrects this as a workaround.
> 
> System: RHEL4
> Syslog-ng: 2.0.5 compiled from source
> 
> Relevant config parts:
> options {
>         flush_lines(16);
>         flush_timeout(1000);
>         time_reopen (10);
>         log_fifo_size (1000);
>         use_dns (yes);
>         use_fqdn (no);
>         create_dirs (no);
>         keep_hostname (yes);
>         stats_freq(0);
>         time_sleep(20);
>         time_reap(10);
> };
> 
> source s_net {
>         tcp(max-connections(512));
> };
> 
> destination d_freemail_pop3{
> file("/data/syslog/FP-$YEAR$MONTH$DAY-$HOUR0000-$HOST.log"
>    template ("$YEAR.$MONTH.$DAY $HOUR:$MIN:$SEC $TZOFFSET $HOST $MSG\n")
>    template_escape(no)
>    group("bar")
>    perm(0660));
> };
> 
> 
> For example 'lsof -c syslog-ng' at 12:20 shows these entries as open
> (it shows more, there are examples only):
> syslog-ng 8314 root   10w   REG              253,0   1840830    3965325
> /data/syslog/TM-20071219-110000-mx08a.log
> syslog-ng 8314 root  249w   REG              253,0   5521445    3965131
> /data/bar/data/target/mail/TM-20071219-100000-mx07a.log (deleted)
> 
> The second is also interesting. After some minutes of an hour change,
> a script moves the log to an other directory. That's why the path
> of the second line is changed. But is it still open by syslog-ng?

syslog-ng keeps the files opened for a little while after the last
message is written to it to avoid open/reopen for each message.

This interval can be controlled using the time_reap() option, which
defaults to 60sec, if I remember correctly.

Decrease time_reap(), or increase the number of file descriptors using
"ulimit -n".

-- 
Bazsi