[syslog-ng] lsof shows open files which should be closed

[Nagy Daniel]

Hello,

The current option is time_reap(10), and according to lsof the files
are still open after several minutes.

Bye,

Daniel


Balazs Scheidler wrote:
> On Wed, 2007-12-19 at 12:31 +0100, Nagy Daniel wrote:
>> Hello,
>>
>> Scenario: this is a log server which receives logs from variuos hosts
>> via TCP. We use time macros, which create a new log file in every hour
>> for each remote host. The problem is, that 'lsof -c syslog-ng'
>> shows, that the logfiles of the previous hours are still open,
>> which exhausts the number of file descriptors after a while.
>> kill -HUP syslog-ng corrects this as a workaround.
>>
>> System: RHEL4
>> Syslog-ng: 2.0.5 compiled from source
>>
>> Relevant config parts:
>> options {
>>         flush_lines(16);
>>         flush_timeout(1000);
>>         time_reopen (10);
>>         log_fifo_size (1000);
>>         use_dns (yes);
>>         use_fqdn (no);
>>         create_dirs (no);
>>         keep_hostname (yes);
>>         stats_freq(0);
>>         time_sleep(20);
>>         time_reap(10);
>> };
>>
>> source s_net {
>>         tcp(max-connections(512));
>> };
>>
>> destination d_freemail_pop3{
>> file("/data/syslog/FP-$YEAR$MONTH$DAY-$HOUR0000-$HOST.log"
>>    template ("$YEAR.$MONTH.$DAY $HOUR:$MIN:$SEC $TZOFFSET $HOST $MSG\n")
>>    template_escape(no)
>>    group("bar")
>>    perm(0660));
>> };
>>
>>
>> For example 'lsof -c syslog-ng' at 12:20 shows these entries as open
>> (it shows more, there are examples only):
>> syslog-ng 8314 root   10w   REG              253,0   1840830    3965325
>> /data/syslog/TM-20071219-110000-mx08a.log
>> syslog-ng 8314 root  249w   REG              253,0   5521445    3965131
>> /data/bar/data/target/mail/TM-20071219-100000-mx07a.log (deleted)
>>
>> The second is also interesting. After some minutes of an hour change,
>> a script moves the log to an other directory. That's why the path
>> of the second line is changed. But is it still open by syslog-ng?
> 
> syslog-ng keeps the files opened for a little while after the last
> message is written to it to avoid open/reopen for each message.
> 
> This interval can be controlled using the time_reap() option, which
> defaults to 60sec, if I remember correctly.
> 
> Decrease time_reap(), or increase the number of file descriptors using
> "ulimit -n".
>