On Wed, 2007-12-19 at 12:31 +0100, Nagy Daniel wrote:
Hello,
Scenario: this is a log server which receives logs from variuos hosts via TCP. We use time macros, which create a new log file in every hour for each remote host. The problem is, that 'lsof -c syslog-ng' shows, that the logfiles of the previous hours are still open, which exhausts the number of file descriptors after a while. kill -HUP syslog-ng corrects this as a workaround.
System: RHEL4 Syslog-ng: 2.0.5 compiled from source
Relevant config parts: options { flush_lines(16); flush_timeout(1000); time_reopen (10); log_fifo_size (1000); use_dns (yes); use_fqdn (no); create_dirs (no); keep_hostname (yes); stats_freq(0); time_sleep(20); time_reap(10); };
source s_net { tcp(max-connections(512)); };
destination d_freemail_pop3{ file("/data/syslog/FP-$YEAR$MONTH$DAY-$HOUR0000-$HOST.log" template ("$YEAR.$MONTH.$DAY $HOUR:$MIN:$SEC $TZOFFSET $HOST $MSG\n") template_escape(no) group("bar") perm(0660)); };
For example 'lsof -c syslog-ng' at 12:20 shows these entries as open (it shows more, there are examples only): syslog-ng 8314 root 10w REG 253,0 1840830 3965325 /data/syslog/TM-20071219-110000-mx08a.log syslog-ng 8314 root 249w REG 253,0 5521445 3965131 /data/bar/data/target/mail/TM-20071219-100000-mx07a.log (deleted)
The second is also interesting. After some minutes of an hour change, a script moves the log to an other directory. That's why the path of the second line is changed. But is it still open by syslog-ng?
syslog-ng keeps the files opened for a little while after the last message is written to it to avoid open/reopen for each message. This interval can be controlled using the time_reap() option, which defaults to 60sec, if I remember correctly. Decrease time_reap(), or increase the number of file descriptors using "ulimit -n". -- Bazsi