[syslog-ng] STATS: dropped with version 1.6.8 under rh el4

[Stephan Hendl]

Hi all,

recently I upgraded one of my central syslog servers from hpux to redhat el4 with syslog-ng-1.6.8-1 and get from time to time messages like

Aug 10 11:43:57 pns1 syslog-ng[10542]: STATS: dropped 19
Aug 10 11:48:57 pns1 syslog-ng[10542]: STATS: dropped 45
Aug 10 11:53:57 pns1 syslog-ng[10542]: STATS: dropped 0
Aug 10 11:58:57 pns1 syslog-ng[10542]: STATS: dropped 36
Aug 10 12:03:57 pns1 syslog-ng[10542]: STATS: dropped 7
Aug 10 12:08:57 pns1 syslog-ng[10542]: STATS: dropped 0
Aug 10 12:13:57 pns1 syslog-ng[10542]: STATS: dropped 0
Aug 10 12:18:57 pns1 syslog-ng[10542]: STATS: dropped 0
Aug 10 12:23:57 pns1 syslog-ng[10542]: STATS: dropped 0
Aug 10 12:28:57 pns1 syslog-ng[10542]: STATS: dropped 0
Aug 10 12:33:57 pns1 syslog-ng[10542]: STATS: dropped 0
Aug 10 12:38:57 pns1 syslog-ng[10542]: STATS: dropped 0
Aug 10 12:43:57 pns1 syslog-ng[10542]: STATS: dropped 0
Aug 10 12:48:57 pns1 syslog-ng[10542]: STATS: dropped 37
Aug 10 12:53:57 pns1 syslog-ng[10542]: STATS: dropped 52
Aug 10 12:58:57 pns1 syslog-ng[10542]: STATS: dropped 0
Aug 10 13:03:57 pns1 syslog-ng[10542]: STATS: dropped 0
Aug 10 13:08:57 pns1 syslog-ng[10542]: STATS: dropped 51
Aug 10 13:13:57 pns1 syslog-ng[10542]: STATS: dropped 0
Aug 10 13:18:57 pns1 syslog-ng[10542]: STATS: dropped 0
Aug 10 13:23:57 pns1 syslog-ng[10542]: STATS: dropped 11
Aug 10 13:28:57 pns1 syslog-ng[10542]: STATS: dropped 1
Aug 10 13:33:57 pns1 syslog-ng[10542]: STATS: dropped 0
Aug 10 13:38:57 pns1 syslog-ng[10542]: STATS: dropped 0

There are 5 servers who are sending their syslog messages via tcp an one 2 servers who are sending via udp to that central server. The load of the machine (2xXenon CPU, 3 GHz, 1 GB RAM) is really low. For revison purposes (maillogdata) I have to keep _all_ syslog lines!!! What can I change in order to avoid dropped lines?

Stephan

top - 15:39:06 up 36 days,  1:49,  2 users,  load average: 0.10, 0.09, 0.04
Tasks: 194 total,   1 running, 193 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.3% us,  0.2% sy,  0.0% ni, 99.2% id,  0.4% wa,  0.0% hi,  0.0% si
Mem:   1034692k total,  1018520k used,    16172k free,   141372k buffers
Swap:  2096440k total,      144k used,  2096296k free,   602228k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
  690 root      16   0 43644 4784 2180 S  0.7  0.5 129:42.96 nco_p_syslog
  779 root      16   0 43644 4784 2180 S  0.3  0.5 254:33.32 nco_p_syslog
10542 root      15   0  1876 1048  776 S  0.3  0.1   6:59.31 syslog-ng
    1 root      16   0  3236  548  472 S  0.0  0.1   0:00.96 init
    2 root      RT   0     0    0    0 S  0.0  0.0   0:04.37 migration/0
    3 root      34  19     0    0    0 S  0.0  0.0   0:00.06 ksoftirqd/0
    4 root      RT   0     0    0    0 S  0.0  0.0   0:03.11 migration/1
    5 root      34  19     0    0    0 S  0.0  0.0   0:00.02 ksoftirqd/1

My syslog-ng.conf looks like:

root at pns1:~# vi /etc/syslog-ng/syslog-ng.conf
# syslog-ng configuration file.
#
# This should behave pretty much like the original syslog on RedHat Linux 9.
# But it could be configured a lot smarter.
#
# See syslog-ng(8) and syslog-ng.conf(5) for more information.

options {sync (0);
         time_reopen (10);
         long_hostnames (off);
         use_dns (yes);
         use_fqdn (no);
         create_dirs (no);
         keep_hostname (yes);
         stats (300);
};

source s_sys { pipe ("/proc/kmsg" log_prefix("kernel: ")); unix-stream ("/dev/log"); internal(); };
source s_tcp { tcp(ip(a.b.c.d) port(10514) keep-alive(no)); };
source s_udp { udp(ip(a.b.c.d) port(514)); };

# Log all kernel messages to the console.
# Logging much else clutters up the screen.
#destination    d_cons  { file("/dev/console"); };
#filter         f_cons  { facility(kern); };
#log { source(s_sys); filter(f_cons); destination(d_cons); };

# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
destination     d_mesg  { file("/var/log/messages"); };
filter          f_mesg  { not facility(mail,news,authpriv,cron) and level(emerg..info); };
log { source(s_sys); filter(f_mesg); destination(d_mesg); };

# The authpriv file has restricted access.
destination     d_auth  { file("/var/log/secure"); };
filter          f_auth  { facility(authpriv); };
log { source(s_sys); filter(f_auth); destination(d_auth); };

# Log all the mail messages in one place.
destination     d_mail  { file("/var/log/maillog" template( "$DATE $HOST $MESSAGE\n")); };
filter          f_mail  { facility(mail); };
log { source(s_sys); filter(f_mail); destination(d_mail); };

# Log cron stuff.
destination     d_cron  { file("/var/log/cron"); };
filter          f_cron  { facility(cron); };
log { source(s_sys); filter(f_cron); destination(d_cron); };

# Everybody gets emergency messages.
destination     d_mlal  { usertty("*"); };
filter          f_mlal  { level(emerg); };
log { source(s_sys); filter(f_mlal); destination(d_mlal); };

# Save news errors of level crit and higher in a special file.
destination     d_spol  { file("/var/log/spooler"); };
filter          f_spol  {facility(uucp) or facility(news) and level(emerg..crit); };
log { source(s_sys); filter(f_spol); destination(d_spol); };

# Save boot messages also to boot.log.
destination     d_boot  { file("/var/log/boot.log"); };
filter          f_boot  { facility(local7); };
log { source(s_sys); filter(f_boot); destination(d_boot); };

# Save all mesages to syslog.lvnbb.de
destination     d_tcp   { tcp("localhost" port(10514)); };
log { source(s_sys); destination(d_tcp); };
log { source(s_tcp); destination(d_tcp); };
log { source(s_udp); destination(d_tcp); };

# Save all messages to pipe /tmp/cic.pipe
destination     d_cic   { pipe("/tmp/cic.pipe" owner("root") group("root") perm(0640) template("$DATE $HOST $MESSAGE\n") template-escape(yes)); };
log { source(s_sys); destination(d_cic); };
log { source(s_tcp); destination(d_cic); };
log { source(s_udp); destination(d_cic); };