[syslog-ng] syslog-ng 3.3.3 repeatedly writes same message to local file when forwarding enabled

[Dave Haywood]

Hi,

  I have a problem with syslog-ng 3.3.3.  When I have forwarding enabled to a remote syslog server (via UDP) syslog-ng repeatedly writes the same message(s) to the log file and only stops when the disk is full.  Using tcpdump on the remote server, I don't see any data arrive from the syslog-ng server so forwarding is not working either.

  When I remove the forwarding part of the config file the local file is written correctly (ie once).  If I remove the local file part from the config file and only enable the forwarding, I see syslog-ng take all the CPU time.  I never see any syslog messages arrive at the remote syslog server.

  I tried:
	1) disabling IPv6 - no change
	2) running outside the chroot jail - no change
	3) running as userid root - no change

  Does anyone have any idea what would cause this?  Debug info below.

  The environment is:

RedHat AS 4.8 (linux 2.6.9-89.ELsmp) on vmware ESXi 4.1.0

All required software built and installed in /usr/local/ :

eventlog_0.2.12.tar.gz
gettext-0.18.1.1.tar.gz
glib-2.29.90.tar.bz2
libdbi-0.8.4.tar.gz
libdbi-drivers-0.8.3.tar.gz
libffi-3.0.9.tar.gz
libnet-0.10.11.tar.gz
pkg-config-0.26.tar.gz
Python-2.7.2.tar.bz2
zlib-1.2.5.tar.bz2
syslog-ng_3.3.3.tar.gz

syslog-ng is running chroot() in directory /data as user syslogng:sysadmins and listens on port 1514.  iptables redirects any incoming port 514 traffic to 1514.  The required /usr/local/ directories are mounted (-o bind) under /data.

syslog-ng 3.3.3
Installer-Version: 3.3.3
Revision:
ssh+git://bazsi@git.balabit//var/scm/git/syslog-ng/syslog-ng-ose--mainline--3.3#master#d199a1980be6b23fe24189e86a882812288e292c
Compile-Date: Dec  8 2011 17:46:40
Default-Modules:
affile,afprog,afsocket,afuser,basicfuncs,csvparser,dbparser,syslogformat,afsql
Available-Modules:
convertfuncs,affile,afmongodb,dummy,basicfuncs,csvparser,confgen,afsql,syslogformat,afuser,afsocket,afprog,afsocket-notls,dbparser
Enable-Debug: off
Enable-GProf: off
Enable-Memtrace: off
Enable-IPv6: on
Enable-Spoof-Source: off
Enable-TCP-Wrapper: on
Enable-Linux-Caps: off
Enable-Pcre: off

Config file:

@version: 3.3

source s_udp { udp(ip("0.0.0.0") port(1514)); };

destination file1 { file("/log/network.log" owner(syslogng)group(sysops) perm(0640) flags(no-multi-line)); };

destination NeDi { udp("192.168.0.7" port(514)); };

log { source(s_udp); destination(file1); };

# enabling the line below breaks logging to the file above

log { source(s_udp); destination(NeDi); };

Debug:


# /usr/local/sbin/syslog-ng --cfgfile=/usr/local/etc/syslog-ng.conf --chroot=/data --user=syslogng --group=sysadmins --persist-file=/log/syslog-ng.persist --foreground --process-mode=foreground --stderr --debug
nanosleep() is not accurate enough to introduce minor stalls on the reader side, multi-threaded performance may be affected;
Trying to open module; module='affile', filename='/usr/local/lib/syslog-ng/libaffile.so'
Trying to open module; module='afprog', filename='/usr/local/lib/syslog-ng/libafprog.so'
Trying to open module; module='afsocket', filename='/usr/local/lib/syslog-ng/libafsocket.so'
Trying to open module; module='afuser', filename='/usr/local/lib/syslog-ng/libafuser.so'
Trying to open module; module='basicfuncs', filename='/usr/local/lib/syslog-ng/libbasicfuncs.so'
Trying to open module; module='csvparser', filename='/usr/local/lib/syslog-ng/libcsvparser.so'
Trying to open module; module='dbparser', filename='/usr/local/lib/syslog-ng/libdbparser.so'
Trying to open module; module='syslogformat', filename='/usr/local/lib/syslog-ng/libsyslogformat.so'
Trying to open module; module='afsql', filename='/usr/local/lib/syslog-ng/libafsql.so'
Syslog connection established; fd='8', server='AF_INET(192.168.0.7:514)', local='AF_INET(0.0.0.0:0)'
Running application hooks; hook='1'
Running application hooks; hook='3'
syslog-ng starting up; version='3.3.3'
Incoming log entry; line='<189>41609: Dec  9 2011 08:41:22.691 UTC: %SYS-5-CONFIG_I: Configured from console by admin on vty0 (FD96:D107:BCF2::3)'
Incoming log entry; line='<189>Dec  9 08:41:24 6500-1 41609: Dec  9 2011 08:41:22.691 UTC: %SYS-5-CONFIG_I: Configured from console by admin on vty0 (FD96:D107:BCF2::3)\x0a'
Incoming log entry; line='<189>Dec  9 08:41:24 localhost 41609: Dec  9 2011 08:41:22.691 UTC: %SYS-5-CONFIG_I: Configured from console by admin on vty0 (FD96:D107:BCF2::3)\x0a'
Incoming log entry; line='<189>Dec  9 08:41:24 localhost 41609: Dec  9 2011 08:41:22.691 UTC: %SYS-5-CONFIG_I: Configured from console by admin on vty0 (FD96:D107:BCF2::3)\x0a'
Incoming log entry; line='<189>Dec  9 08:41:24 localhost 41609: Dec  9 2011 08:41:22.691 UTC: %SYS-5-CONFIG_I: Configured from console by admin on vty0 (FD96:D107:BCF2::3)\x0a'
....forever....