[syslog-ng] really need help...not writing files

Tue Aug 26 19:48:07 CEST 2014

Please forgive me if this has already been addressed. If so will you point me to it?
Syslog-ng will all of a sudden, stop writing files.

I'm running syslog-ng 3.5.3
Installer-Version: 3.5.3
Revision: ssh+git://algernon@git.balabit/var/scm/git/syslog-ng/syslog-ng-ose--mainline--3.5#master#ccb05a22408ba4c837d998b2538854d994f845a5
Compile-Date: Jan  8 2014 13:35:02
Available-Modules: afsocket,afprog,dbparser,system-source,affile,syslogformat,linux-kmsg-format,csvparser,afmongodb,afsocket-tls,confgen,afuser,afstomp,afsocket-notls,basicfuncs,cryptofuncs,afamqp
Enable-Debug: off
Enable-GProf: off
Enable-Memtrace: off
Enable-IPv6: on
Enable-Spoof-Source: off
Enable-TCP-Wrapper: on
Enable-Linux-Caps: off
Enable-Pcre: on

The service status is showing running but not writing log files.  We are logging  udp from about 2400 devices
When it dies strace shows:

epoll_ctl(3, EPOLL_CTL_DEL, 10, {0, {u32=19726648, u64=19726648}}) = 0
write(110, "\1\0\0\0\0\0\0\0", 8)       = 8
epoll_wait(3, {{EPOLLIN, {u32=19641320, u64=19641320}}}, 11, 3414) = 1
read(6, "\1\0\0\0\0\0\0\0", 8)          = 8
futex(0x7fe1b000bd34, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x7fe1b000bd30, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1
futex(0x7fe21cfaad68, FUTEX_WAKE_PRIVATE, 1) = 1
fcntl(10, F_GETFD)                      = 0x1 (flags FD_CLOEXEC)
fcntl(10, F_GETFL)                      = 0x802 (flags O_RDWR|O_NONBLOCK)
setsockopt(10, SOL_SOCKET, SO_OOBINLINE, [1], 4) = 0
write(110, "\1\0\0\0\0\0\0\0", 8)       = 8
epoll_ctl(3, EPOLL_CTL_ADD, 10, {0, {u32=19726648, u64=19726648}}) = 0
epoll_wait(3, {{EPOLLIN, {u32=19641320, u64=19641320}}}, 12, 3413) = 1
read(6, "\1\0\0\0\0\0\0\0", 8)          = 8
futex(0x7fe1b000bd34, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x7fe1b000bd30, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1
futex(0x7fe21cfaad68, FUTEX_WAKE_PRIVATE, 1) = 1
futex(0x14e25a0, FUTEX_WAKE_PRIVATE, 1) = 1
epoll_wait(3, {{EPOLLIN, {u32=19641320, u64=19641320}}}, 12, 0) = 1
read(6, "\1\0\0\0\0\0\0\0", 8)          = 8
futex(0x7fe1b000bd34, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x7fe1b000bd30, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1
futex(0x7fe21cfaad68, FUTEX_WAKE_PRIVATE, 1) = 1
write(110, "\1\0\0\0\0\0\0\0", 8)       = 8
futex(0x12d02c0, FUTEX_WAIT_PRIVATE, 2, NULL) = ? ERESTARTSYS (To be restarted)
--- SIGTERM (Terminated) @ 0 (0) ---
write(17, "\1\0\0\0\0\0\0\0", 8)        = 8
rt_sigreturn(0x7fe21cfab740)            = 202
futex(0x12d02c0, FUTEX_WAIT_PRIVATE, 2, NULL <unfinished ...>

here is where I did a restart
+++ killed by SIGKILL +++

top - 12:45:56 up 133 days, 23:11, 13 users,  load average: 1.06, 1.13, 1.14
Tasks: 634 total,   2 running, 632 sleeping,   0 stopped,   0 zombie
Cpu(s):  4.2%us,  2.0%sy,  0.0%ni, 93.6%id,  0.0%wa,  0.0%hi,  0.2%si,  0.0%s
Mem:  32898840k total, 31285296k used,  1613544k free,   128188k buffers
Swap: 16777212k total,   684800k used, 16092412k free, 29249028k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
1631 root      20   0 1725m 104m 2892 S 28.4  0.3  10:38.46 syslog-ng        2843 root      20   0 1725m 104m 2892 S 17.5  0.3   0:04.35 syslog-ng
2795 root      20   0 1725m 104m 2892 S 15.8  0.3   0:11.99 syslog-ng        2842 root      20   0 1725m 104m 2892 S 13.9  0.3   0:02.68 syslog-ng
2793 root      20   0 1725m 104m 2892 S 13.5  0.3   0:14.54 syslog-ng        2855 root      20   0 1725m 104m 2892 R 13.5  0.3   0:00.41 syslog-ng
2776 root      20   0 1725m 104m 2892 S 12.2  0.3   0:18.57 syslog-ng       43203 root      20   0  359m 101m  10m S 11.9  0.3  15:35.10 splunkd
2794 root      20   0 1725m 104m 2892 S  9.6  0.3   0:14.62 syslog-ng        2791 root      20   0 1725m 104m 2892 S  9.2  0.3   0:11.89 syslog-ng
2697 root      20   0 1725m 104m 2892 S  6.3  0.3   0:31.74 syslog-ng       43204 root      20   0  359m 101m  10m S  4.9  0.3   8:01.72 splunkd
2825 root      20   0 1725m 104m 2892 S  2.3  0.3   0:07.73 syslog-ng        2841 root      20   0 1725m 104m 2892 S  1.6  0.3   0:03.30 syslog-ng ...

Also, one other problem I have is the syslog-ng log file says:
Aug 26 11:48:49 sopher1 syslog-ng[488]: Input is valid utf8, but the log message is not tagged as such, this performs worse than enabling validate-utf8 flag on input; value='758AARULOCAL01'

My config specifies flags(validate-utf8):

source s_udp { udp( port(514) so_rcvbuf(15000000) log_iw_size(50000) log_msg_size(65535) log_fetch_limit(50000) flags(validate-utf8));};
Frank Wilkinson
(205)934-3540 w

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.balabit.hu/pipermail/syslog-ng/attachments/20140826/6aada582/attachment-0001.htm 