Please forgive me if this has already been addressed. If so will you point me to it?

Syslog-ng will all of a sudden, stop writing files.

 

I’m running syslog-ng 3.5.3

Installer-Version: 3.5.3

Revision: ssh+git://algernon@git.balabit/var/scm/git/syslog-ng/syslog-ng-ose--mainline--3.5#master#ccb05a22408ba4c837d998b2538854d994f845a5

Compile-Date: Jan  8 2014 13:35:02

Available-Modules: afsocket,afprog,dbparser,system-source,affile,syslogformat,linux-kmsg-format,csvparser,afmongodb,afsocket-tls,confgen,afuser,afstomp,afsocket-notls,basicfuncs,cryptofuncs,afamqp

Enable-Debug: off

Enable-GProf: off

Enable-Memtrace: off

Enable-IPv6: on

Enable-Spoof-Source: off

Enable-TCP-Wrapper: on

Enable-Linux-Caps: off

Enable-Pcre: on

 

 

The service status is showing running but not writing log files.  We are logging  udp from about 2400 devices

When it dies strace shows:

 

epoll_ctl(3, EPOLL_CTL_DEL, 10, {0, {u32=19726648, u64=19726648}}) = 0

write(110, "\1\0\0\0\0\0\0\0", 8)       = 8

epoll_wait(3, {{EPOLLIN, {u32=19641320, u64=19641320}}}, 11, 3414) = 1

read(6, "\1\0\0\0\0\0\0\0", 8)          = 8

futex(0x7fe1b000bd34, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x7fe1b000bd30, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1

futex(0x7fe21cfaad68, FUTEX_WAKE_PRIVATE, 1) = 1

fcntl(10, F_GETFD)                      = 0x1 (flags FD_CLOEXEC)

fcntl(10, F_GETFL)                      = 0x802 (flags O_RDWR|O_NONBLOCK)

setsockopt(10, SOL_SOCKET, SO_OOBINLINE, [1], 4) = 0

write(110, "\1\0\0\0\0\0\0\0", 8)       = 8

epoll_ctl(3, EPOLL_CTL_ADD, 10, {0, {u32=19726648, u64=19726648}}) = 0

epoll_wait(3, {{EPOLLIN, {u32=19641320, u64=19641320}}}, 12, 3413) = 1

read(6, "\1\0\0\0\0\0\0\0", 8)          = 8

futex(0x7fe1b000bd34, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x7fe1b000bd30, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1

futex(0x7fe21cfaad68, FUTEX_WAKE_PRIVATE, 1) = 1

futex(0x14e25a0, FUTEX_WAKE_PRIVATE, 1) = 1

epoll_wait(3, {{EPOLLIN, {u32=19641320, u64=19641320}}}, 12, 0) = 1

read(6, "\1\0\0\0\0\0\0\0", 8)          = 8

futex(0x7fe1b000bd34, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x7fe1b000bd30, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1

futex(0x7fe21cfaad68, FUTEX_WAKE_PRIVATE, 1) = 1

write(110, "\1\0\0\0\0\0\0\0", 8)       = 8

futex(0x12d02c0, FUTEX_WAIT_PRIVATE, 2, NULL) = ? ERESTARTSYS (To be restarted)

--- SIGTERM (Terminated) @ 0 (0) ---

write(17, "\1\0\0\0\0\0\0\0", 8)        = 8

rt_sigreturn(0x7fe21cfab740)            = 202

futex(0x12d02c0, FUTEX_WAIT_PRIVATE, 2, NULL <unfinished ...>

 

here is where I did a restart

+++ killed by SIGKILL +++

 

top - 12:45:56 up 133 days, 23:11, 13 users,  load average: 1.06, 1.13, 1.14

Tasks: 634 total,   2 running, 632 sleeping,   0 stopped,   0 zombie

Cpu(s):  4.2%us,  2.0%sy,  0.0%ni, 93.6%id,  0.0%wa,  0.0%hi,  0.2%si,  0.0%s

Mem:  32898840k total, 31285296k used,  1613544k free,   128188k buffers

Swap: 16777212k total,   684800k used, 16092412k free, 29249028k cached

 

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND

1631 root      20   0 1725m 104m 2892 S 28.4  0.3  10:38.46 syslog-ng        2843 root      20   0 1725m 104m 2892 S 17.5  0.3   0:04.35 syslog-ng

2795 root      20   0 1725m 104m 2892 S 15.8  0.3   0:11.99 syslog-ng        2842 root      20   0 1725m 104m 2892 S 13.9  0.3   0:02.68 syslog-ng

2793 root      20   0 1725m 104m 2892 S 13.5  0.3   0:14.54 syslog-ng        2855 root      20   0 1725m 104m 2892 R 13.5  0.3   0:00.41 syslog-ng

2776 root      20   0 1725m 104m 2892 S 12.2  0.3   0:18.57 syslog-ng       43203 root      20   0  359m 101m  10m S 11.9  0.3  15:35.10 splunkd

2794 root      20   0 1725m 104m 2892 S  9.6  0.3   0:14.62 syslog-ng        2791 root      20   0 1725m 104m 2892 S  9.2  0.3   0:11.89 syslog-ng

2697 root      20   0 1725m 104m 2892 S  6.3  0.3   0:31.74 syslog-ng       43204 root      20   0  359m 101m  10m S  4.9  0.3   8:01.72 splunkd

2825 root      20   0 1725m 104m 2892 S  2.3  0.3   0:07.73 syslog-ng        2841 root      20   0 1725m 104m 2892 S  1.6  0.3   0:03.30 syslog-ng

 

Also, one other problem I have is the syslog-ng log file says:

Aug 26 11:48:49 sopher1 syslog-ng[488]: Input is valid utf8, but the log message is not tagged as such, this performs worse than enabling validate-utf8 flag on input; value='758AARULOCAL01'

 

My config specifies flags(validate-utf8):

 

source s_udp { udp( port(514) so_rcvbuf(15000000) log_iw_size(50000) log_msg_size(65535) log_fetch_limit(50000) flags(validate-utf8));};

Frank Wilkinson

(205)934-3540 w