[syslog-ng] syslog-ng 3.19 not processing /dev/log on RH#L6

[Evan Rempel]

We just had a network even in our environment that resulted in many hosts being isolated on the network in some very odd ways.
The hosts had some connections inbound, but no outbound and the network seemed to come and go very quickly.


syslog-ng detected that it lost connection to its down-stream syslog collectors a did the normal connection reaping. We have this set for 5 seconds.
At some point the syslog-ng process terminated and the supervisor process launched a new child. This worked well, but in
this odd senario this happened quickly and at on point the supervisor process failed to reap one of its defunct children but still launched a new child
process. The defunct process seemed to hold onto the /dev/log handle so the new child could not get access to it and the result is that
we lost hours of logs that flow through the /dev/log socket.
When we discovered the issue, there where two children of the supervisor process. One sas in a "defunct" state while the other was running,
but only logging kernel messages (from /dev/kmsg) and from the internal syslog-ng source.
I suspect that the child reaping process on SIGCHLD does not use a while loop and misses some children.

Evan.