Andreas Kurz wrote:
On 10/18/07, Balazs Scheidler <bazsi@balabit.hu> wrote:
On Wed, 2007-10-17 at 10:42 -0700, Evan Rempel wrote:
We recently has an unscheduled power outage in our data center. Our servers came back prior to our DNS being available (actually, prior to the network coming back up). All hosts running syslog-ng consumed their log filespace as fast as the disks would allow writing which took about 2 minutes.
The problem we seem to have encountered is that our source section and destination definitions are;
source local { unix-stream("/dev/log" max-connections(200)); file("/proc/kmsg" log_prefix("kernel: ")); tcp( localip(127.0.0.1) port(514) ); internal(); };
destination syslogServer1 { tcp("syslog.uvic.ca" log_fifo_size(50000) ); };
It seems that if syslog.uvic.ca could not be resolved, syslog-ng took it upon itself to use 127.0.0.1 as its destination and started logging to itself. Chaining of hostnames is on, which means that we could see a message path of
local@myhost.uvic.ca/local@myhost.uvic.ca/local@myhost.uvic.ca/local@myhost.uvic.ca/local@myhost.uvic.ca
until some maximum length was reached and the hostname field became truncated.
This should be easy to repeat if you use a source like the one above, disconnect the network and start syslog-ng. I was already thinking about the idea of dropping internal messages generated while another internal message is being delivered to prevent such loops.
Just out of curiosity:
What is the intention of the line 'tcp( localip(127.0.0.1) port(514) );' ?
Which local logging messages are not processed if just using:
source local { unix-stream("/dev/log" max-connections(200)); file("/proc/kmsg" log_prefix("kernel: ")); internal(); };
We use multiple instances of syslog-ng on our systems to get logs from applications that can not syslog. We moved away from having the "system syslog" tail files and read pipes and placed this functionality into a separate syslog-ng that is started with and shutdown with the applications init scripts. These other instances of syslog-ng have NO WAY to "syslog" files as the author has declined the suggestion of having a "syslog" destination in syslog-ng (I know, I can write a patch :-). Anyhow, the only way to get data back into the "system syslog" which is where the internal messages need to go, is to have some supported destination. We have been using a syslog port, but I think we will be moving to a pipe for security reasons. I hope this is at least a little clear :-) -- Evan Rempel