[syslog-ng] losing msgs while writing to unix pipe

Ramesh Uppuluri ruppuluri at netcordia.com
Tue Feb 20 16:28:55 CET 2007


I configured syslog-ng to use the template to generate mysql INSERT 
statements and write to a
unix pipe. I am using a shell script that reads from this pipe and 
replays these messages to mysql.
In normal load conditions all the messages are making it to the db. It 
even worked fine when
I pumped in 100msgs/sec. However when I sent a burst of 500msgs only 
around 180msgs are
making it through. When I try the same rate writing to a text file (not 
using unix pipe), all the msgs
make it through. so, it looks like the msgs are getting dropped due to 
the pipe overflow. Did anybody
encounter the same problem? If so, do you have any suggestions to 
overcome this problem?

When I replayed the INSERT commands using unix cat to the pipe and write 
it to the db using the
same script, I was able to write 2500rows/sec. I thing cat call blocks. 
so, is the problem related to
how syslong-ng writes to the pipe?

thanks
ramesh



More information about the syslog-ng mailing list