<HTML>
<HEAD>
<TITLE>Re: [syslog-ng] syslog_ng and NFS</TITLE>
</HEAD>
<BODY>
<FONT FACE="Calibri, Verdana, Helvetica, Arial"><SPAN STYLE='font-size:11pt'>lseek before write will work as long as:<BR>
<BR>
</SPAN></FONT><OL><LI><FONT FACE="Calibri, Verdana, Helvetica, Arial"><SPAN STYLE='font-size:11pt'>syslog-ng isn’t multi-threaded
</SPAN></FONT><LI><FONT FACE="Calibri, Verdana, Helvetica, Arial"><SPAN STYLE='font-size:11pt'>there aren’t multiple syslog-ng’s running on the same server pointing at the same file<BR>
</SPAN></FONT></OL><FONT FACE="Calibri, Verdana, Helvetica, Arial"><SPAN STYLE='font-size:11pt'><BR>
I tested that out (turning off APPEND and adding an lseek). NFS performance still great. I modified my conf to point all log messages to a single file (well configured multiple destinations to point to the same file) and things seemed to work fine over NFS – no mangling of messages. So with limited testing – seems to be a good solution.<BR>
<BR>
thanks.<BR>
<BR>
<BR>
<BR>
<BR>
On 5/29/08 12:35 PM, "Balazs Scheidler" <<a href="bazsi@balabit.hu">bazsi@balabit.hu</a>> wrote:<BR>
<BR>
</SPAN></FONT><BLOCKQUOTE><FONT FACE="Calibri, Verdana, Helvetica, Arial"><SPAN STYLE='font-size:11pt'>On Thu, 2008-05-29 at 11:07 -0700, Chris Wagner wrote:<BR>
> We run syslog-ng (both 1.x and now 2.0.9) on FreeBSD writing log files<BR>
> to a NetApp NFS server. At a very low load we start dropping inbound<BR>
> packets (udp, large socket buffers). A quick tcpdump showed a<BR>
> pathological interaction with NFS revolving around how NFS (at least<BR>
> on BSD) deals with APPEND mode. The upshot is that for each write, it<BR>
> flushes the attributes and actually re-reads the partial file block<BR>
> back from the server!<BR>
<BR>
ouch. that sounds bad.<BR>
<BR>
><BR>
> A simple change to the source – to remove O_APPEND had great results –<BR>
> we can handle 10x the number of inbound messages.<BR>
><BR>
> Our .conf file has every logging host going to a separate file, and<BR>
> with a quick scan of the code it sure looked like syslog-ng won’t open<BR>
> the same file more than once anyway (I could be wrong here).<BR>
<BR>
It depends on the configuration, if you configure the same file multiple<BR>
times, syslog-ng will open them multiple times. O_APPEND ensures that it<BR>
properly writes to the end of file all times.<BR>
<BR>
What do you think, would an lseek + write combination solve the issue?<BR>
e.g. open without O_APPEND, and use lseek(fd, 0, SEEK_END) before<BR>
calling write().<BR>
<BR>
> So, a couple questions (I scanned the last few months of archives but<BR>
> didn’t see any discussion about this) - what breaks if anything by<BR>
> turning off APPEND, is this interesting to others/how might I get this<BR>
> into future releases ( I can see this being a per-destination option).<BR>
<BR>
Sure, I'm interested in performance corner cases like this. The only<BR>
question is how to solve it without affecting others.<BR>
<BR>
--<BR>
Bazsi<BR>
<BR>
<BR>
______________________________________________________________________________<BR>
Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng">https://lists.balabit.hu/mailman/listinfo/syslog-ng</a><BR>
Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng">http://www.balabit.com/support/documentation/?product=syslog-ng</a><BR>
FAQ: <a href="http://www.campin.net/syslog-ng/faq.html">http://www.campin.net/syslog-ng/faq.html</a><BR>
<BR>
<BR>
</SPAN></FONT></BLOCKQUOTE><FONT FACE="Calibri, Verdana, Helvetica, Arial"><SPAN STYLE='font-size:11pt'><BR>
<BR>
-- <BR>
Debugging is twice as hard as writing code in the first place.<BR>
Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it.<BR>
Brian Kernighan <BR>
</SPAN></FONT>
</BODY>
</HTML>