lseek before write will work as long as: 1. syslog-ng isn¹t multi-threaded 2. there aren¹t multiple syslog-ng¹s running on the same server pointing at the same file I tested that out (turning off APPEND and adding an lseek). NFS performance still great. I modified my conf to point all log messages to a single file (well configured multiple destinations to point to the same file) and things seemed to work fine over NFS no mangling of messages. So with limited testing seems to be a good solution. thanks. On 5/29/08 12:35 PM, "Balazs Scheidler" <bazsi@balabit.hu> wrote:
On Thu, 2008-05-29 at 11:07 -0700, Chris Wagner wrote:
We run syslog-ng (both 1.x and now 2.0.9) on FreeBSD writing log files to a NetApp NFS server. At a very low load we start dropping inbound packets (udp, large socket buffers). A quick tcpdump showed a pathological interaction with NFS revolving around how NFS (at least on BSD) deals with APPEND mode. The upshot is that for each write, it flushes the attributes and actually re-reads the partial file block back from the server!
ouch. that sounds bad.
A simple change to the source to remove O_APPEND had great results we can handle 10x the number of inbound messages.
Our .conf file has every logging host going to a separate file, and with a quick scan of the code it sure looked like syslog-ng won¹t open the same file more than once anyway (I could be wrong here).
It depends on the configuration, if you configure the same file multiple times, syslog-ng will open them multiple times. O_APPEND ensures that it properly writes to the end of file all times.
What do you think, would an lseek + write combination solve the issue? e.g. open without O_APPEND, and use lseek(fd, 0, SEEK_END) before calling write().
So, a couple questions (I scanned the last few months of archives but didn¹t see any discussion about this) - what breaks if anything by turning off APPEND, is this interesting to others/how might I get this into future releases ( I can see this being a per-destination option).
Sure, I'm interested in performance corner cases like this. The only question is how to solve it without affecting others.
-- Bazsi
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html
-- Debugging is twice as hard as writing code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it. Brian Kernighan