<p dir="ltr">At least pls open an issue on github, that makes it easier to track and not forget.</p>
<p dir="ltr">Thanks</p>
<div class="gmail_quote">On Apr 30, 2016 6:38 AM, "Scheidler, Balázs" <<a href="mailto:balazs.scheidler@balabit.com">balazs.scheidler@balabit.com</a>> wrote:<br type="attribution"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><p dir="ltr">Thanks for diagnosing this. Lseek is used as it was resolving a performance issue on NFS on freebsd about a decade ago.</p>
<p dir="ltr">See this thread:</p>
<p dir="ltr"><a href="https://lists.balabit.hu/pipermail/syslog-ng/2008-May/011696.html" target="_blank">https://lists.balabit.hu/pipermail/syslog-ng/2008-May/011696.html</a></p>
<p dir="ltr">We should probably make the o_append mode default and only use seek+write on NFS.</p>
<p dir="ltr">Can you submit a patch to do this?<br>
Thanks</p>
<div class="gmail_quote">On Apr 29, 2016 4:17 PM, "Jim Segrave" <<a href="mailto:jes@j-e-s.net" target="_blank">jes@j-e-s.net</a>> wrote:<br type="attribution"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">On a set of very busy Usenet servers, we are seeing problems with using<br>
syslog-ng together with copytruncate in logrotation. I am not sure why,<br>
but syslog-ng opens log files without<br>
having O_APPEND set, and appends log mesages with separate syscalls - an<br>
lseek to SEEK_END, followed by a write. On a busy server- perhaps 300<br>
logs/sec each about 120 bytes long, we sometimes find that logrotate,<br>
which is configured to do a copy-truncate, actually produces a sparse<br>
file which begins with blocks of zeroes up to the pre-rotate log file size,<br>
followed by logs written after the copytruncate completes.<br>
<br>
Looking at the Linux /proc/PID/fdinfo information, I see that the<br>
logfiles are not opened in append mode, although they are opened O_WR. I<br>
haven't begun looking through the source to see if syslog-ng ever tries<br>
to overwrite data it's already logged by seeking backwards in the file -<br>
I must confess I can't think of any reason to do so. The only other<br>
reason for not having O_APPEND set would be to avoid problems for<br>
people using NFS to collect logs centrally by having different servers<br>
all writing to one NFS file.<br>
<br>
As I see it, the sequence leading to this problem is:<br>
<br>
syslog-ng logrotate copytruncate function<br>
------------- -------------------------------------------<br>
,,,<br>
begin copytruncate, reach EOF on the copy<br>
lseek(fd, SEEK_END)<br>
<------------------------------------------------------------- the end<br>
of the file before logrotate ftruncates() it<br>
<br>
ftruncate(fd, 0)<br>
write(fd, log_msg, log_msg_size)<br>
<----------------------------------------- the write takes place at the<br>
old EOF, so the kernel considers the file to have empty blocks from 0 to<br>
that point<br>
<br>
<br>
If there is a reason that some users need the logfile opened without<br>
O_APPEND mode, then making that choice a configuration option would be<br>
helpful. For those using O_APPEND, the lseek() calls are harmless and<br>
won't add any more resource usage than is already present, but file<br>
truncation will prevent a zero filled pad being prefixed to the file.<br>
<br>
<br>
<br>
______________________________________________________________________________<br>
Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" rel="noreferrer" target="_blank">https://lists.balabit.hu/mailman/listinfo/syslog-ng</a><br>
Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" rel="noreferrer" target="_blank">http://www.balabit.com/support/documentation/?product=syslog-ng</a><br>
FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" rel="noreferrer" target="_blank">http://www.balabit.com/wiki/syslog-ng-faq</a><br>
<br>
</blockquote></div>
<br>______________________________________________________________________________<br>
Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" rel="noreferrer" target="_blank">https://lists.balabit.hu/mailman/listinfo/syslog-ng</a><br>
Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" rel="noreferrer" target="_blank">http://www.balabit.com/support/documentation/?product=syslog-ng</a><br>
FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" rel="noreferrer" target="_blank">http://www.balabit.com/wiki/syslog-ng-faq</a><br>
<br>
<br></blockquote></div>