[syslog-ng] disk based buffering

[Szalai, Attila]

The trick is (if I remember correctly) that even in reliable disk buffer, the content of the buffer is not _flushed_ to the disk with every write. It is (and this is maybe only in my fading memories) a memory mapped file and that is the kernel responsibility to synchronize it. This is why it not safe to kernel crashes. But if you stop the syslog-ng, the content should appear in the queue.

From: syslog-ng-bounces at lists.balabit.hu [mailto:syslog-ng-bounces at lists.balabit.hu] On Behalf Of Scheidler, Balázs
Sent: Wednesday, September 28, 2016 12:48 PM
To: Fabien Wernli; Syslog-ng users' and developers' mailing list
Subject: Re: [syslog-ng] disk based buffering

log-qout-size() controls the size of the memory queue that sits next to the disk buffer, and it defaults to 64 elements. However it shouldn't apply to reliable disk buffers (e.g. reliable() setting), because in that case everything touches the disk before syslog-ng would be sending them on.
Obviously reliable(yes) is a lot slower.
Bazsi



--
Bazsi

On Wed, Sep 28, 2016 at 9:02 AM, Fabien Wernli <wernli at in2p3.fr<mailto:wernli at in2p3.fr>> wrote:
On Wed, Sep 28, 2016 at 03:02:19AM +0200, thejaguar at tutanota.de<mailto:thejaguar at tutanota.de> wrote:
> So with dqtool ( cool) I noticed that actual problem is that syslog-ng is not writing to disk queue immediately when a message is generated.  The message goes to usual place like var/log/message etc but not in the queue where it should because it has not been transmitted to remote host.  Now I am thinking there has to be a memory limit before it starts flushing/dumping unsent message to disk queue ? I tried mem-buf-size(1) and log-fifo-size(1) but none of them works. Is it a static value or is it configurable ?

If I'm not mistaken the message will go to the destination's memory queue
first; if the latter is full, only then will it make it to the diskq.

You can check the status of the queue using the control socket
(syslog-ng-ctl stats)

______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq



________________________________

NOTICE: Morgan Stanley is not acting as a municipal advisor and the opinions or views contained herein are not intended to be, and do not constitute, advice within the meaning of Section 975 of the Dodd-Frank Wall Street Reform and Consumer Protection Act. If you have received this communication in error, please destroy all electronic and paper copies and notify the sender immediately. Mistransmission is not intended to waive confidentiality or privilege. Morgan Stanley reserves the right, to the extent permitted under applicable law, to monitor electronic communications. This message is subject to terms available at the following link: http://www.morganstanley.com/disclaimers  If you cannot access these links, please notify us by reply message and we will send the contents to you. By communicating with Morgan Stanley you consent to the foregoing and to the voice recording of conversations with personnel of Morgan Stanley.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.balabit.hu/pipermail/syslog-ng/attachments/20160928/531f43f4/attachment-0001.htm