[syslog-ng] Disk based queuing with rsyslog?

[PÁSZTOR György]

Hi,

"Gergely Nagy" <algernon at balabit.com> írta 2018-05-09 12:35-kor:
> >>>>> "Delon" == Delon Lee Di Lun <lee.delon2005 at gmail.com> writes:
> 
>     Delon> However, now. I see the documentation mention that flow control can be used
>     Delon> in conjunction with disk based buffering. How does flow control plays a
>     Delon> part?
> 
> The documentation has a chapter on flow-control, which explains the
> purpose, and the behaviour of it:
>  https://syslog-ng.com/documents/html/syslog-ng-ose-latest-guides/en/syslog-ng-ose-guide-admin/html/concepts-flow-control.html
> 
>     Delon> Flow control only works within the syslog-ng application? From the source
>     Delon> to dest? Or it has the capability to detect the flow rate from the remote
>     Delon> syslog-ng client and can "command" the syslog-ng client to slow down if the
>     Delon> syslog-ng server cannot handle the load?
> 
> Yes, it only works within a single syslog-ng instance. But if you have
> syslog-ng on both client and server, if flow-control kicks in on the
> server, and it slows down accepting messages, then flow-control will
> also kick in on the client (because it detects that messages are not
> going out fast enough), and it will start buffering (to disk too, if so
> configured). This way, you'll only lose messages if all queues fill up.

One extra thought to extend algernon's comment: to work this properly, you
must use tcp. If you use udp, the client will not have proper feedback if
the server couldn't process a message. In that case kernel's udp buffer
just silently drop the message.
In case of tcp: if the server process doesn't read (fast enough) from the
socket, then the underlying kernel won't send back the tcp acknowledments.

Cheers,
Gyu