[syslog-ng] query on using throttle in syslog-ng.conf file

[Balazs Scheidler]

Hi,

On Sun, Feb 7, 2021, 07:18 SIMON BABY <simonkbaby at gmail.com> wrote:

> Hello Team,
>
> I am new to this group and I have a query on adding the throttle attribute
> in syslog-ng.conf file. My target is little slow to process all the
> messages sent by the sender and sometimes  the link connected to the target
> device is broken. I am thinking of slowing down the sender by adding the
> throttle attribute.
> I have the below queries:
>
> 1) What exactly the throttle confoguration does?
>

It limits the number of syslog messages to be sent to the device per second.

The implementation allows short spikes of traffic where the short term rate
is higher, but over a few second the average stabilizes to the value
specified.

It uses a tbf like algorithm.


2) What does throttle(500) mean ? will it send 500 Bytes per second or 500
> messages per second? What does the message here mean ? Can it be the entire
> message sent by the application ? Is there an upper limit and lower limit ?
>

500 messages, not bytes. The maximum message size can be controlled using
the log-msg-size() option.


3) Any side effect of my system if I am going to use throttle().
>

If your input rate is higher than the output, syslog-ng would either need
to store the incoming messages (memory or disk), backpressure to the source
if possible (using flow control and a tcp based transport) or drop them.

There are a number of options that control this behavior.

flags(flow-control) to turn on flow control on a log path

log-fifo-size() for controlling the memory buffer size at the destination

disk-buffer() for allowing the excess to overflow to disk

transport(tcp) or transport(tls) on the source to select the transport
protocol


4) Any other method  in syslog-ng to delay the logs sending at the sender?
>

Depending on your use-case a flow controlled path end-to-end (application,
client-syslog-ng, server syslog-ng, final destination) could work too. In
that case, syslog-ng would automatically converge to the amount of messages
the destination is able to consume.

5) My destination configuration is below. Is it a valid configuration ?
>
>
> destination logFiler { file("/var/log/wq.log"
>
>     template("${FULLDATE}${TZ} ${HOST} ${PROGRAM} [$LEVEL] ${MSG}\n")
>
>     template_escape(yes)
>
>     throttle(500));};
>

This is, however this is a file() destination, where the throttle option
may have limited use.


>
> Thank you for your time.
>
> Regards
> Simon
>
>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation:
> http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20210207/83a2316c/attachment.html>