Hi Balaza, Thank you so much for helping. My end target is a boot CF card which is a SATA device operating in AHCI mode (ATA device). The log file target in the syslog configuration is writing's on this device. What is the best option in my case to control the sending rate of writing to the file? During heavy writing into the log, the device driver on the CF card could not handle it and eventually the ATA link was broken. So I am trying to control the rate of message writing into the file. Can you please suggest the best way to limit the rate of messages writing into the file with syslog-ng. Thank you once again for your time and helping. Regards Simon On Sun, Feb 7, 2021 at 1:46 AM Balazs Scheidler <bazsi77@gmail.com> wrote:
Hi,
On Sun, Feb 7, 2021, 07:18 SIMON BABY <simonkbaby@gmail.com> wrote:
Hello Team,
I am new to this group and I have a query on adding the throttle attribute in syslog-ng.conf file. My target is little slow to process all the messages sent by the sender and sometimes the link connected to the target device is broken. I am thinking of slowing down the sender by adding the throttle attribute. I have the below queries:
1) What exactly the throttle confoguration does?
It limits the number of syslog messages to be sent to the device per second.
The implementation allows short spikes of traffic where the short term rate is higher, but over a few second the average stabilizes to the value specified.
It uses a tbf like algorithm.
2) What does throttle(500) mean ? will it send 500 Bytes per second or 500
messages per second? What does the message here mean ? Can it be the entire message sent by the application ? Is there an upper limit and lower limit ?
500 messages, not bytes. The maximum message size can be controlled using the log-msg-size() option.
3) Any side effect of my system if I am going to use throttle().
If your input rate is higher than the output, syslog-ng would either need to store the incoming messages (memory or disk), backpressure to the source if possible (using flow control and a tcp based transport) or drop them.
There are a number of options that control this behavior.
flags(flow-control) to turn on flow control on a log path
log-fifo-size() for controlling the memory buffer size at the destination
disk-buffer() for allowing the excess to overflow to disk
transport(tcp) or transport(tls) on the source to select the transport protocol
4) Any other method in syslog-ng to delay the logs sending at the sender?
Depending on your use-case a flow controlled path end-to-end (application, client-syslog-ng, server syslog-ng, final destination) could work too. In that case, syslog-ng would automatically converge to the amount of messages the destination is able to consume.
5) My destination configuration is below. Is it a valid configuration ?
destination logFiler { file("/var/log/wq.log"
template("${FULLDATE}${TZ} ${HOST} ${PROGRAM} [$LEVEL] ${MSG}\n")
template_escape(yes)
throttle(500));};
This is, however this is a file() destination, where the throttle option may have limited use.
Thank you for your time.
Regards Simon
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq