[syslog-ng] Request for config help

[Pradeep Gotaparthi]

Hi folks,

It would be really helpful if any of you are able to help with the below
query.

Thanks!

On Sat, Jun 5, 2021 at 12:56 AM Pradeep Gotaparthi <pradeep.gp at gmail.com>
wrote:

> Hi Balazs,
>
> Sorry, couldnt get back to you earlier.  Below is a sample of config
>
> destination d_msg_04 {
>     tcp(
>         "172.49.20.136"
>         port(2193)
>         # log-fifo-size should be atleast max-connections * log-fetch-limit
>         # 500 * 10 (default value of log-fetch-limit)
>         log-fifo-size(5000)
>         # throttle to max 5k logs
>         throttle(5000)
>         disk-buffer(
>             # number of bytes to store in memory
>             mem-buf-size(10000)
>             # number of bytes to store on disk
>             disk-buf-size(134217728000) # 100GB
>
>             reliable(yes)
>             # directory location to persist messages
>             dir("/data/store")
>         )
>         persist-name(d_msg_04)
>     );
> };
>
> The throttle value as you can see is set to 5000.  Throttling is an
> important part of using syslog-ng for us here and it does a good job of
> it.   The exception is when there is a backlog built up on syslog-ng (it
> received a lot more logs than it could push out honoring the throttle
> limit),  and we stop or restart syslog-ng.   At that time, it appears to be
> "flushing out" whatever messages it has in its buffer and disregard the
> throttle limit at this time.     This is causing issues for us.
>
> So, wanted to understand if there is a way to prevent this.  Tried
> tinkering with the log-fifo-size thinking it may prevent too many messages
> in the buffer and there by help with this problem but it didnt help.
>
> It would be great if you could help us with this.
>
>
> Thanks !
>
>
>
>
> On Wed, Apr 21, 2021 at 10:29 AM Balazs Scheidler <bazsi77 at gmail.com>
> wrote:
>
>> Hi,
>>
>> I am not sure I understand your problem. Can you elaborate it a bit?
>> Configuration, the problem as you see it and your expectations would help a
>> lot to respond to your question.
>>
>> On Wed, Apr 21, 2021, 02:50 Pradeep Gotaparthi <pradeep.gp at gmail.com>
>> wrote:
>>
>>> Hi,
>>>
>>> Could anybody help us out with the below query?
>>>
>>>
>>> Thanks.
>>>
>>> On Fri, Mar 26, 2021 at 12:06 AM Pradeep Gotaparthi <
>>> pradeep.gp at gmail.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> We have been using syslog-ng for over a year now and it is working
>>>> great, thanks!
>>>>
>>>> One issue we face is that when we stop the process when there is some
>>>> buffer, the threshold is ignored and a bunch of logs get pushed out (as
>>>> much as 10 times the threshold value).   This is causing problems on the
>>>> receiving end as it can't handle the load.  We thought the log-fifo-size
>>>> parameter can help as syslog uses this parameter to store logs in memory
>>>> but it is not helping.
>>>>
>>>> It would be great if you can guide us.
>>>>
>>>>
>>>> Thanks.
>>>>
>>>
>>> ______________________________________________________________________________
>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>> Documentation:
>>> http://www.balabit.com/support/documentation/?product=syslog-ng
>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>>
>>>
>> ______________________________________________________________________________
>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>> Documentation:
>> http://www.balabit.com/support/documentation/?product=syslog-ng
>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20210614/b9d4610c/attachment.html>