[syslog-ng] Need clarification on different behavior of forwarding rules in syslog-ng

[Scheidler, Balázs]

What is your syslog-ng version? Pretty old versions (I think before 3.2?),
syslog-ng buffered output messages until we've reached the flush-lines()
setting, unless the time specified in flush-timeout()  has elapsed.

This is not the current behavior anymore. Maybe you have an old version?
You say, that you have 400 events per second, so flush-lines(100) should be
4 times every second.

Anyhow, the version of syslog-ng you are using would be pretty important to
have.

-- 
Bazsi

On Wed, Apr 18, 2018 at 10:06 AM, Kavita Mohite (kmohite) <kmohite at cisco.com
> wrote:

> Hello Support Team,
>
>
>
> In the syslog-ng configuration, we have 2 forward rules as follows:
>
>
>
> 1. One rule reads from source (port 514) and writes syslogs to the file
> for all the syslogs greater than debug.
>
> 2. Second rule reads from source (port 514) and forwards the the syslog
> to the destination if a set of filter rules match. These set of filter
> rules are called whitelist filter rules are filter rules containing match
> filter and priority filter in each rule. As there are multiple filter
> rules, they are ORed to create a main filter rule.
>
>
>
> Now we have syslogs coming in at a rate of  400 eps. Following is our
> observation:
>
>
>
> A. If the syslog matches the filter in point 2 above, then following
> happens
>
> ·  It is forwarded to the destination provided in point 2.
>
> ·   Also at the same time it is written to a file as per rule in point 1.
>
>
>
> B. If the syslog does not match the filter in point 2 above, then
> following happens
>
> ·  It is NOT forwarded to the destination provided in point 2 as there is
> no match.
>
> ·  It is suppose to still write to file as the rule in 1 matches. *But
> this does not happen.*
>
>
>
> We are seeing that if the filter in point 2 matches then both the forward
> rules get executed immediately and we see the log written to file also
> immediately. But in case of B, if the syslog does not match we are seeing
> that the write to file is happening g in bulk for 30 minutes. *Why is
> this ? Why does it not write to file immediately in case of (B) but does it
> in case of A ?*
>
>
>
> Below is the snap of the config values we are using in the syslog-ng.conf
>
>
>
> options {
>
>         flush_lines (100);
>
>         log_fetch_limit(1000);
>
>         log_iw_size(10000);
>
>         time_reopen (10);
>
>         log_fifo_size (10000);
>
>         chain_hostnames (off);
>
>         use_dns (no);
>
>         use_fqdn (no);
>
>         create_dirs (yes);
>
>         keep_hostname (yes);
>
>         threaded(yes);
>
> };
>
> Thanks
>
> Kavita
>
> ____________________________________________________________
> __________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?
> product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20180418/ce937c60/attachment-0001.html>