[syslog-ng] grouping-by parser questions

[Scheidler, Balázs]

Hi,

On Jun 6, 2016 11:17 AM, "Fekete, Róbert" <robert.fekete at balabit.com> wrote:
>
> Hi Bazsi,
>
> I've started to document the grouping-by parser, and have a few
questions/comments about it:
>
> * It seems that some of the grouping-by options are the same (or very
similar) to the correlation-related attributes of the pattern database, but
have different names. Could we name them consistently where they are the
same? (I haven't checked the correlation module from Rust, but maybe we
could align that as well.)
> For example:
> grouping-by  |  patterndb
>   scope          | context-scope
>   timeout       | context-timeout
>   aggregate   | message or action
>

I omitted the "context" prefix on purpose, they are important in the
patterndb context as rules have correllation and non correllation related
groups of options. With groupingby it would be kind of redundant.

I was thinking on aggregate() a lot, and decided to use something that is
closer to the "groupingby" term, group by in SQL works with aggregate
functions, in a sense they produce aggregates over various dimensions. In
patterndb, you can generate multiple actions for a rule.

Anyway, naming should probably be discussed in person.

>
>   * In the original commit message, you mention three possible values for
the 'scope' option, whereas the context-scope in the patterndb has four
(program). Are these deliberately different, or they use the same code?

They use the same code, so it should be the same

>
>  * grouping-by doesn't look to me as an actual parser. From the existing
objects, it resembles a filter more (IMHO), but I'd rather categorize it as
something else that transforms/processes the incoming data, and should be
therefore in a separate configuration object (along with the geoip parser).

I agree, we currently only have parsers/rewrite/filter stuff only. It might
make sense to create a more generalized concept though. I am not sure it is
worth it, but we already had similar usecases where we couldnt categorize
some kind of functionality. But let's test whether we can find a
descriptive name for it. How would you call "generic processing" in the
config?

Btw, I stand by my decision that it is not a filter, it never drops
messages, whereas the primary function of filters is to drop messages.

>
> Robert
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.balabit.hu/pipermail/syslog-ng/attachments/20160606/1b3b7c11/attachment.htm