[syslog-ng] [GSoC] Interested in Python support for Correlation Library

[Tibor Benke]

Hi,

See my answers inline.

> - While the reason to switch away from handlebars makes sense - Why do we
> want to using Python as templating language? What are some of the use
> cases, where Python could be beneficial over any other templating language?
>

Users can freely write their own functions. We can provide some functions
either in Rust or Python (it should be transparent to the users), but we
can't cover everything. I know some people who would definitely write their
own extensions.

And the Python language is a way more powerfool tool than a simple
templating language.

- Are there any existing projects using Python as templating language? How
> is this going to be different than passing the context data to a Python
> script?
>

Maybe the "templating language" term I used in the project description was
a little bit ambigious. I'd like to use Python as a general purpose
language to process the context data and generate a new log message from it.

I think embedding Python (or Lua) into softwares written in other languages
is quite common.

I'd prefer this [0] library for the Rust <-> Python interaction. I used it
for a side project [1], it's quite flexible and actively
developed/maintained. You can check the examples in the documentation and
the tests in the source code.

[0] https://github.com/dgrunwald/rust-cpython
[1] https://github.com/ihrwein/python-parser

I'd like to see the final solution to be "composable". I mean, by providing
small, reusable components users can compose the generated log message as
they want. For example, the following line can generate a log message:

logmsg(ctx, "context name is {context.name}, max(foo)={max_foo}",
foo="bar", quz="baz", max_foo=max(ctx, "foo"))

where ctx contains the context data, logmsg and max is a python/Rust
function. The syntax isn't settled down :)


>
- For the SQL-like syntax mentioned in project description - Can we design
> it as Python library using Rust via CFFI [0].
>
see my previous answer


> I have filled the student application form. I'm already trying to
> understand the correlation library and playing around with existing tests.
> How can I get more involved and better understand the requirements of the
> project?
>

First, I'd check what information is available in the contexts (your
context data):
https://github.com/ihrwein/correlation/blob/master/src/config/action/message/renderer_context.rs#L36-L48

You you can use {{context_len}} to get the number of messages in the
context, etc...

IMHO the hardest part is using the [0] libray. It's well designed, but if
you are new to Rust you may find it difficult to use. So I'd try that
first. I don't know about any bugs in the correlation library, but you may
try to refactor some code (just to see the internals).

The requirements are listed in the idea page as "Tasks".

BR,
Tibor
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.balabit.hu/pipermail/syslog-ng/attachments/20160320/7403d7ba/attachment.htm