[syslog-ng] [GSoC] Interested in Python support for Correlation Library

[Akshay Aurora]

Hi Tibor,

Thanks for replying, I'll try and get my hands dirty with rust-cpython.

- As a user looking for a solution for correlating messages, Python as
template language definitely makes a lot of sense. It allows us to offer a
more complete solution - allowing user to write their own utilities while
we can also provide a good helper library piggybacking on Rust.

- Whats you take on a generic script solution that takes context
information as arguments, this allows user to use language of his choice.
Is performance the only concern here? This shouldn't be very difficult to
implement.

Are we expecting, users to still write python code (template) in JSON? It
might get ugly defining functions within JSON. We could probably consider
giving users config option to include their Python module.

ᐧ

On Sun, Mar 20, 2016 at 8:01 PM, Tibor Benke <ihrwein at gmail.com> wrote:

> Hi,
>
> See my answers inline.
>
>> - While the reason to switch away from handlebars makes sense - Why do we
>> want to using Python as templating language? What are some of the use
>> cases, where Python could be beneficial over any other templating language?
>>
>
> Users can freely write their own functions. We can provide some functions
> either in Rust or Python (it should be transparent to the users), but we
> can't cover everything. I know some people who would definitely write their
> own extensions.
>
> And the Python language is a way more powerfool tool than a simple
> templating language.
>
> - Are there any existing projects using Python as templating language? How
>> is this going to be different than passing the context data to a Python
>> script?
>>
>
> Maybe the "templating language" term I used in the project description was
> a little bit ambigious. I'd like to use Python as a general purpose
> language to process the context data and generate a new log message from it.
>
> I think embedding Python (or Lua) into softwares written in other
> languages is quite common.
>
> I'd prefer this [0] library for the Rust <-> Python interaction. I used it
> for a side project [1], it's quite flexible and actively
> developed/maintained. You can check the examples in the documentation and
> the tests in the source code.
>
> [0] https://github.com/dgrunwald/rust-cpython
> [1] https://github.com/ihrwein/python-parser
>
> I'd like to see the final solution to be "composable". I mean, by
> providing small, reusable components users can compose the generated log
> message as they want. For example, the following line can generate a log
> message:
>
> logmsg(ctx, "context name is {context.name}, max(foo)={max_foo}",
> foo="bar", quz="baz", max_foo=max(ctx, "foo"))
>
> where ctx contains the context data, logmsg and max is a python/Rust
> function. The syntax isn't settled down :)
>
>
>>
> - For the SQL-like syntax mentioned in project description - Can we design
>> it as Python library using Rust via CFFI [0].
>>
> see my previous answer
>
>
>> I have filled the student application form. I'm already trying to
>> understand the correlation library and playing around with existing tests.
>> How can I get more involved and better understand the requirements of the
>> project?
>>
>
> First, I'd check what information is available in the contexts (your
> context data):
> https://github.com/ihrwein/correlation/blob/master/src/config/action/message/renderer_context.rs#L36-L48
>
> You you can use {{context_len}} to get the number of messages in the
> context, etc...
>
> IMHO the hardest part is using the [0] libray. It's well designed, but if
> you are new to Rust you may find it difficult to use. So I'd try that
> first. I don't know about any bugs in the correlation library, but you may
> try to refactor some code (just to see the internals).
>
> The requirements are listed in the idea page as "Tasks".
>
> BR,
> Tibor
>
>
>
>
>
>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation:
> http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>
>


-- 
- Akshay Aurora
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.balabit.hu/pipermail/syslog-ng/attachments/20160321/5592e02d/attachment-0001.htm