[syslog-ng] WANTED: feature owner of syslog-ng command line tool :)

[Balint Kovacs]

Hi,

This is a great one, it will help sysadmins tremendously. Some of it is
available in "pbdtool match", but it's not a very flexible as the
available syslog-ng facilities (filter, template) are hardwired. On the
other hand, it helps with gathering user stories, I've been using this
for the following tasks:

- patterndb building process -> testing the current patterndb coverage
and identify outliers in existing log data
- "offline" processing log messages, without running a syslog-ng daemon
    - basic filtering, "grep on steroids"
    - advanced, message parser (e.g. patterndb) based filtering
    - rewrite, adding metadata and anonymisation
    - transform output using templates
    - postprocessing old raw logs with new patterndb rules

What is not available in pbdtool is the heuristics in the initial
message parsing of the source drivers, it would be great to have that
implemented in the stdin driver, as it would save a great deal of time
and resources if you don't have to transform the source material to a
format that pdbtool likes.

I for myself see specifying the configuration in the command line a bit
clumsy, I would not want to deal with escaping quotation marks and
parentheses and rather use an input file. Also, the supplied config
snippet can grow very fast to a level where it is not quite well
readable without indentation. Nevertheless this might be useful for
quick, one-shot tests.

It would also be great if this tool could be used to test very complex
log routing setups by being able to use your _existing_ syslog-ng
configuration and feed a test message to it. It could prefix the output
with the name of the chosen destination or it you want to take the idea
further, you could specify which source and destination should be
attached to stdin and stdout respectively, overriding the configured
source and destination with the new drivers. The closest thing to this,
that comes to my mind, is when you are stubbing out input and output
streams of a library function within a unit test. I'm not sure that this
fits in the scope of the GSoC project, but would save quite a lot of
debug time on some setups I've encountered.

Best regards,
Balint

On 05/21/2016 03:02 PM, Scheidler, Balázs wrote:
> Hi,
>
> This is an email where I tried to describe what I had in mind with the
> original idea:
>
> http://comments.gmane.org/gmane.comp.syslog-ng/17449
>
> It would be great to have this implemented.
>
> Cheers,
> Bazsi
>
>
> -- 
> Bazsi
>
> On Fri, May 20, 2016 at 2:10 PM, Noémi Ványi <sitbackandwait at gmail.com
> <mailto:sitbackandwait at gmail.com>> wrote:
>
>     Hello!
>
>     A new feature is going to be developed, as a part of GSOC, so
>     syslog-ng can be started as a command line tool.
>
>     What could the CLI do for you?
>
>       * pipelines of complex filtering, rewriting could be tested more
>         easily, thus configuration could become simpler than before
>       * existing logs could be transformed
>
>     The format of the new command line options is not decided yet. So
>     your help and insight is needed to design the interface and the
>     usage of the tool that you and other people around the world would
>     use. :)
>
>     If you are intereted in contributing to the CLI, thus contributing
>     to the syslog-ng community, please send an email to this thread.
>     :) I am available at gitter for further info. :)
>
>     Regards
>     kvch
>
>     ______________________________________________________________________________
>     Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>     Documentation:
>     http://www.balabit.com/support/documentation/?product=syslog-ng
>     FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>
>
>
>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.balabit.hu/pipermail/syslog-ng/attachments/20160523/b8147136/attachment-0001.htm