Hi,

This is a great one, it will help sysadmins tremendously. Some of it is available in "pbdtool match", but it's not a very flexible as the available syslog-ng facilities (filter, template) are hardwired. On the other hand, it helps with gathering user stories, I've been using this for the following tasks:

- patterndb building process -> testing the current patterndb coverage and identify outliers in existing log data
- "offline" processing log messages, without running a syslog-ng daemon
    - basic filtering, "grep on steroids"
    - advanced, message parser (e.g. patterndb) based filtering
    - rewrite, adding metadata and anonymisation
    - transform output using templates
    - postprocessing old raw logs with new patterndb rules

What is not available in pbdtool is the heuristics in the initial message parsing of the source drivers, it would be great to have that implemented in the stdin driver, as it would save a great deal of time and resources if you don't have to transform the source material to a format that pdbtool likes.

I for myself see specifying the configuration in the command line a bit clumsy, I would not want to deal with escaping quotation marks and parentheses and rather use an input file. Also, the supplied config snippet can grow very fast to a level where it is not quite well readable without indentation. Nevertheless this might be useful for quick, one-shot tests.

It would also be great if this tool could be used to test very complex log routing setups by being able to use your _existing_ syslog-ng configuration and feed a test message to it. It could prefix the output with the name of the chosen destination or it you want to take the idea further, you could specify which source and destination should be attached to stdin and stdout respectively, overriding the configured source and destination with the new drivers. The closest thing to this, that comes to my mind, is when you are stubbing out input and output streams of a library function within a unit test. I'm not sure that this fits in the scope of the GSoC project, but would save quite a lot of debug time on some setups I've encountered.

Best regards,
Balint

On 05/21/2016 03:02 PM, Scheidler, Balázs wrote:
Hi,

This is an email where I tried to describe what I had in mind with the original idea:

http://comments.gmane.org/gmane.comp.syslog-ng/17449

It would be great to have this implemented.

Cheers,
Bazsi


--
Bazsi

On Fri, May 20, 2016 at 2:10 PM, Noémi Ványi <sitbackandwait@gmail.com> wrote:
Hello!

A new feature is going to be developed, as a part of GSOC, so syslog-ng can be started as a command line tool.

What could the CLI do for you?
  • pipelines of complex filtering, rewriting could be tested more easily, thus configuration could become simpler than before
  • existing logs could be transformed
The format of the new command line options is not decided yet. So your help and insight is needed to design the interface and the usage of the tool that you and other people around the world would use. :)

If you are intereted in contributing to the CLI, thus contributing to the syslog-ng community, please send an email to this thread. :) I am available at gitter for further info. :)

Regards
kvch

______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq





______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq