[syslog-ng] WANTED: feature owner of syslog-ng command line tool :)

[Scheidler, Balázs]

Really good ideas here. Thanks for sharing.

Some answers inline.

On May 23, 2016 6:06 PM, "Balint Kovacs" <kovacs.balint at balasys.hu> wrote:
>
> Hi,
>
> This is a great one, it will help sysadmins tremendously. Some of it is
available in "pbdtool match", but it's not a very flexible as the available
syslog-ng facilities (filter, template) are hardwired. On the other hand,
it helps with gathering user stories, I've been using this for the
following tasks:
>
> - patterndb building process -> testing the current patterndb coverage
and identify outliers in existing log data
> - "offline" processing log messages, without running a syslog-ng daemon
>     - basic filtering, "grep on steroids"
>     - advanced, message parser (e.g. patterndb) based filtering
>     - rewrite, adding metadata and anonymisation
>     - transform output using templates
>     - postprocessing old raw logs with new patterndb rules
>
> What is not available in pbdtool is the heuristics in the initial message
parsing of the source drivers, it would be great to have that implemented
in the stdin driver, as it would save a great deal of time and resources if
you don't have to transform the source material to a format that pdbtool
likes.
>
> I for myself see specifying the configuration in the command line a bit
clumsy, I would not want to deal with escaping quotation marks and
parentheses and rather use an input file.

Yeah, I think both are needed. Awk supports both specifying it via the
command line and specifying a file.

Using apostrophes you can specify parentheses without escaping. Anyway,we
are in agreement here.

Also, the supplied config snippet can grow very fast to a level where it is
not quite well readable without indentation. Nevertheless this might be
useful for quick, one-shot tests.
>
> It would also be great if this tool could be used to test very complex
log routing setups by being able to use your _existing_ syslog-ng
configuration and feed a test message to it. It could prefix the output
with the name of the chosen destination or it you want to take the idea
further, you could specify which source and destination should be attached
to stdin and stdout respectively, overriding the configured source and
destination with the new drivers.

The closest thing to this, that comes to my mind, is when you are stubbing
out input and output streams of a library function within a unit test. I'm
not sure that this fits in the scope of the GSoC project, but would save
quite a lot of debug time on some setups I've encountered.

Good ideas. I am not sure how to handle complex source/destination setups.

Also this could be extremely helpful when combined with the built-in
interactive config debugger (eg -i in 3.7 and current master)

>
> Best regards,
> Balint
>
>
> On 05/21/2016 03:02 PM, Scheidler, Balázs wrote:
>>
>> Hi,
>>
>> This is an email where I tried to describe what I had in mind with the
original idea:
>>
>> http://comments.gmane.org/gmane.comp.syslog-ng/17449
>>
>> It would be great to have this implemented.
>>
>> Cheers,
>> Bazsi
>>
>>
>> --
>> Bazsi
>>
>> On Fri, May 20, 2016 at 2:10 PM, Noémi Ványi <sitbackandwait at gmail.com>
wrote:
>>>
>>> Hello!
>>>
>>> A new feature is going to be developed, as a part of GSOC, so syslog-ng
can be started as a command line tool.
>>>
>>> What could the CLI do for you?
>>> pipelines of complex filtering, rewriting could be tested more easily,
thus configuration could become simpler than before
>>> existing logs could be transformed
>>> The format of the new command line options is not decided yet. So your
help and insight is needed to design the interface and the usage of the
tool that you and other people around the world would use. :)
>>>
>>> If you are intereted in contributing to the CLI, thus contributing to
the syslog-ng community, please send an email to this thread. :) I am
available at gitter for further info. :)
>>>
>>> Regards
>>> kvch
>>>
>>>
______________________________________________________________________________
>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>> Documentation:
http://www.balabit.com/support/documentation/?product=syslog-ng
>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>>
>>>
>>
>>
>>
>>
______________________________________________________________________________
>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>> Documentation:
http://www.balabit.com/support/documentation/?product=syslog-ng
>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>
>
>
>
______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation:
http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.balabit.hu/pipermail/syslog-ng/attachments/20160523/d65dc977/attachment.htm