[syslog-ng] Regarding GSoC16 Project: syslog-ng as a command line tool

[Scheidler, Balázs]

Hi,

I've Cced the syslog-ng list, so others can see this description as well.

The goal here is to change syslog-ng to be able to process input from stdin
and produce output to stdout and exit when the EOF is reached, while
processing a "potentially" simplified syslog-ng configuration block. A bit
similar to how awk works, but instead of the awk  language, use syslog-ng's
constructs.

E.g.

$ syslog-ng -P 'parser { kv-parser(); }; destination {
stdout(template("$(format-json *"))); };'  < log-file-with-keywords.log >
log-file-in-json

Where -P would be equivalent to --pipe, the next argument is a
configuration snippet. This snippet would get all logs as read from stdin()
and that would be default functionality provided by the --pipe construct.
Probably --pipe would use a configuration file something like:

log {
  source { stdin(); };
  # program supplied by --pipe would be included here
  pipe-program();
};

This is the basic idea, some customization would be useful, like:

   - being able to specify the source as well (so the above wouldn't be
   hardwired, only used as a default)
   - maybe use a default for output as well, so I wouldn't have to include
   it in the actual language
   - examples: a few examples how to process existing log files from
   postfix, iptables, snort/suricata, etc to convert their logs into json.
   This would be both a set of examples and could also drive the functionality
   further, so we don't miss important options.

Code wise, this would need something like:

   - stdin source: would probably only be an SCL wrapper around the
   existing file() driver; e.g. file("/dev/stdin"); check for portability on
   various UNIXes (freebsd, solaris, macosx)
   - stdout destination: would probably only be an SCL wrapper around the
   existing file driver, e.g. file("/dev/stdout");
   - stderr destinatiion: would probably only be an SCL wrapper around the
   existing file driver, e.g. file("/dev/stderr");
   - a means to construct the complete configuration as passed in by
   command line arguments; would probably need to register confgen blocks, and
   a larger configuration frame where they are inserted.
   - a means to terminate syslog-ng when we read the entire input and all
   outputs are finished. (would probably be the most difficult)
   - tests

Hope this helps.

Bazsi

-- 
Bazsi

On Mon, Mar 7, 2016 at 6:37 AM, Viraj Madhawa <emadhawa23 at gmail.com> wrote:

> Dear Sir,
>
> I'm a computer engineering undergraduate student of University of
> Peradeniya Sri Lanka. During my internship period I done a log managment
> system project to a bank (using syslog protocol) . I'm quite interested
> about this project idea  and I like to get more details about it (ask some
> questions).
>
> --
> Thank you,
> Viraj Premaratne
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.balabit.hu/pipermail/syslog-ng/attachments/20160307/e8ca8f68/attachment.htm