<div dir="ltr"><div><div><div><div><div><div><div><div>Hi,<br><br></div><div>I've Cced the syslog-ng list, so others can see this description as well.<br></div><div><br></div>The goal here is to change syslog-ng to be able to process input from stdin and produce output to stdout and exit when the EOF is reached, while processing a "potentially" simplified syslog-ng configuration block. A bit similar to how awk works, but instead of the awk language, use syslog-ng's constructs.<br><br></div>E.g. <br><br></div>$ syslog-ng -P 'parser { kv-parser(); }; destination { stdout(template("$(format-json *"))); };' < log-file-with-keywords.log > log-file-in-json<br><br></div>Where -P would be equivalent to --pipe, the next argument is a configuration snippet. This snippet would get all logs as read from stdin() and that would be default functionality provided by the --pipe construct. Probably --pipe would use a configuration file something like:<br><br></div>log {<br></div> source { stdin(); };<br></div> # program supplied by --pipe would be included here<br></div> pipe-program();<br>};<br><div><div><div><div><div><div><br></div><div>This is the basic idea, some customization would be useful, like:<br><ul><li>being able to specify the source as well (so the above wouldn't be hardwired, only used as a default)</li><li>maybe use a default for output as well, so I wouldn't have to include it in the actual language</li><li>examples: a few examples how to process existing log files from postfix, iptables, snort/suricata, etc to convert their logs into json. This would be both a set of examples and could also drive the functionality further, so we don't miss important options.</li></ul><p>Code wise, this would need something like:</p><ul><li>stdin source: would probably only be an SCL wrapper around the existing file() driver; e.g. file("/dev/stdin"); check for portability on various UNIXes (freebsd, solaris, macosx)<br></li><li>stdout destination: would probably only be an SCL wrapper around the existing file driver, e.g. file("/dev/stdout");</li><li>stderr destinatiion: would probably only be an SCL wrapper around the existing file driver, e.g. file("/dev/stderr");</li><li>a means to construct the complete configuration as passed in by command line arguments; would probably need to register confgen blocks, and a larger configuration frame where they are inserted.<br></li><li>a means to terminate syslog-ng when we read the entire input and all outputs are finished. (would probably be the most difficult)</li><li>tests</li></ul><p>Hope this helps.</p><p>Bazsi<br></p></div></div></div></div></div></div></div><div class="gmail_extra"><br clear="all"><div><div class="gmail_signature"><div dir="ltr">-- <br>Bazsi<br></div></div></div>
<br><div class="gmail_quote">On Mon, Mar 7, 2016 at 6:37 AM, Viraj Madhawa <span dir="ltr"><<a href="mailto:emadhawa23@gmail.com" target="_blank">emadhawa23@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Dear Sir, <div><br></div><div>I'm a computer engineering undergraduate student of University of Peradeniya Sri Lanka. During my internship period I done a log managment system project to a bank (using syslog protocol) . I'm quite interested about this project idea and I like to get more details about it (ask some questions). <span class="HOEnZb"><font color="#888888"><br clear="all"><div><br></div>-- </font></span></div><span class="HOEnZb"><font color="#888888"><div>Thank you, <br><div><div dir="ltr">Viraj Premaratne<div><br></div><div><br></div></div></div>
</div></font></span></div>
</blockquote></div><br></div>