[syslog-ng] RFC: debugger

[Scheidler, Balázs]

Hi,

I was involved in a number of projects where syslog-ng was used in a
relatively complex setup: we wanted to structure incoming log messages and
transform them into nice & clean JSON messages.

In a number of cases, this required several parsers to get right
(csv-parser, db-parser and sometimes regexps) and the resulting
configuration was pretty complicated.

I've figured, that if we have a domain specific language in the form of the
syslog-ng configuration file, we should also have a debugger that makes it
easy to find out what went wrong. And this is what I have did, some
proof-of-concept code is available on the f/debugger branch on github:

https://github.com/balabit/syslog-ng/tree/f/debugger

Here's how it works:

# start up syslog-ng with the -i option to indicate we
# want the debugger enabled
$ sbin/syslog-ng -iFe

Once syslog-ng starts up, you'll see something like this:
Waiting for breakpoint...

For now, all processing elements within syslog-ng (called LogPipe) breaks
into the debugger. Later, I want to implement real breakpoints, where you
can selectively attach breakpoints to elements in the configuration file.

# on another console send a log message to syslog-ng:
$ logger almafa

This is what happens on the syslog-ng side:

Breakpoint hit etc/syslog-ng.conf:11:2
11           unix-stream("log");
Dec 15 16:14:05  bazsi: almafa
(syslog-ng)

At a breakpoint, syslog-ng displays where the break has occurred, quotes
the source line and displays the message that is being processed.

Now it becomes interactive, you can inspect the message with various
commands:

(syslog-ng) help
syslog-ng interactive console, the following commands are available

  help, h, or ?            Display this help
  continue or c            Continue until the next breakpoint
  print, p                 Print the current log message
  drop, d                  Drop the current message
  quit, q                  Tell syslog-ng to exit
(syslog-ng) p $MSG
almafa
(syslog-ng) p
MESSAGE=almafa
PROGRAM=bazsi
LEGACY_MSGHDR=bazsi:
.unix.pid=18703
.unix.uid=1000
.unix.gid=1000
TAGS=

(syslog-ng) p "$(format-json --key *)"
{"_unix":{"uid":"1000","pid":"18703","gid":"1000"},  ... }


With the "continue" command, you can request syslog-ng to proceed to the
next LogPipe, where it stops again. "drop" drops the current message,
"quit" tells syslog-ng to exit.

I would really like to extend this to further ways, like:

   - real breakpoints, instead of stopping everywhere
   - inject messages
   - change messages to make configuration testable
   - single-step (to the next primitive LogPipe) and step-over (to the next
   'real' LogPipe that is found in the configuration file)
   - interactive REPL to template functions to make it easier to interact
   with them

What do you think?
-- 
Bazsi
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.balabit.hu/pipermail/syslog-ng/attachments/20141215/d6957f63/attachment.htm