[syslog-ng] Re: syslog-ng: new destination driver

[Balazs Scheidler]

cc-ing to the syslog-ng mailing list.

On Mon, Jul 19, 1999 at 03:31:21PM -0700, Jeremy Fitzhardinge wrote:
> Hi,
> 
> We need a syslog which can parse messages and then farm them out to
> other alert mechanisms, which are dynamic and configurable, such as SNMP
> traps, emails, pagers, etc...  My original idea was to add a regexp
> mechanism to syslog and fire off lines which match to the various
> programs.  After looking around, it looks like your syslog-ng is the
> closest to our requirements, but it doesn't have the mechanism to
> actually send lines to a program.

Indeed it does, but the program statement you suggest below is on my todo
list. To send lines to a program, you could use a named pipe.

mkfifo /var/run/pager

the program would then:
#!/bin/sh
cat /var/run/pager | batch_and_send 

and in syslog-ng declare a destination:

destination { pipe("/var/run/pager"); };

However since it's impossible to pass arguments to the given script this
way, a program destination driver should - and will - be implemented.

> 
> I'm thinking of something along the lines of:
> 
> destination pager { prog("/usr/local/bin/pager.pl") };
> 
> Each destination using a prog() would start a new instance, which is
> persistent for the duration of syslog's life.  This is so that they can
> batch things together, rather than sending one email for each line of a
> multi-line message.
> 
> I've been going through the source to see how to implement it, and it
> looks pretty clean.  The only thing I don't understand is all the .x
> files and make_class.  How do these fit together, and what do I need to
> do to implement a new afprog.*?

syslog-ng uses make_class as a preprocessor to create data structures for
object classes, and to generate code needed by the mark & sweep garbage
collector. To create a new destination driver, you would create a class from
(or directly use) log_dest_driver defined in destinations.h. If you look at
affile.c (as one of the simplest destination driver), it derives a class:

/* CLASS:
     (class
       (name affile_dest)
       (super log_dest_driver)
       (vars
         (name string)
         (flags simple int)
         (dest object io_fd)
         (dest_buf object abstract_buffer)
         (sync_freq simple UINT32)))
*/

The constructor of a class is called make_<classname> by convention, so
affile_dest is created by make_affile_dest:

struct log_dest_driver *make_affile_dest(const char *name, int flags)
{
        NEW(affile_dest, self);

        self->super.super.init = do_init_affile_dest;
        self->super.super.handler = do_handle_affile_log;
        self->super.log_fifo_size = -1;
        self->name = c_format_cstring("%z", name);
        self->flags = flags;
        self->sync_freq = -1; /* use global setting */
        return &self->super;
}

This returns "struct log_dest_driver *" because the derived class itself is
private to this module, and externally only the log_dest_driver is known.
NEW creates a new instance of affile_dest, and stores a pointer to this
instance in self. Then the method pointers are initialized. 

Internally log messages flow in log-pipes:

log_source_driver ->
	log_source_group ->
		log_center ->
			log_dest_group ->
				log_dest_driver

Several log_source_drivers may point to the same source_group, and several
source_groups may point to a log_center. There's only one log_center per
configuration, and log_center is the one which applies filter rules, and in
case they match, it chooses the appropriate log_dest_group, thus sends the
log message to all the log_dest_driver belonging to that group.

An element of this pipe must be a descendant of log_handler (defined in
log.h) Since several element has a next pointer (all except the last
log_dest_driver) a less general class: log_handler_pipe is introduced here,
which has an additional next field.

The above pipe is constructed as the configuration file is read, and if the
configuration is found to be syntactically valid, everything is initialized.
(this is log_handler.init) 

Source drivers open source sockets, and install read hooks to get events on
them. As a message is read it is passed towards the central log_center using
the next->handler method. There's an optional method in log_handlers, which
is called destroy. It is called when the log pipe is destroyed.

So this is the internal data structure in syslog-ng, for detailed
information about the class definition language parsed by make_class,
download lsh (ftp://ftp.lysator.liu.se/pub/security/lsh/lsh*.tar.gz) and
read the file HACKING. make_class requires scsh (Scheme shell) to run,
though make_class is being ported to more free implementations of scheme,
like guile.

I hope this helps,
-- 
Bazsi
PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1
     url: http://www.balabit.hu/pgpkey.txt