[syslog-ng] Unix-stream destination as listen socket

[Balazs Scheidler]

On Tue, 2005-10-18 at 10:46 -0400, Peter Nahas wrote:
> Hello everyone,
> 
> I am currently working on a project using syslog-ng-1.9.5 (soon to 
> upgrade to 1.9.6) which requires a syslog-ng destination to be a Unix 
> socket.  The problem is that the syslog-ng daemon will always be started 
> before the destination application.  So to prevent the loss of messages 
> due to the reconnect timer, the syslog-ng destination Unix socket  will 
> have to  listen()/accept() for a connection rather than connect() to an 
> external socket.  If this enhancement would be used by others, I would 
> like to write this properly.  Therefore I have the following questions:

I'd say there different ways to implement the same:
* persistent buffering: in addition to using a RAM based buffer, use a
circular buffer stored on disk; this way your application can remain a
UNIX domain server, and syslog-ng would retry connections until it comes
up
* use program() destination, which sends the messages to you via your
standard input
* use some kind of external relaying application, that integrates
syslog-ng via either program/unix-stream and would relay messages to
your application once it started up

> A) If this enhancement were written, would it be accepted into the 
> repository?  If not, I will simply hack up the current 
> AFUnixDestinationDriver to do what I want. 

Without seeing the actual implementation it's hard to say, however it
would be nice to keep the syslog-ng model

> B) Is there any specification document which would assist me in making 
> this enhancement? 

Apart from the source? Well, no there's none. The actual design of
syslog-ng is quite simple however: everything is a LogPipe, and LogPipes
attached together perform the task at hand. 

source driver 
  -> source group 
    -> log center 
      -> destination group 
        -> dest driver

Some drivers use some additional LogPipes for things like TCP connections.

A LogPipe is a half-duplex pipe processing log messages, new messages are 
injected to the next pipe element using log_pipe_queue(). The basic interface
implemented by all elements is here:

typedef struct _LogPipe
{
  guint ref_cnt;
  struct _LogPipe *pipe_next;
  gboolean (*init)(struct _LogPipe *self, GlobalConfig *cfg, PersistentConfig *persist);
  gboolean (*deinit)(struct _LogPipe *self, GlobalConfig *cfg, PersistentConfig *persist);
  void (*queue)(struct _LogPipe *self, LogMessage *msg, gint path_flags);
  void (*free_fn)(struct _LogPipe *self);
  void (*notify)(struct _LogPipe *self, struct _LogPipe *sender, gint notify_code, gpointer user_data);
} LogPipe;



> C) Would this best be done by adding an additional option to unix-stream 
> for destinations or by creating a separate destination driver? 
> D) It appears as if the AFSocketDestinationDriver is not designed to 
> perform listen()/accept(), but the AFSocketSourceDriver is.  Would it be 
> possible to set up the AFUnixDestinationDriver to use the 
> AFSocketSourceDriver as a parent, but attach it to a log writer?

I'd recommend rethinking the basic idea again. There should be a lot of
different possibilities to integrate your application that are already
implemented.

That said, the functionality you want to implement is really orthogonal
to both unix-stream sources and destinations. This is somewhat similar
to the 'call-home' way of retrieving log information in which case the
receiver connects 'out', then receives messages.

In the implementation it would be something like AFSocketCHSourceDriver
and AFSocketCHDestinationDriver (CH is for call-home), which'd probably
belong to afsocket.c. Then both the INET and the UNIX socket based
source/destination drivers would benefit.

-- 
Bazsi