[syslog-ng]syslog-ng development release 1.9.2

[Balazs Scheidler]

Sorry, I accidentaly sent an incomplete message yesterday. I'm adding
the missing comments now.

On Mon, 2005-01-03 at 22:20 +0100, Balazs Scheidler wrote:
> Hi,
> 
> It is my pleasure to announce the availability of the latest development
> release for syslog-ng, and a related library called eventlog.
> 
> This release is actually usable, and although it is still not
> recommended to be used on production systems, running it on a spare
> system with a real log feed would be very useful indeed, to see how it
> behaves with various real life configurations.
> 
> The major differences between syslog-ng 1.9.2 and the releases in the
> 1.6.x branch are outlined in the NEWS file attached to the end of this
> message.
> 

Well, syslog-ng 1.9.2 is available at the usual location at
http://www.balabit.hu/downloads/syslog-ng/1.9/src/

You need to download both eventlog and syslog-ng, compile/install
eventlog first and then syslog-ng.

Documentation is more or less missing, though the documentation for
version 1.6 is still very useful, syslog-ng should be compatible with
the old version with some minor exceptions:

The following keywords are currently not yet known by this version of
syslog-ng, but support for these will be added:
@@ -1,9 +1,7 @@
-       { "bad_hostname",       KW_BAD_HOSTNAME },
-       { "check_hostname",     KW_CHECK_HOSTNAME },
-       { "netmask",            KW_NETMASK },
-       { "remove_if_older",    KW_REMOVE_IF_OLDER },
-       { "sanitize-filenames", KW_SANITIZE_FILENAMES },
-       { "sanitize_filenames", KW_SANITIZE_FILENAMES },
-       { "spoof-source",       KW_SPOOF_SOURCE },
-       { "spoof_source",       KW_SPOOF_SOURCE },
-       { "stats",              KW_STATS_FREQ },
-       { "tcp-keep-alive",     KW_TCP_KEEP_ALIVE },
-       { "tcp_keep_alive",     KW_TCP_KEEP_ALIVE },


The following keywords do not exactly match with their 1.6 counterparts,
but this will be fixed:
-       { "mark",               KW_MARK_FREQ },
+       { "mark_freq",          KW_MARK_FREQ },
-       { "sync",               KW_SYNC_FREQ },
+       { "sync_freq",          KW_SYNC_FREQ },
-       { "pad_size",           KW_PAD_SIZE },
+       { "padding",            KW_PADDING },

Otherwise the new syslog-ng provides the following nifty features:

- flow controlled log paths: it is possible to avoid message drops by
not reading incoming messages sources while the output buffer is full.
For example:

  source s_stream { unix-stream("/dev/log"); };
  destination d_net { tcp("1.2.3.4" port(2000)); };
  log { source(s_stream); destination(d_net); flags(flow-control); };

That's it. This will avoid message drops within syslog-ng, provided the
sum of all source window sizes (configurable via log_iw_size) is less
than or equal to the output driver's log_fifo_size. Syslog-ng simply
reads the incoming sources at the rate of the output drivers.

- automatic respawning of program destinations: while it might make the
syslog server easily DoSable, syslog-ng now automatically restarts the
programs run via the program destination.

- following regular files as input, e.g. this actually works:

  source s_file { file("/var/log/apache/access.log" follow_freq(10)); };

  The above will check /var/log/apache/access.log every 10 seconds and
read out all new messages since the last check. Although this is a nice
feature syslog-ng does not currently preserve the last ofset during
restarts.

- earlier noticing of TCP connection drops: instead of waiting for a
write() syscall to fail, continously check whether the server closed the
connection.

- the possibility to define templates once and reuse it from
file/protocol destinations, e.g:

  template t_sample { template("$HOUR abcdef ${MSG}\n"); };
  destination d_all { file("logs/logfile" template(t_sample));  };

  or

  # use t_sample for file destinations by default
  options { file_template(t_sample); };

  or
  # use t_sample for protocol destinations by default
  options { proto_template(t_sample); };


- the ability to use different time stamp formats:

  options { ts_format(iso); };

  will use ISO8661 timestamps everywhere

- the ability to work with timezones: support timezone information on
incoming messages via the ISO timestamp, support timezone conversion,
etc:

  options { tz_convert(+0100); };
  
  or

  # for messages without a timestamp containing timezone
  source s_udp { udp(timezone(+0500)); };

  or 
  destination d_net { tcp("1.2.3.4" tz_convert(+0100)); };

- the ability to use unparsed log sources

  source s_src { unix-stream("/dev/log" flags(no_parse)); };

I think those are the important changes. But which is even more
important is that the new codebase is more compact and makes adding new
features easier.

-- 
Bazsi