[syslog-ng] syslog-ng OSE 3.1beta1 released

[Robert Fekete]

I hope to release the updated administrator guide on Monday.

Regards,

Robert

Balazs Scheidler wrote:

> Dear syslog-ng users,
> 
> I'm proud to announce that syslog-ng OSE 3.1 has been released and
> uploaded to our webserver. This version is new in two ways:
> 
> 1) of course it has new features, see below for the most interesting
> bits
> 
> 2) it is a "feature release", which means that once syslog-ng 3.2 or
> syslog-ng 4.0 is released, the support for this release will be ceased.
> See our new version policy at this link:
> 
> https://www.balabit.com/network-security/syslog-ng/opensource-logging-system/roadmap.bbx
> 
> Since the documentation is not yet up to date with this beta release,
> I'll try to include the most crucial information about the new features
> right here in this announcement.
> 
> For those who hurry, here's a link for the source code:
> 
> https://www.balabit.com/downloads/files/syslog-ng/open-source-edition/3.1beta1/source/syslog-ng_3.1beta1.tar.gz
> 
> And here are the binaries for Linux/FreeBSD systems:
> 
> https://www.balabit.com/network-security/syslog-ng/opensource-logging-system/
> 
> Select the Downloads tab, and in the Version selector select 3.1beta1.
> 
> What is new in syslog-ng OSE 3.1
> --------------------------------
> 
> * Support for patterndb v3 format, along with a bunch of new
>   parsers: ANYSTRING, IPv6, IPvANY and FLOAT.
> 
>   Patterndb (more exactly the db-parser()) is a high performance message 
>   classifier and information extraction tool, that makes it easy to get away
>   from the unstructured nature of syslog.
> 
>   Patterndb has evolved since it was first introduced in syslog-ng 3.0. It is at 
>   the 3rd iteration, hopefully slowly reaching its final form.
> 
>   Patterndb in general and the v1 format database is described in the syslog-ng 
>   manual at:
>   
>   http://www.balabit.com/dl/html/syslog-ng-admin-guide_en.html/ch02s12.html
> 
>   The XML schemas that describe the different patterndb versions are available in
>   the syslog-ng source tree:
> 
>   http://git.balabit.hu/?p=bazsi/syslog-ng-3.1.git;a=tree;f=doc/xsd;hb=HEAD
> 
>   The changes in the patterndb format as they evolved were described in Marton Illes's blog at
>   
>   http://marci.blogs.balabit.com/2009/06/new-db-parser-format-and-other.html
> 
>   But see the other related posts as well.
>  
>   Old patterndb databases can be converted to the new format by putting them
>   in a directory and using the pdbtool utility using the command:
> 
>   $ pdbtool merge -p /opt/syslog-ng/var/patterndb.xml -D /opt/syslog-ng/etc/patterns.d
> 
>   Assuming the installation prefix of syslog-ng is /opt/syslog-ng
> 
>   Some v2 format patterns are distributed by BalaBit itself for its SSB product, 
>   download location:
> 
>   https://www.balabit.com/downloads/files/patterndb/1.0-20081117/patterndb/
> 
>   You can convert these db files using pdbtool as described above.
> 
>   Work is ongoing to publish a more comprehensive patterndb, but more on that
>   in a separate post.
> 
> * Added a new "pdbtool" utility to manage patterndb files: convert
>   them from v1 or v2 format, merge mulitple patterndb files into one
>   and look up matching patterns given a specific message.
> 
>   See the manpage and Marci's post:
> 
>   http://marci.blogs.balabit.com/2009/08/db-parser-new-utility-pdbtool.html
> 
> * Support for message tags: tags can be assigned to log messages as
>   they enter syslog-ng: either by the source driver or via patterndb.
>   Later it these tags can be used for efficient filtering.
> 
>   http://marci.blogs.balabit.com/2009/05/tag-support-in-syslog-ng.html
> 
> * Added support for rewriting structured data.
> 
>   Earlier structured data fields in the new RFC5424 style syslog protocol
>   were only read-only values that could be referenced in a template, but they 
>   couldn't be changed, and neither was it possible to add new fields in an 
>   already existing syslog message.
> 
>   Now all these became possible by using the same syntax that didn't work 
>   earlier, e.g.
> 
>   rewrite r_sd { set("55555" value(".SDATA.meta.sequenceId")); };
> 
> * Macros and name-value pairs got a little tighter integration,
>   in filters where syslog-ng 3.0 was limited to only use name-value
>   pairs, with 3.1 you can also use macros.
> 
>   The following now works:
> 
>   match("<regexp>" value("R_DATE"));
> 
>   syslog-ng is now warning you in case you are using '$' prefix in 
>   the value syntax.
> 
> * Enhanced dynamic name-value performance by a factor of three.
> 
>   The summary says it all, the performance dynamic name-value pairs that
>   the various parsers produce got faster, thus the performance penalty of
>   structuring the incoming messages got smaller.
> 
> * Some parsers got additional features: NUMBER is now able to parse
>   hexadecimal numbers, ESTRING is now able to search for a sequence
>   of characters as the end of the string.
> 
>   These are patterndb parsers to make it easier to describe log messages.
> 
> * Added non-standard and non-portable facility codes (range 10-15),
>   decouple syslog-ng facility name information from the system used
>   to compile syslog-ng on.
> 
>   Until this time the facility codes as understood by syslog-ng were
>   dependant on the platform syslog-ng was compiled on. This is not true
>   anymore, syslog-ng comes with its own "facility" code assignments, based
>   on the RFC, and adding some non-standard values found on various 
>   UNIX systems.
> 
> Any feedback, success/failure reports is more than appreciated.
>