[syslog-ng] pdbtool patternize update and my syslog-ng 3.2 branch

[Matthew Hall]

Hello Peter,

Thanks for the update on the status of patterndb patternize.

I wondered if the memory leaks you said existed in the old version had 
been fixed, you did not say one way or the other in your mail.

I also wonder if anybody at Balabit could tell me how to build a copy of 
your Git tree on RHEL 4 or RHEL 5. I get problems because the PCRE is 
too old but when I switch to new PCRE, PCRE will not build because the 
autotools and pkg-config are too old.

It's a problem for me because unfortunately my company only supports 
RHEL here and otherwise I have to run it in an Ubuntu 10.04 or Debian VM 
with way too little memory for the tool to run right.

Would it be possible to build a version of your tree for RHEL 4 or 5?

Matthew.

On Fri, Sep 24, 2010 at 11:27:48AM +0200, Peter Gyongyosi wrote:
> 
> 
> Hello,
> 
> As the patterndb project is starting to gain some momentum I thought
> it'd be the right time to port my patternize tool to the new,
> plugin-based 3.2 codebase as the first step towards getting it
> integrated --- and to be able to use the fancy new pdbtool features
> along with patternize. To those who are unfamiliar with it,
> patternize is an addition to pdbtool that makes it possible to
> automatically generate a pattern database from raw logs using
> statistical data clustering methods: you can read more about it in
> this blog post:
> http://gyp.blogs.balabit.com/2010/01/introducing-pdbtool-patternize/
> 
> Besides the port to the new codebase, it's received some fixes and
> new features since my original post:
> 
>  * multiple small internal bugfixes to get rid of weird errors
>  * added the option "/--named-parsers/" that names the found
> @ESTRING at s like "/.dict.string0,1,2,3.../"
>  * Balint Kovacs has sent three contributions: added support for
> reading the logfile from the standard input, escaping special
> characters in the output and putting examples in the XML that can be
> used for self-testing.
> 
> It can be found in my public syslog-ng 3.2 tree:
> http://git.balabit.hu/?p=gyp/syslog-ng-3.2.git;a=summary
> 
> If you're already using it (I've received some feedback so I guess
> some of you do), please note that most probably this 3.2-based
> branch will get the fixes and new features from now on.
> 
> It's only received a basic sanity check and the unit tests do pass,
> so as usual, handle it with care and all feedback is welcome.
> 
> greets,
> Peter
> 
> ps.: the branch also contains a patch that fixes a wrong section
> name in pdbtool's man page and I'll try to update the whole manpage
> a bit when adding a section for patternize soon -- Bazsi, you might
> want to pull those to the mainline.