[syslog-ng] pdbtool patternize update and my syslog-ng 3.2 branch

[Matthew Hall]

Further investigation...

The Balabit RPMs must be built specially because the daemon looks mostly 
statically linked. Is there some secret to building code for old Linuxes 
like the BalaBit RPMs?

Matthew.

On Fri, Sep 24, 2010 at 10:57:38AM -0700, Matthew Hall wrote:
> Hello Peter,
> 
> Thanks for the update on the status of patterndb patternize.
> 
> I wondered if the memory leaks you said existed in the old version had 
> been fixed, you did not say one way or the other in your mail.
> 
> I also wonder if anybody at Balabit could tell me how to build a copy of 
> your Git tree on RHEL 4 or RHEL 5. I get problems because the PCRE is 
> too old but when I switch to new PCRE, PCRE will not build because the 
> autotools and pkg-config are too old.
> 
> It's a problem for me because unfortunately my company only supports 
> RHEL here and otherwise I have to run it in an Ubuntu 10.04 or Debian VM 
> with way too little memory for the tool to run right.
> 
> Would it be possible to build a version of your tree for RHEL 4 or 5?
> 
> Matthew.
> 
> On Fri, Sep 24, 2010 at 11:27:48AM +0200, Peter Gyongyosi wrote:
> > 
> > 
> > Hello,
> > 
> > As the patterndb project is starting to gain some momentum I thought
> > it'd be the right time to port my patternize tool to the new,
> > plugin-based 3.2 codebase as the first step towards getting it
> > integrated --- and to be able to use the fancy new pdbtool features
> > along with patternize. To those who are unfamiliar with it,
> > patternize is an addition to pdbtool that makes it possible to
> > automatically generate a pattern database from raw logs using
> > statistical data clustering methods: you can read more about it in
> > this blog post:
> > http://gyp.blogs.balabit.com/2010/01/introducing-pdbtool-patternize/
> > 
> > Besides the port to the new codebase, it's received some fixes and
> > new features since my original post:
> > 
> >  * multiple small internal bugfixes to get rid of weird errors
> >  * added the option "/--named-parsers/" that names the found
> > @ESTRING at s like "/.dict.string0,1,2,3.../"
> >  * Balint Kovacs has sent three contributions: added support for
> > reading the logfile from the standard input, escaping special
> > characters in the output and putting examples in the XML that can be
> > used for self-testing.
> > 
> > It can be found in my public syslog-ng 3.2 tree:
> > http://git.balabit.hu/?p=gyp/syslog-ng-3.2.git;a=summary
> > 
> > If you're already using it (I've received some feedback so I guess
> > some of you do), please note that most probably this 3.2-based
> > branch will get the fixes and new features from now on.
> > 
> > It's only received a basic sanity check and the unit tests do pass,
> > so as usual, handle it with care and all feedback is welcome.
> > 
> > greets,
> > Peter
> > 
> > ps.: the branch also contains a patch that fixes a wrong section
> > name in pdbtool's man page and I'll try to update the whole manpage
> > a bit when adding a section for patternize soon -- Bazsi, you might
> > want to pull those to the mainline.