[syslog-ng] patterndb: collect login/logout samples

[ILLES, Marton]

Hi,

I took the liberty and created my own patterndb git tree, so i can track
my patches there and Bazsi can easily pull from there. You can check it
at git.balabit.hu:
http://git.balabit.hu/?p=marci/syslog-ng-patterndb.git;a=summary

I have added a small python script test-patterns.py which can be used to
automatically check the example messages against the patterns, while it
also verifies the parsed name/value pairs. It is kind of a handy tool
when you poke with the patterns and want to run automatic tests. It
requires pdbtool and python xml package.

I have fixed the sshd.pdb example messages and extended them to check
for name/value pairs as well.

I have also modified to patterns to use the ESTRING/ANYSTRING parsers
instead of the STRING/IPv4/NUMBER parsers as the previous ones are
faster, and they should be used when possible. The STRING/IPv4/NUMBER
parser parse the message char by char, while the ESTRING/QSTRING parsers
are looking for an stop character/string and parse everything till than.
The ANYSTRING parser on the other hand simply parses everything till the
end of the message, so it is handy to parse the rest of the message into
a name/value pair. The QSTRING/ESTRING parsers are especially useful
when the type of the parsed part is not important, eg: we do not want to
handle an ip address or a number specially later. (This was the case in
the sshd messages, though it might make sense to extend the policy to
define which parser should be used in some cases...)


Bazsi, please pull my tree:

Marton Illes (2):
      added test-patterns.py script to test the patterns with the
example log messages
      access/sshd.pdb: fixed example messages and added test_values
      access/sshd.pdb: use ESTRING/ANYSTRING parser instead of
STRING/IPv4/NUMBER


Marci


On Tue, 2010-07-13 at 13:25 +0200, Balazs Scheidler wrote:
> Hi,
> 
> After getting the generic patterndb policy into shape, I'd like to start
> collecting log samples, preferably in a domain that is useful for
> everyone.
> 
> My target is at first is login/logout/login failure events. I'd start
> with a generic Linux installation and try to cover all applications that
> perform authentication.
> 
> As a starter, I've commited access/sshd.pdb, containing three rules for
> OpenSSH login/logout/login failure events.
> 
> I'd head towards standard services, ftp, pop3 and imap authentication,
> using their "default" implementation in Ubuntu/Debian. (if there's no
> default, I'll just pick one at random).
> 
> If any of you can collect these 3 samples of any of the applications
> that you run daily on your system and submit them here, it'd be
> tremendous use and would be appreciated.
> 
> The format of the submission would be preferred in patterndb format (see
> the ssh sample I've just pushed), but if you are afraid of that, even
> simple samples would be useful, I'll do the markup myself.
> 
-- 
Key fingerprint = F78C 25CA 5F88 6FAF EA21 779D 3279 9F9E 1155 670D