[syslog-ng] pattern usage and optimization

[Joel Carnat]

Hi,

I want to parse my sshd logs to store information in a remote database.
I already did it using logstash. But I just discovered syslog-ng can do 
such things using patterndb.

I could manage to setup a few <pattern> but I have difficulties building 
generic rules.
I end up with 4 or 5 rules where I can only deal with one or two using 
logstash.
So I expect to be missing something with patterns :)

Here's a log example:
Disconnected from user joe 192.168.0.5 port 50121
Disconnected from invalid user www 192.168.0.7 port 6794 [preauth]
Disconnected from authenticating user root 192.168.0.3 port 52591 
[preauth]

So I wrote those three patterns:
<!-- Disconnected from user joe 192.168.0.5 port 50121 -->
<pattern>@ESTRING:EVENT: from @user @ESTRING:USERNAME: @@ESTRING:IP: 
@port @NUMBER:PORT:@</pattern>
<!-- Disconnected from invalid user www 192.168.0.7 port 6794 [preauth] 
-->
<pattern>@ESTRING:EVENT: from @invalid user @ESTRING:USERNAME: 
@@ESTRING:IP: @port @NUMBER:PORT:@@ANYSTRING:EXTRA:@</pattern>
<!-- Disconnected from authenticating user root 192.168.0.3 port 52591 
[preauth] -->
<pattern>@ESTRING:EVENT: from @authenticating user @ESTRING:USERNAME: 
@@ESTRING:IP: @port @NUMBER:PORT:@@ANYSTRING:EXTRA:@</pattern>

To me, those 3 lines can be described using a single expression this way 
:
("Disconnected from") ("user"|"invalid user"|"authenticating user") 
(username) (ip_host) port (ip_port)(empty|extra_stuff)

Basically, the features I couldn't find are :
- "match a defined string and affect to variable"
- "match a string or another and affect to variable"
- "match a string or EOL and affect to variable if not empty".

Is it possible to have a single pattern that would lead to have:
- EVENT = "Disconnected from"
- METHOD = "user" | "invalid user" | "authenticating user"
- USERNAME = <parsed username>
- IP = <parsed ip address>
- PORT = <parsed port number>
- EXTRA = <empty> | <parsed extra information>

Thanks for you help.