[syslog-ng] Bazsi's blog: Syslog-ng correllation

[Balazs Scheidler]

Syslog-ng correllation

I think we’ve reached an important milestone with syslog-ng: log message
correllation was added to db-parser(). As you probably know dbparser and
its sister project patterndb is able to transform unstructured syslog
messages into a normalized format: the human readable string content
becomes a set of name-value pairs. The problem is that in a lot of cases
messages miss one or two details that would really be needed to
understand them and this information usually comes in a followup
message.

For example: one message in postfix logs contain the sender address and
while the recipient information comes in the next message. It is trivial
to understand that in reality most cases you want the information in
sender,recipient pairs. Another example is sshd, where the
authentication failure comes in one and the exact reason for the failure
comes in the next.

Currently what you can do with syslog-ng is to put the separate messages
into two SQL tables and join them at query time. This gets ugly quite
fast: increased storage needs, the hassle with managing two tables
instead of one and not to mention the increase of the time needed to
query the database. Sometimes the sole reason for creating SQL tables in
this case is to perform the correllation, otherwise you’d be happier
with a CSV file.

And that’s what became possible now with the latest git commit of
syslog-ng 3.2. The idea is simple: when a patterndb rule matches, you
can tell syslog-ng to remember that message by adding it to a
correllation state. This state is identified with information extracted
from the message making it a unique session identifier. When the next
line comes in you can reference the information stored earlier.

Basically the correllation state is a list of log messages associated
with a session id. To add a new message to this state, you need a store
rule:

<rule id=”…”>
<patterns>
<pattern>foo session: @STRING:sessionid@, param:
@STRING:param@</pattern>
</patterns>
<store id=”$sessionid” timeout=”60″/> 
</rule>

The id attribute of the store element specifies a template containing
any syslog-ng name-value pairs, probably extracted from the current
message itself.

When the final information comes in you can use the join attribute of
the values tag:

<rule id=”…”>
<patterns>
<pattern>bar session: @STRING:sessionid@</pattern>
</patterns>
<values join=”$sessionid”>
<value name=”param”>${param}@1</value>
</values>
</rule>

here the join attribute specifies the session to look up (which must
match in the two messages), and if there’s a match all messages stored
in the correllation state becomes available when evaluating the
name-value pairs associated with the current message.

The key here is the new syntax in the template string “@1″ appended to a
name-value pair reference. After the “@” character, you can reference a
message in the correllation state by specifying the index backward from
the current message. This way @0 is the current message, @1 is the one
prior to the current one, @2 is before that and so on.

There are more complex ways to use/query the contents of the
correllation state, but those will appear in a followup post. Stay
tuned!



-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.balabit.hu/pipermail/syslog-ng/attachments/20100929/ecbbf5a3/attachment.htm