<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 TRANSITIONAL//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; CHARSET=UTF-8">
<META NAME="GENERATOR" CONTENT="GtkHTML/3.28.3">
<TITLE>@@weblog</TITLE>
</HEAD>
<BODY>
<BR>
<H1>
<B><FONT SIZE="6"><A HREF="http://bazsi.blogs.balabit.com/2010/09/syslog-ng-correllation/">Syslog-ng correllation</A></FONT></B>
</H1>
I think we’ve reached an important milestone with syslog-ng: log message correllation was added to db-parser(). As you probably know dbparser and its sister project patterndb is able to transform unstructured syslog messages into a normalized format: the human readable string content becomes a set of name-value pairs. The problem is that in a lot of cases messages miss one or two details that would really be needed to understand them and this information usually comes in a followup message.<BR>
<BR>
For example: one message in postfix logs contain the sender address and while the recipient information comes in the next message. It is trivial to understand that in reality most cases you want the information in sender,recipient pairs. Another example is sshd, where the authentication failure comes in one and the exact reason for the failure comes in the next.<BR>
<BR>
Currently what you can do with syslog-ng is to put the separate messages into two SQL tables and join them at query time. This gets ugly quite fast: increased storage needs, the hassle with managing two tables instead of one and not to mention the increase of the time needed to query the database. Sometimes the sole reason for creating SQL tables in this case is to perform the correllation, otherwise you’d be happier with a CSV file.<BR>
<BR>
And that’s what became possible now with the latest git commit of syslog-ng 3.2. The idea is simple: when a patterndb rule matches, you can tell syslog-ng to remember that message by adding it to a correllation state. This state is identified with information extracted from the message making it a unique session identifier. When the next line comes in you can reference the information stored earlier.<BR>
<BR>
Basically the correllation state is a list of log messages associated with a session id. To add a new message to this state, you need a store rule:<BR>
<BR>
<rule id=”…”><BR>
<patterns><BR>
<pattern>foo session: @STRING:sessionid@, param: @STRING:param@</pattern><BR>
</patterns><BR>
<B><store id=”$sessionid” timeout=”60″/> </B><BR>
</rule><BR>
<BR>
The id attribute of the store element specifies a template containing any syslog-ng name-value pairs, probably extracted from the current message itself.<BR>
<BR>
When the final information comes in you can use the join attribute of the values tag:<BR>
<BR>
<rule id=”…”><BR>
<patterns><BR>
<pattern>bar session: @STRING:sessionid@</pattern><BR>
</patterns><BR>
<values <B>join=”$sessionid”</B>><BR>
<value name=”param”><B>${param}@1</B></value><BR>
</values><BR>
</rule><BR>
<BR>
here the join attribute specifies the session to look up (which must match in the two messages), and if there’s a match all messages stored in the correllation state becomes available when evaluating the name-value pairs associated with the current message.<BR>
<BR>
The key here is the new syntax in the template string “@1″ appended to a name-value pair reference. After the “@” character, you can reference a message in the correllation state by specifying the index backward from the current message. This way @0 is the current message, @1 is the one prior to the current one, @2 is before that and so on.<BR>
<BR>
There are more complex ways to use/query the contents of the correllation state, but those will appear in a followup post. Stay tuned!<BR>
<BR>
<BR>
<BR>
</BODY>
</HTML>