[syslog-ng] Tag support in syslog-ng

[ILLES, Marton]

Hi,

In the recent discussion on db-parser() I mentioned that I would like to
add tag support for messages. The whole feature is not 100%, but here is
the first phase. I wrote about it in my blog:
http://marci.blogs.balabit.com/2009/05/tag-support-in-syslog-ng.html and
here it is:
 
I had some time so I created a tagging framework in syslog-ng. The
patches could be found in my syslog-ng git repo
(http://git.balabit.hu/?p=marci/syslog-ng-3.0.git;a=summary). At the
current state it is possible to create tags on the fly, attach them to
messages and latter match on them in a filter() expression. A tag is
simply identified by it's name, obviously these names must be unique.
Internally tags are stored as bitmask vector so it is very fast to
set/unset or match on them. Though as of now the internal representation
limits the number of tags to 8159. I think it ought to be enough for the
near future at least. I dare not to say that it ought to be enough for
anyone. :)

As of now messages could be tagged on sources only and messages are
automatically tagged on sources with a special tag representing the
source ".source.". Matching on the source this way is much faster than
matching on the ${SOURCE} macro as macro matching involves a string
comparison or even a regular expression matching while tag matching is a
simple bitwise comparison. (I am planning to add automatic tags on
sources for the source driver and other hopefully useful information.)

Here is an example on how to use it:

source s_tcp1 {
	tcp(ip(192.168.1.1) port(1514) tags("tcp", "router"));
};

source s_tcp2 {
	tcp(ip(192.168.1.2) port(1514) tags("tcp", "windows));
};

source s_udp1 {
	udp(ip(192.168.1.1) port(1514) tags("udp", "router"));
};

#Match messages from s_tcp1 source
filter f_tcp1 {
	tags(".source.s_tcp1");
};

filter f_router {
	tags("router");
};

#Match on tags "tcp" or "udp"
filter f_net {
	tags("tcp", "udp");
};

Next step is to add tagging support for the db-parser() so messages can
be tagged that way and add support for dynamic SQL table scheme for
tags. This w ay one can easily match and parse messages and put the
parsed information in to different SQL tables.

cheers,

Marton
-- 
Key fingerprint = F78C 25CA 5F88 6FAF EA21 779D 3279 9F9E 1155 670D