Hi Lars, First of all thanks for your message. Any kind of feedback is very much appreciated, basically these make me want to work on the code further :) So I'd like to urge everyone to post their opinions, they really make my day. Comments on your points are below, inserted into your message. On Wed, 2010-10-20 at 21:57 -0400, Lars Kellogg-Stedman wrote:
I've been playing with 3.2beta1 recently and getting my feet wet with the patterndb support, which I haven't had a chance to work with before. I have a few thoughts regarding the patterndb rule syntax, mostly targeted at making things a little bit easier to work with.
- Rule IDs
Is there any particular reason why unique IDs were selected as rule identifiers? They're not particularly meaningful to people, and they're hard to talk about. It's much easier to say, "we're suddently seeing lots of matches for ssh-accept-connection" than it is to say, "we're suddenly seeing lots of matches for 4dd5a329-da83-4876-a431-ddcb59c2858c". With the current version of syslog-ng it looks like it's possible to use arbitrary identifiers in place of UUIDs, and that's what I'm doing for my local rulesets.
This even makes classification metadata more useful, because .classifier.rule_id=ssh-accept-connection is immediately meaningful, while a UUID is useless unless I go grepping around the database.
Well, I don't really made up my mind how rule_id's should be used. They were proposed by Marci (who implemented the patterndb syntax in the first place). syslog-ng doesn't really care, but they must be unique. These are useful as we can attach a lot of information to the patterndb rule. For example, if you use a "<description>" tag, then you can retrieve this said description when you browse the log, based on the unique ID. This is what SSB (our syslog-ng appliance product) does for example.
- Whitespace
Log messages tend to be long, which makes them unwieldy in a number of situations. It would be nice if instead of this:
<pattern>...some very long log message that makes my life difficult and my patterns hard to read ...</pattern>
I could do this:
<pattern collapse_whitespace="yes">...some very long log message that has been conveniently wrapped to that it's easier to edit, email, and otherwise work with.</pattern>
Specifically, enabling "collapse_whitespace" would transform any sequence of whitespace to a single " ".
As proposed here this would be a completely backwords-compatible change because the default behavior would remain the same.
Interesting idea and of course doable, but then if there's indeed multiple spaces in the message, you get in trouble.
- Reusable patterns
We use Cisco firewall modules on our network. When these devices log a connection-related message, the source and destination address look something like this:
ircs:3610:67.186.94.126/41004
That's:
interface:vlan:ip/port
Which becomes:
@ESTRING:fwsm.src_if::@@NUMBER:fwsm.src_vlan@:@IPv4:fwsm.src_ip@/@NUMBER:fwsm.src_port@
When this occurs throughout the ruleset, and multiple times within a single message, it really lowers the readability of the rules. I wish there was a way to modularize this so that I could create custom types, something like this:
<type name="FWSM_ADDRSPEC"> @ESTRING:iface::@@NUMBER:vlan@:@IPv4:ip@/@NUMBER:fwsm.port@ </type>
And then do this:
<pattern>Accepted src @FWSM_ADDRSPEC:fwsm.src:@ dst @FWSM_ADDRSPEC:fwsm.dst:@</pattern>
And get this:
fwsm.src.iface fwsm.src.ip fwsm.src.port
This is a good idea. Although I'm a bit fiddling with the idea to extend the patterndb syntax a little bit. Nothing concrete yet, but reusable components will necessarily be a part of the picture. -- Bazsi