Hi, Is there a way to create a "group" of sources for log rules? This would be quite helpful when using the new include functionality for configuration files. To give you an idea of my setup: I have two files. One is a site config, which will be the same for every host. It contains global configuration and well-known destinations. The other file differs per-host, with different sources. I would like to be able to set up log rules for routing messages from those disparate sources to the well-known destinations inside the site config, but there doesn't seem to be any way to do it. Here's a pseudo-example. Site config: options { ... }; destination d_whatever { ... }; Host 1 config: source s_one { ... }; log { source(s_one); destination(d_whatever); }; Host 2 config: source s_two_alpha { ... }; source s_two_beta { ... }; log { source(s_two_alpha); source(s_two_beta); destination(d_whatever); }; Ideally, I'd be able to do something like this: Site config: options { ... }; destination d_whatever { ... }; log { source(s_aggregate); destination(d_whatever); }; Host 1 config: source s_one { ... }; source s_aggregate { source(s_one); }; Host 2 config: source s_two_alpha { ... }; source s_two_beta { ... }; source s_aggregate { source(s_two_alpha); source(s_two_beta); }; This setup maintains a nice, clean abstraction of the logging rules away from the source data. Is there any way to do something like this? Thanks, Joe
On Fri, 2009-02-06 at 18:12 -0500, Joe Shaw wrote:
Hi,
Is there a way to create a "group" of sources for log rules? This would be quite helpful when using the new include functionality for configuration files.
To give you an idea of my setup: I have two files. One is a site config, which will be the same for every host. It contains global configuration and well-known destinations. The other file differs per-host, with different sources. I would like to be able to set up log rules for routing messages from those disparate sources to the well-known destinations inside the site config, but there doesn't seem to be any way to do it. Here's a pseudo-example.
Site config: options { ... };
destination d_whatever { ... };
Host 1 config: source s_one { ... };
log { source(s_one); destination(d_whatever); };
Host 2 config: source s_two_alpha { ... }; source s_two_beta { ... };
log { source(s_two_alpha); source(s_two_beta); destination(d_whatever); };
Ideally, I'd be able to do something like this:
Site config: options { ... };
destination d_whatever { ... };
log { source(s_aggregate); destination(d_whatever); };
Host 1 config: source s_one { ... };
source s_aggregate { source(s_one); };
Host 2 config: source s_two_alpha { ... }; source s_two_beta { ... };
source s_aggregate { source(s_two_alpha); source(s_two_beta); };
This setup maintains a nice, clean abstraction of the logging rules away from the source data. Is there any way to do something like this?
Interesting idea. The only similar functionality what we have right now is the use of 'catch-all' flag. A catch-all log rule behaves as if _all_ sources would be specified. This is much less flexible what you have described though. Is this enough for you? -- Bazsi
Hi, On Sun, Feb 8, 2009 at 10:08 AM, Balazs Scheidler <bazsi@balabit.hu> wrote:
Interesting idea. The only similar functionality what we have right now is the use of 'catch-all' flag. A catch-all log rule behaves as if _all_ sources would be specified. This is much less flexible what you have described though.
Is this enough for you?
Not exactly; I'd like some sources (such as internal()) to go to a local file and not over the wire to the centralized host. Might be something to consider adding to a future roadmap... I'm sure it's not long enough. ;) Thanks, Joe
On Mon, 2009-02-09 at 11:36 -0500, Joe Shaw wrote:
Hi,
On Sun, Feb 8, 2009 at 10:08 AM, Balazs Scheidler <bazsi@balabit.hu> wrote:
Interesting idea. The only similar functionality what we have right now is the use of 'catch-all' flag. A catch-all log rule behaves as if _all_ sources would be specified. This is much less flexible what you have described though.
Is this enough for you?
Not exactly; I'd like some sources (such as internal()) to go to a local file and not over the wire to the centralized host.
Might be something to consider adding to a future roadmap... I'm sure it's not long enough. ;)
I'll see what I can do about this. What about the following idea: * make it possible to mark sources with tags * create a filter that matches tags Then you could do something like: source s_udp { udp(); tags(net); }; source s_tcp { tcp(); tags(net); }; filter f_net_messages { tags(net); }; log { flags(catch-all); filter(f_net_messages); destination(...); }; I'd have to work on the syntax a bit more, but I hope the general idea is visible. -- Bazsi
Hi, On Sun, Feb 15, 2009 at 4:31 AM, Balazs Scheidler <bazsi@balabit.hu> wrote:
I'll see what I can do about this. What about the following idea: * make it possible to mark sources with tags * create a filter that matches tags
Then you could do something like:
source s_udp { udp(); tags(net); }; source s_tcp { tcp(); tags(net); };
filter f_net_messages { tags(net); };
log { flags(catch-all); filter(f_net_messages); destination(...); };
I'd have to work on the syntax a bit more, but I hope the general idea is visible.
Ah, interesting. Yes, I think this would work just fine. Joe
On Sun, 2009-02-15 at 14:40 -0500, Joe Shaw wrote:
Hi,
On Sun, Feb 15, 2009 at 4:31 AM, Balazs Scheidler <bazsi@balabit.hu> wrote:
I'll see what I can do about this. What about the following idea: * make it possible to mark sources with tags * create a filter that matches tags
Then you could do something like:
source s_udp { udp(); tags(net); }; source s_tcp { tcp(); tags(net); };
filter f_net_messages { tags(net); };
log { flags(catch-all); filter(f_net_messages); destination(...); };
I'd have to work on the syntax a bit more, but I hope the general idea is visible.
Ah, interesting. Yes, I think this would work just fine.
I gave some more thought to that, and I think I'd assign these options to the log message, not the source group. This would make it possible to filter based on this value and also to change the tag assignments using a rewrite rule. I'd also assign the initial tags to the log source driver, not the source group. e.g. it would be something like this instead: source s_udp { udp(tags(net)); }; source s_tcp { udp(tags(net)); }; filter f_net_messages { tags(net); }; log { flags(catch-all); filter(f_net_messages); destination(...); }; On the internal side, I think I could make this at least as fast as comparing facility/priority values by using a dynamically allocated bitmask for the implementation, though that would limit the maximum number of different tags to 32. Hmm.. the same could be used for db-parser() classification. This idea seems to be more and more appealing to me. -- Bazsi
participants (2)
-
Balazs Scheidler
-
Joe Shaw