setting facility/level of source log messages ?
Hi, I often meet the case where I need to centralize some purely applicative log messages (ie they don't even have a PRI header). So syslog-ng sets a default facility and severity for these messages, like kern facility and emergency severity for a source file with a "no-parse" option. Is there a clean way to change these default values by modifying the configuration file ? (and not using a source program() nor a tail | logger trick). I've already written a quick and dirty patch for my needs, but it would be great to have a native way of doing the same thing (I may have missed something in the documentation). Best Regards. -- Xavier
On Mon, 2009-03-02 at 15:11 +0100, Xavier Lapie wrote:
Hi,
I often meet the case where I need to centralize some purely applicative log messages (ie they don't even have a PRI header). So syslog-ng sets a default facility and severity for these messages, like kern facility and emergency severity for a source file with a "no-parse" option.
Is there a clean way to change these default values by modifying the configuration file ? (and not using a source program() nor a tail | logger trick).
I've already written a quick and dirty patch for my needs, but it would be great to have a native way of doing the same thing (I may have missed something in the documentation).
In 3.0 OSE there are options to set these for file sources, but no options to set them for no-parse sources. I'll take care about this when I get there, but please file a bugzilla ticket. Alternatively you could submit your patches, that could potentially solve this problem faster. -- Bazsi
On Mon, Mar 02, 2009 at 03:25:53PM +0100, Balazs Scheidler wrote:
In 3.0 OSE there are options to set these for file sources, but no options to set them for no-parse sources. I'll take care about this when I get there, but please file a bugzilla ticket.
Alternatively you could submit your patches, that could potentially solve this problem faster.
Hi, The patch is attached to this mail, and adds two new options for source files in the syslog-ng configuration file: fake_facility(facility) and fake_level(level) These options don't have any effect when used without the no-parse flag. Ex: source s_file { file("/tmp/f1" fake_facility(local7) fake_level(debug) flags(no-parse)); file("/tmp/f2" fake_facility(local4) fake_level(warn) flags(no-parse)); }; -- Xavier
Hi, Thanks for your contribution, it is really appreciated. However as I said the 3.0 branch already has a less general solution to the same problem. Could you update your patch against 3.0, making the file specific options LogReader specific? Also, since your patch is not in the 'trivial' category, I'd like to ask you to sign a copy of the 'Contributory License Agreement' and send it via e-mail (to me) or fax (to +3612080875). This allows me to include this patch into the commercial version of syslog-ng, needed to keep the two in sync. (I'm porting patches between OSE and PE). You can find this agreement here: https://www.balabit.com/dl/CLA_patch.pdf Thanks. On Tue, 2009-03-03 at 16:02 +0100, Xavier Lapie wrote:
On Mon, Mar 02, 2009 at 03:25:53PM +0100, Balazs Scheidler wrote:
In 3.0 OSE there are options to set these for file sources, but no options to set them for no-parse sources. I'll take care about this when I get there, but please file a bugzilla ticket.
Alternatively you could submit your patches, that could potentially solve this problem faster.
Hi,
The patch is attached to this mail, and adds two new options for source files in the syslog-ng configuration file:
fake_facility(facility) and fake_level(level) These options don't have any effect when used without the no-parse flag.
Ex: source s_file { file("/tmp/f1" fake_facility(local7) fake_level(debug) flags(no-parse)); file("/tmp/f2" fake_facility(local4) fake_level(warn) flags(no-parse)); };
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html
-- Bazsi
On Wed, Mar 04, 2009 at 10:19:55AM +0100, Balazs Scheidler wrote:
Hi,
Thanks for your contribution, it is really appreciated. However as I said the 3.0 branch already has a less general solution to the same problem. Could you update your patch against 3.0, making the file specific options LogReader specific?
Hi, I hope this version is ok for your 3.0 branch. -- Xavier
On Wed, 2009-03-04 at 14:26 +0100, Xavier Lapie wrote:
On Wed, Mar 04, 2009 at 10:19:55AM +0100, Balazs Scheidler wrote:
Hi,
Thanks for your contribution, it is really appreciated. However as I said the 3.0 branch already has a less general solution to the same problem. Could you update your patch against 3.0, making the file specific options LogReader specific?
Hi,
I hope this version is ok for your 3.0 branch.
I have added this functionality, but I'm afraid I basically rewrote your patch completely. Instead of using a separate fake_facility/fake_level keywords, I used the already existing facility/level options. Also, I have changed the behaviour slightly which covers your case but also fits to the syslog-ng model a bit better: instead of overwrite the facility/level value, the user can specify the default facility/level values, and it gets used when the message does not specify one (e.g. no-parse, or a message without a proper fac/level value) So from now on, it is possible to do things like: source aaa { udp(facility(syslog) level(emerg)); }; And if the message does not have a proper syslog header containing the above values, it'll use the ones specified in the configuration file. Now as I think of it, it might be better to really use a separate keyword for this, e.g. default-facility() and default-level() Any opinions? The patch in its current form can be found here: http://git.balabit.hu/?p=bazsi/syslog-ng-3.0.git;a=commit;h=2bddf91e2dc3b1f5... Tomorrow's nightly snapshot should contain it. -- Bazsi
On Tue, 2009-03-10 at 22:51 +0100, Balazs Scheidler wrote:
On Wed, 2009-03-04 at 14:26 +0100, Xavier Lapie wrote:
On Wed, Mar 04, 2009 at 10:19:55AM +0100, Balazs Scheidler wrote:
Hi,
Thanks for your contribution, it is really appreciated. However as I said the 3.0 branch already has a less general solution to the same problem. Could you update your patch against 3.0, making the file specific options LogReader specific?
Hi,
I hope this version is ok for your 3.0 branch.
I have added this functionality, but I'm afraid I basically rewrote your patch completely. Instead of using a separate fake_facility/fake_level keywords, I used the already existing facility/level options. Also, I have changed the behaviour slightly which covers your case but also fits to the syslog-ng model a bit better:
instead of overwrite the facility/level value, the user can specify the default facility/level values, and it gets used when the message does not specify one (e.g. no-parse, or a message without a proper fac/level value)
So from now on, it is possible to do things like:
source aaa { udp(facility(syslog) level(emerg)); };
And if the message does not have a proper syslog header containing the above values, it'll use the ones specified in the configuration file.
Now as I think of it, it might be better to really use a separate keyword for this, e.g. default-facility() and default-level()
Any opinions?
Hmm.. I was thinking about this feature a bit more, and it occurred to me that it would be possible to implement this also as a rewrite function, e.g. source src { ... }; rewrite r_fac { set("syslog" value("FACILITY")); set("crit" value("LEVEL")); }; I'm not sure that we need both features: * to specify the default priority value for any kind of log source * to make it possible to rewrite the value I find the 2nd more general. What do you all think? -- Bazsi
On Wed, Mar 11, 2009 at 08:20:03AM +0100, Balazs Scheidler wrote:
So from now on, it is possible to do things like:
source aaa { udp(facility(syslog) level(emerg)); };
And if the message does not have a proper syslog header containing the above values, it'll use the ones specified in the configuration file.
Now as I think of it, it might be better to really use a separate keyword for this, e.g. default-facility() and default-level() Any opinions?
Hi, Thanks ! I think it's better to have a separate keyword for this case.
Hmm.. I was thinking about this feature a bit more, and it occurred to me that it would be possible to implement this also as a rewrite function, e.g.
source src { ... };
rewrite r_fac { set("syslog" value("FACILITY")); set("crit" value("LEVEL")); };
I'm not sure that we need both features: * to specify the default priority value for any kind of log source * to make it possible to rewrite the value
I find the 2nd more general. What do you all think?
I had not though about the rewrite thing, but it sounds good too. The two features look good to me. At first I just wanted to set a value facility/level to messages who did not have one. But it could be usefull to have the more general one, with the rewrite rule. I just hope it is not a performance penalty to call a rewrite rule instead of just setting the facility/level when the LogMessage structure is created. By the way, talking about performance, I have started looking for using polling instead of follow-freq(). I'll really have many source log files to check. The more files are to be checked, the more stat() and lseek() are called by syslog-ng. It could be a performance gain to use poll() or epoll() (on linux at least for this one) on the source files to check for new messages. It seems that polling is only applicable on /proc/kmsg for now. Am I wrong or did I miss something (or both !) ? Regards. -- Xavier
On Wed, 2009-03-11 at 11:15 +0100, Xavier Lapie wrote:
On Wed, Mar 11, 2009 at 08:20:03AM +0100, Balazs Scheidler wrote:
So from now on, it is possible to do things like:
source aaa { udp(facility(syslog) level(emerg)); };
And if the message does not have a proper syslog header containing the above values, it'll use the ones specified in the configuration file.
Now as I think of it, it might be better to really use a separate keyword for this, e.g. default-facility() and default-level() Any opinions?
Hi,
Thanks ! I think it's better to have a separate keyword for this case.
Right, I'll change that.
Hmm.. I was thinking about this feature a bit more, and it occurred to me that it would be possible to implement this also as a rewrite function, e.g.
source src { ... };
rewrite r_fac { set("syslog" value("FACILITY")); set("crit" value("LEVEL")); };
I'm not sure that we need both features: * to specify the default priority value for any kind of log source * to make it possible to rewrite the value
I find the 2nd more general. What do you all think?
I had not though about the rewrite thing, but it sounds good too.
The two features look good to me. At first I just wanted to set a value facility/level to messages who did not have one. But it could be usefull to have the more general one, with the rewrite rule. I just hope it is not a performance penalty to call a rewrite rule instead of just setting the facility/level when the LogMessage structure is created.
sure the above example would be somewhat slower, as set() is operating on strings, thus the 'syslog' value would have to be converted to a numeric value at runtime. a less general rewrite() object would be quite fast though. e.g. rewrite r_fac { set-facility("syslog"); set-level("crit"); };
By the way, talking about performance, I have started looking for using polling instead of follow-freq(). I'll really have many source log files to check. The more files are to be checked, the more stat() and lseek() are called by syslog-ng. It could be a performance gain to use poll() or epoll() (on linux at least for this one) on the source files to check for new messages.
It seems that polling is only applicable on /proc/kmsg for now. Am I wrong or did I miss something (or both !) ?
poll() does not work for files, as regular files are always indicated as readable. And since epoll() uses the same infrastructure I doubt it would give you useful input. -- Bazsi
On Wed, 2009-03-11 at 22:45 +0100, Balazs Scheidler wrote:
On Wed, 2009-03-11 at 11:15 +0100, Xavier Lapie wrote:
On Wed, Mar 11, 2009 at 08:20:03AM +0100, Balazs Scheidler wrote:
So from now on, it is possible to do things like:
source aaa { udp(facility(syslog) level(emerg)); };
And if the message does not have a proper syslog header containing the above values, it'll use the ones specified in the configuration file.
Now as I think of it, it might be better to really use a separate keyword for this, e.g. default-facility() and default-level() Any opinions?
Hi,
Thanks ! I think it's better to have a separate keyword for this case.
Right, I'll change that.
This message is to let you know that I performed this change. This patch changes syslog-ng to use separate keywords for supplying default facility/level information for sources: commit 62dd98b5a803275954a09fc04feb6cea9ac7f5d1 Author: Balazs Scheidler <bazsi@balabit.hu> Date: Wed Apr 29 17:45:59 2009 +0200 [config parser] don't use facility/level keywords for specifying the default values for file sources There was an experimental feature to add support for controlling the default facility/priority values for sources in case the incoming message lacked this value. This used the same 'facility' and 'level' keywords but this was misleading as it does not override the value received if it was present. This patch changes it to use distinct keywords. -- Bazsi
participants (2)
-
Balazs Scheidler
-
bana＠docisland.org