On Thu, 2007-08-16 at 10:07 -0700, Eli Stair wrote:
I've got a problem with some network devices that is leading me to need to find some way to do regex rewriting of portions of the message body of syslog messages.
Problem: I've got a bunch of Foundry devices that put their hostname followed by a comma in the body of the message (and some that do not). Some of these look like this:
{ 2007-08-16:2007-08-16T09:50:16-07:00 hostname [hostname.local7.notice] hostname, Linecard Module 13 temperature 50.0 C degrees is normal }
# template("$R_ISODATE $HOST [$PROGRAM.$FACILITY.$PRIORITY] $MSG\n")
As you can see from the template, the second hostname reported with the comma is part of the MSG body. For reasons of properly searching/indexing this data I need to strip this out. I've seen mention of a tool called 'syslog-mailer' that sounds like it would do the job somewhat. Additionally, I've seen blog chatter about potentially adding full regex rewrite capability to syslog-ng in the recent past. I can't find evidence of either of these methods however.
My first try at solving this using an external program showed that when passing data OUT of syslog-ng to a defined program, only the message body is sent and before application of a template, the other information is dropped. Thus it's not possible to do processing of the whole payload externally, re-import the data via a socket and finish writing because the facility and HOST information is all gone!
I'm looking at writing a log proxy using Net::Dev::Tools::Syslog in perl to handle listening, rewriting if necessary, and forwarding full messages on to syslog-ng after. I'd just like to know if there are any better suggestions, or if this has been done before successfully in another way?
Cheers, and thanks for any insight.
You could do something like this: f_strip_hostname { match("^[\-a-zA-Z0-0]+,(.*)$") or match("^(.*)$"); }; destination d_out { file("/var/log/messages" template("$R_ISODATE $HOST [$PROGRAM.$FACILITY.$PRIORITY] $1\n"); }; The filter will make $1 equal to the message part without a hostname, either because there was a hostname (first match), or because there wasn't. Improving the hostname pattern would reduce ambiguity, as the pattern will strip everything till the first comma. -- Bazsi