Feature request: cleaned-up PROGRAM macro?
Hi there As part of last weeks discussion over how to scale a searchable Enterprise Syslog environment, I've been wondering how far I could get using syslog-ng's built in functions - such as the templates. So I think it could be useful to be able to filter syslog messages into the following structure - as it gives an immediate "sorted" view: destination d_dir_messages { file("/var/log/syslog/$HOST/$YEAR/$MONTH/$DAY/$PROGRAM/raw" template("$R_ISODATE $MSGONLY\n") ...etc }; You can see you could now search quickly (well, quicker ;-) by date, syslog host and program. Symlinks could be used to create "meta directories" of related terms - e.g. an "Email/" dir could contain symlinks to all dirs containing $PROGRAM dirs called "smtpd", "qmail", "postfix", "msexchange", etc. However - there's a big nasty warning on PROGRAM - it's raw. So it could contain '/', '`' - anything. So I went to do it "the right way" and create my own "program" destination device that would clean things up and create "sanitized" directories. But it's too hard! I have to take on all the open file management that syslog-ng automatically does for me, etc. Either than or resort to "open-write-close" type thingies - yucky :-( So my feature request would be that there be a CLEANPROGRAM macro - or the like. Something that simply removes nasty chars such as "`|;'"\/}]{[&*#!" would probably do. The better option would to declare what was valid - but then you'd run into the whole Unicode/non-ASCII thing - just what is "invalid" these days? Thanks for listening -- Cheers Jason Haar Information Security Manager, Trimble Navigation Ltd. Phone: +64 3 9635 377 Fax: +64 3 9635 417 PGP Fingerprint: 7A2E 0407 C9A6 CAF6 2B9F 8422 C063 5EBB FE1D 66D1
Urgh! Not as easy as I made out. Anyone seen what Cisco thinks passes for PROGRAM names? I'm afraid those names need serious massaging :-( Jason Haar wrote:
Hi there
As part of last weeks discussion over how to scale a searchable Enterprise Syslog environment, I've been wondering how far I could get using syslog-ng's built in functions - such as the templates.
So I think it could be useful to be able to filter syslog messages into the following structure - as it gives an immediate "sorted" view:
destination d_dir_messages { file("/var/log/syslog/$HOST/$YEAR/$MONTH/$DAY/$PROGRAM/raw" template("$R_ISODATE $MSGONLY\n") ...etc };
You can see you could now search quickly (well, quicker ;-) by date, syslog host and program. Symlinks could be used to create "meta directories" of related terms - e.g. an "Email/" dir could contain symlinks to all dirs containing $PROGRAM dirs called "smtpd", "qmail", "postfix", "msexchange", etc.
However - there's a big nasty warning on PROGRAM - it's raw. So it could contain '/', '`' - anything.
So I went to do it "the right way" and create my own "program" destination device that would clean things up and create "sanitized" directories. But it's too hard! I have to take on all the open file management that syslog-ng automatically does for me, etc. Either than or resort to "open-write-close" type thingies - yucky :-(
So my feature request would be that there be a CLEANPROGRAM macro - or the like. Something that simply removes nasty chars such as "`|;'"\/}]{[&*#!" would probably do. The better option would to declare what was valid - but then you'd run into the whole Unicode/non-ASCII thing - just what is "invalid" these days?
Thanks for listening
-- Cheers Jason Haar Information Security Manager, Trimble Navigation Ltd. Phone: +64 3 9635 377 Fax: +64 3 9635 417 PGP Fingerprint: 7A2E 0407 C9A6 CAF6 2B9F 8422 C063 5EBB FE1D 66D1
participants (1)
-
Jason Haar