Re: [syslog-ng] Using regexp in match()
I'm trying to setup central syslog-ng server for my Exchange servers. On windows servers I use Epilog agent (brother of Snare) forwarding tracking logs to central syslog.
Say the format is: filed1\011field2\011, so fields in String are separated by '\011'.
I tried:
filter f_parse { match("([^\\011]*)\\011([^\\011]*)\\011"); };
Is this your last filter? The message remembers the matches of the last filter only so you have to ensure that this regexp is executed last.
Thanks for your answer Bazsi. Finally i got it. Obviously I have used bad regexp. Exchange tracking log uses tabs as delimiters. But when I saved $MSG string to text log, tabs was changed to '\011'. So now I changed my regexp to use tabs as delimiters: filter f_parsing { match("([^\t]*)\t([^\t]*)\t"); }; This works like charm and saves first two tab delimited fields (date and time in this case) to $1 and $2. I will wrote some HOWTO when i finish the configuration completely. Thaks for your time. Jan
------------ Původní zpráva ------------ Od: Jan Kreps <krepsj@seznam.cz> Předmět: Re: [syslog-ng] Using regexp in match() Datum: 10.7.2008 10:29:35 ----------------------------------------
I'm trying to setup central syslog-ng server for my Exchange servers. On windows servers I use Epilog agent (brother of Snare) forwarding tracking logs to central syslog.
Obviously I have used bad regexp. Exchange tracking log uses tabs as delimiters. But when I saved $MSG string to text log, tabs was changed to '\011'.
So now I changed my regexp to use tabs as delimiters:
filter f_parsing { match("([^\t]*)\t([^\t]*)\t"); };
This works like charm and saves first two tab delimited fields (date and time in this case) to $1 and $2.
Some more remarks to subject. Syslog-ng Administrator guide 1.1.1 Edition from May 2008 says on page 111: "The regular expressions can use up to 255 regexp matches ($1 ... $255)." and on page 112: "Regarding braces around macro names, the following two formats are equivalent "$MSG" and "${MSG}"." Fact is that for regexp matches greater than 9 it MUST be in ${} format. If you use for instance $12 it resolves as content of $1 + character '2'. I found that I have to use ${} syntax in ChangeLog remark for patch 137. I guess that should be stated in Admin Guide more clearly. In this respect, statement "The regular expressions can use up to 255 regexp matches ($1 ... $255)" is not true and should be corrected. And I have one question about file creation. When something is logged a file is created according to destination(). But when I delete the file, it's not created anymore, until restart of syslog-ng. Is that correct/expected behaviour? I run Debian Etch and syslog-ng 2.0.9-1 from unstable branch (in stable is still version 2.0.0) Greets Jan
when I delete the file, it's not created anymore, until restart of syslog-ng. Is that correct/expected behaviour?
I would guess that the answer is yes, because that's typical behavior for a Linux/Unix app. The rm command does not destroy files, it just removes references to them from directories. A file gets destroyed only after the last reference to it has been released (think of it like the destructor in C++). If an application (e.g. syslog-ng) has the file open for read or write, then the file handle in that app is another reference to the file. So the file still exists after rm (because there are still references to it) even though it is no longer listed in any directory. The destructor will be called when the last reference (the file handle) is released. Joe. -----Original Message----- From: syslog-ng-bounces@lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Jan Kreps Sent: 22 July 2008 08:45 To: Syslog-ng users' and developers' mailing list Subject: Re: [syslog-ng] Using regexp in match()
------------ Původní zpráva ------------ Od: Jan Kreps <krepsj@seznam.cz> Předmět: Re: [syslog-ng] Using regexp in match() Datum: 10.7.2008 10:29:35 ----------------------------------------
I'm trying to setup central syslog-ng server for my Exchange servers. On windows servers I use Epilog agent (brother of Snare) forwarding tracking logs to central syslog.
Obviously I have used bad regexp. Exchange tracking log uses tabs as delimiters. But when I saved $MSG string to text log, tabs was changed to '\011'.
So now I changed my regexp to use tabs as delimiters:
filter f_parsing { match("([^\t]*)\t([^\t]*)\t"); };
This works like charm and saves first two tab delimited fields (date and time in this case) to $1 and $2.
Some more remarks to subject. Syslog-ng Administrator guide 1.1.1 Edition from May 2008 says on page 111: "The regular expressions can use up to 255 regexp matches ($1 ... $255)." and on page 112: "Regarding braces around macro names, the following two formats are equivalent "$MSG" and "${MSG}"." Fact is that for regexp matches greater than 9 it MUST be in ${} format. If you use for instance $12 it resolves as content of $1 + character '2'. I found that I have to use ${} syntax in ChangeLog remark for patch 137. I guess that should be stated in Admin Guide more clearly. In this respect, statement "The regular expressions can use up to 255 regexp matches ($1 ... $255)" is not true and should be corrected. And I have one question about file creation. When something is logged a file is created according to destination(). But when I delete the file, it's not created anymore, until restart of syslog-ng. Is that correct/expected behaviour? I run Debian Etch and syslog-ng 2.0.9-1 from unstable branch (in stable is still version 2.0.0) Greets Jan ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html
------------ Původní zpráva ------------ Od: Fegan, Joe <Joe.Fegan@hp.com> Předmět: Re: [syslog-ng] Using regexp in match() Datum: 22.7.2008 16:23:01 ----------------------------------------
If an application (e.g. syslog-ng) has the file open for read or write, then the file handle in that app is another reference to the file. So the file still exists after rm (because there are still references to it) even though it is no longer listed in any directory. The destructor will be called when the last reference (the file handle) is released.
Joe.
Well, now I understand lot of things better. Thanks for your clear explanation Joe. Jan
On Tue, 2008-07-22 at 09:45 +0200, Jan Kreps wrote:
------------ Původní zpráva ------------ Od: Jan Kreps <krepsj@seznam.cz> Předmět: Re: [syslog-ng] Using regexp in match() Datum: 10.7.2008 10:29:35 ----------------------------------------
I'm trying to setup central syslog-ng server for my Exchange servers. On windows servers I use Epilog agent (brother of Snare) forwarding tracking logs to central syslog.
Obviously I have used bad regexp. Exchange tracking log uses tabs as delimiters. But when I saved $MSG string to text log, tabs was changed to '\011'.
So now I changed my regexp to use tabs as delimiters:
filter f_parsing { match("([^\t]*)\t([^\t]*)\t"); };
This works like charm and saves first two tab delimited fields (date and time in this case) to $1 and $2.
Some more remarks to subject.
Syslog-ng Administrator guide 1.1.1 Edition from May 2008 says on page 111:
"The regular expressions can use up to 255 regexp matches ($1 ... $255)."
and on page 112:
"Regarding braces around macro names, the following two formats are equivalent "$MSG" and "${MSG}"."
Fact is that for regexp matches greater than 9 it MUST be in ${} format. If you use for instance $12 it resolves as content of $1 + character '2'. I found that I have to use ${} syntax in ChangeLog remark for patch 137. I guess that should be stated in Admin Guide more clearly. In this respect, statement "The regular expressions can use up to 255 regexp matches ($1 ... $255)" is not true and should be corrected.
right, this should be fixed in the documentation.
And I have one question about file creation.
When something is logged a file is created according to destination(). But when I delete the file, it's not created anymore, until restart of syslog-ng. Is that correct/expected behaviour?
I run Debian Etch and syslog-ng 2.0.9-1 from unstable branch (in stable is still version 2.0.0)
Yes, you need to send the HUP signal to syslog-ng in order to reopen log files. This is the way UNIX works. (the application does not know about the deletion of a logfile) -- Bazsi
participants (3)
-
Balazs Scheidler
-
Fegan, Joe
-
Jan Kreps