Regular expressions in rewrite
Is there a way to use regular expressions when setting a value in a rewrite rule? Something like: rewrite r_rewrite_set_host{ subst("^[a-z]+\-([a-z]+).*$", "$1", value("HOST")); set("HOST" value("location") condition( filter(f_filter_hostnames) ) ); }; But without changing the actual log message. The goal is to create a variable from a regular expression of the host field in the message that I can then use in a destination path. Thanks, -Mark Mark Faine System Administrator SAIC/NICS 215 Wynn Dr. 5065 Huntsville, AL 35805 256-961-1295 (Desk) 256-617-4861 (Work Cell)
On Fri, Aug 02, 2019 at 01:40:13PM +0000, Faine, Mark R. (MSFC-IS40)[NICS] wrote:
Is there a way to use regular expressions when setting a value in a rewrite rule? [...] But without changing the actual log message. The goal is to create a variable from a regular expression of the host field in the message that I can then use in a destination path.
You could copy the content of HOST into another macro, then use subst on the copy: rewrite r_rewrite_set_host{ set("$HOST", value("copy_of_host")); subst("^[a-z]+\-([a-z]+).*$", "$1", value("copy_of_host")); set("$copy_of_host" value("location") condition( filter(f_filter_hostnames) ) ); unset('copy_of_host'); } Note that you could also use named matches in subst: subst("^[a-z]+\-(?<location>[a-z]+).*$", "$1", value("copy_of_host"));
I would rephrase your question: Is there a way to extract information with regular expression out of a message without changing the given value? In short: yes, but not in one step. I would do something similar to Fabien's answer, you need to copy the given value to protect it. I was thinking on how can you save some steps, maybe with using the regex capture groups $0, $1 variables, E.g. filter { match("^[a-z]+\-([a-z]+).*$", value("HOST") flags(store-matches)); } rewrite { set("$1" value("location")); }; Or how to save using "copy_of_host" temporary variable, but in every case you end up with additional complexity. Regards, Gabor ________________________________ From: syslog-ng <syslog-ng-bounces@lists.balabit.hu> on behalf of Fabien Wernli <wernli@in2p3.fr> Sent: Monday, August 5, 2019 10:12 To: syslog-ng@lists.balabit.hu <syslog-ng@lists.balabit.hu> Subject: Re: [syslog-ng] Regular expressions in rewrite CAUTION: This email originated from outside of the organization. Do not follow guidance, click links, or open attachments unless you recognize the sender and know the content is safe. On Fri, Aug 02, 2019 at 01:40:13PM +0000, Faine, Mark R. (MSFC-IS40)[NICS] wrote:
Is there a way to use regular expressions when setting a value in a rewrite rule? [...] But without changing the actual log message. The goal is to create a variable from a regular expression of the host field in the message that I can then use in a destination path.
You could copy the content of HOST into another macro, then use subst on the copy: rewrite r_rewrite_set_host{ set("$HOST", value("copy_of_host")); subst("^[a-z]+\-([a-z]+).*$", "$1", value("copy_of_host")); set("$copy_of_host" value("location") condition( filter(f_filter_hostnames) ) ); unset('copy_of_host'); } Note that you could also use named matches in subst: subst("^[a-z]+\-(?<location>[a-z]+).*$", "$1", value("copy_of_host")); ______________________________________________________________________________ Member info: https://nam05.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.balabit.hu%2Fmailman%2Flistinfo%2Fsyslog-ng&data=02%7C01%7Cgabor.nagy%40oneidentity.com%7C9ae0dd2587e04543dfe208d7197c9f22%7C91c369b51c9e439c989c1867ec606603%7C0%7C0%7C637005895380866512&sdata=GegzgWOACep%2B4YsTraFe%2F7MZ3%2BWNsb1x%2FHJtasU5qpc%3D&reserved=0 Documentation: https://nam05.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fsupport%2Fdocumentation%2F%3Fproduct%3Dsyslog-ng&data=02%7C01%7Cgabor.nagy%40oneidentity.com%7C9ae0dd2587e04543dfe208d7197c9f22%7C91c369b51c9e439c989c1867ec606603%7C0%7C0%7C637005895380866512&sdata=uIo3wnwp9BLyVWNiMK8CrveMuQkm5pihfs3wdvSOjHk%3D&reserved=0 FAQ: https://nam05.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fwiki%2Fsyslog-ng-faq&data=02%7C01%7Cgabor.nagy%40oneidentity.com%7C9ae0dd2587e04543dfe208d7197c9f22%7C91c369b51c9e439c989c1867ec606603%7C0%7C0%7C637005895380876506&sdata=u7Ol9JwXwLNuEZDmiZtE7qRZp3mix73IUBH9ORyUYII%3D&reserved=0
Thanks, this seems to work, though I had to change the unset to: unset(value("copy_of_host")) I can't find anything in the docs for unset without using value(). Is this correct or does it change what is being done? Thanks, -Mark -----Original Message----- From: syslog-ng <syslog-ng-bounces@lists.balabit.hu> On Behalf Of Fabien Wernli Sent: Monday, August 5, 2019 03:12 To: syslog-ng@lists.balabit.hu Subject: [EXTERNAL] Re: [syslog-ng] Regular expressions in rewrite On Fri, Aug 02, 2019 at 01:40:13PM +0000, Faine, Mark R. (MSFC-IS40)[NICS] wrote:
Is there a way to use regular expressions when setting a value in a rewrite rule? [...] But without changing the actual log message. The goal is to create a variable from a regular expression of the host field in the message that I can then use in a destination path.
You could copy the content of HOST into another macro, then use subst on the copy: rewrite r_rewrite_set_host{ set("$HOST", value("copy_of_host")); subst("^[a-z]+\-([a-z]+).*$", "$1", value("copy_of_host")); set("$copy_of_host" value("location") condition( filter(f_filter_hostnames) ) ); unset('copy_of_host'); } Note that you could also use named matches in subst: subst("^[a-z]+\-(?<location>[a-z]+).*$", "$1", value("copy_of_host")); ______________________________________________________________________________ Member info: https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.balabit.hu_mailma... Documentation: https://urldefense.proofpoint.com/v2/url?u=http-3A__www.balabit.com_support_... FAQ: https://urldefense.proofpoint.com/v2/url?u=http-3A__www.balabit.com_wiki_sys...
Hi Mark, You are right, unset(value("copy_of_host")) is the way to go. 🙂 Regards, Attila ________________________________ From: syslog-ng <syslog-ng-bounces@lists.balabit.hu> on behalf of Faine, Mark R. (MSFC-IS40)[NICS] <mark.faine@nasa.gov> Sent: Thursday, September 5, 2019 3:17 PM To: wernli@in2p3.fr <wernli@in2p3.fr>; Syslog-ng users' and developers' mailing list <syslog-ng@lists.balabit.hu> Subject: Re: [syslog-ng] [EXTERNAL] Re: Regular expressions in rewrite CAUTION: This email originated from outside of the organization. Do not follow guidance, click links, or open attachments unless you recognize the sender and know the content is safe. Thanks, this seems to work, though I had to change the unset to: unset(value("copy_of_host")) I can't find anything in the docs for unset without using value(). Is this correct or does it change what is being done? Thanks, -Mark -----Original Message----- From: syslog-ng <syslog-ng-bounces@lists.balabit.hu> On Behalf Of Fabien Wernli Sent: Monday, August 5, 2019 03:12 To: syslog-ng@lists.balabit.hu Subject: [EXTERNAL] Re: [syslog-ng] Regular expressions in rewrite On Fri, Aug 02, 2019 at 01:40:13PM +0000, Faine, Mark R. (MSFC-IS40)[NICS] wrote:
Is there a way to use regular expressions when setting a value in a rewrite rule? [...] But without changing the actual log message. The goal is to create a variable from a regular expression of the host field in the message that I can then use in a destination path.
You could copy the content of HOST into another macro, then use subst on the copy: rewrite r_rewrite_set_host{ set("$HOST", value("copy_of_host")); subst("^[a-z]+\-([a-z]+).*$", "$1", value("copy_of_host")); set("$copy_of_host" value("location") condition( filter(f_filter_hostnames) ) ); unset('copy_of_host'); } Note that you could also use named matches in subst: subst("^[a-z]+\-(?<location>[a-z]+).*$", "$1", value("copy_of_host")); ______________________________________________________________________________ Member info: https://nam05.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.balabit.hu%2Fmailman%2Flistinfo%2Fsyslog-ng&data=02%7C01%7Cattila.szakacs%40oneidentity.com%7C606d92cc831042c4c0e608d73203713c%7C91c369b51c9e439c989c1867ec606603%7C0%7C0%7C637032862689057245&sdata=sotKBOfA%2BKH7rCQn6g7Q9k8kbKeRysWAHXWUTpKbtwU%3D&reserved=0= Documentation: https://nam05.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fsupport%2Fdocumentation%2F%3Fproduct%3Dsyslog-ng&data=02%7C01%7Cattila.szakacs%40oneidentity.com%7C606d92cc831042c4c0e608d73203713c%7C91c369b51c9e439c989c1867ec606603%7C0%7C0%7C637032862689057245&sdata=ReyBNEcyBUCAOrOmw4utFVYnbTKc8aqqjXST%2FoTZIZ8%3D&reserved=0= FAQ: https://nam05.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fwiki%2Fsyslog-ng-faq&data=02%7C01%7Cattila.szakacs%40oneidentity.com%7C606d92cc831042c4c0e608d73203713c%7C91c369b51c9e439c989c1867ec606603%7C0%7C0%7C637032862689057245&sdata=%2BJNBJmGRhREZfQk8FEMJT7AMR71LXwcZzEUXbCidVB0%3D&reserved=0= ______________________________________________________________________________ Member info: https://nam05.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.balabit.hu%2Fmailman%2Flistinfo%2Fsyslog-ng&data=02%7C01%7Cattila.szakacs%40oneidentity.com%7C606d92cc831042c4c0e608d73203713c%7C91c369b51c9e439c989c1867ec606603%7C0%7C0%7C637032862689057245&sdata=sotKBOfA%2BKH7rCQn6g7Q9k8kbKeRysWAHXWUTpKbtwU%3D&reserved=0 Documentation: https://nam05.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fsupport%2Fdocumentation%2F%3Fproduct%3Dsyslog-ng&data=02%7C01%7Cattila.szakacs%40oneidentity.com%7C606d92cc831042c4c0e608d73203713c%7C91c369b51c9e439c989c1867ec606603%7C0%7C0%7C637032862689057245&sdata=ReyBNEcyBUCAOrOmw4utFVYnbTKc8aqqjXST%2FoTZIZ8%3D&reserved=0 FAQ: https://nam05.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fwiki%2Fsyslog-ng-faq&data=02%7C01%7Cattila.szakacs%40oneidentity.com%7C606d92cc831042c4c0e608d73203713c%7C91c369b51c9e439c989c1867ec606603%7C0%7C0%7C637032862689057245&sdata=%2BJNBJmGRhREZfQk8FEMJT7AMR71LXwcZzEUXbCidVB0%3D&reserved=0
participants (4)
-
Attila Szakacs (aszakacs)
-
Fabien Wernli
-
Faine, Mark R. (MSFC-IS40)[NICS]
-
Gabor Nagy (gnagy)