Value is dropped or unset in resolved destination template
I have a destination template in my destination (d_default) that uses $location destination d_default { file("`BASEPATH`/$location/$(lowercase ${HOST})/$app/$(lowercase ${HOST})_$app.log" create_dirs(yes) flags("threaded", "no-multi-line")); }; That is set by a series of filters that look like this log { if { filter... rewrite { set("foo" value("location")); }; }; if { filter... rewrite { set("bar" value("location")); }; }; ... }; This appears to be working, I can see it in the logs: Setting value; name='location', value='unknown', msg='0x7f190c001e00' # Unknown is a default value that would be appropriate on this testing server Later additional log paths are applied like so: log { source(default); filter(f_asa); rewrite(r_asa); destination(d_default); flags(final ); }; ... log { source(default); filter(f_fw); rewrite(r_fw); destination(d_default); flags(final ); }; ... However, when I go to the destination on the filesystem the location is empty so the files are being written as though location was this: ("`BASEPATH`//$(lowercase ${HOST})/$app/$(lowercase ${HOST})_$app.log" Any ideas what could be causing this? Thanks, Mark
Hi Mark, It's really hard to tell what's happening without seeing your full configuration. Remember messages can go through multiple logpaths, some of which the variables are probably empty in. That being said, if I were you I'd use a default value for your macros in any case, much safer e.g.: destination d_default { file("`BASEPATH`/${location:-hidden}/$(lowercase ${HOST})/${app:-unknown}/$(lowercase ${HOST})_${app:-unknown}.log" create_dirs(yes) flags("threaded", "no-multi-line")); };
Hi Mark, Fabien is right, you should have a default value. You are using the rewrite rules that set $location inside an if statement. Maybe what happens is that in some cases a log doesn't match which leads to an unset $location. Don't you have directories with $location's value too? So you have both: ("`BASEPATH`//$(lowercase ${HOST})/$app/$(lowercase ${HOST})_$app.log" ("`BASEPATH`/$location/$(lowercase ${HOST})/$app/$(lowercase ${HOST})_$app.log" Or $location is always empty on the destination side? Gabor Fabien Wernli <wernli@in2p3.fr> ezt írta (időpont: 2022. márc. 23., Sze, 15:04):
Hi Mark,
It's really hard to tell what's happening without seeing your full configuration. Remember messages can go through multiple logpaths, some of which the variables are probably empty in.
That being said, if I were you I'd use a default value for your macros in any case, much safer e.g.:
destination d_default { file("`BASEPATH`/${location:-hidden}/$(lowercase ${HOST})/${app:-unknown}/$(lowercase ${HOST})_${app:-unknown}.log" create_dirs(yes) flags("threaded", "no-multi-line")); };
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
Can I OR filters when they are in the form filter(filter_name); such as filter(f_foo); or filter(f_bar); or does it have to be the long form: filter { message='foo' or message='bar' } The problem I'm having is that my filters are very large and I need to compare four of them for each message on the log path and so I don't want to write them inline inside of the log path. Thanks, -Mark From: syslog-ng <syslog-ng-bounces@lists.balabit.hu> On Behalf Of Nagy Gábor Sent: Wednesday, March 23, 2022 10:03 To: wernli@in2p3.fr; Syslog-ng users' and developers' mailing list <syslog-ng@lists.balabit.hu> Subject: [EXTERNAL] Re: [syslog-ng] Value is dropped or unset in resolved destination template Hi Mark, Fabien is right, you should have a default value. You are using the rewrite rules that set $location inside an if statement. Maybe what happens is that in some cases a log doesn't match which leads to an unset $location. Don't you have directories with $location's value too? So you have both: ("`BASEPATH`//$(lowercase ${HOST})/$app/$(lowercase ${HOST})_$app.log" ("`BASEPATH`/$location/$(lowercase ${HOST})/$app/$(lowercase ${HOST})_$app.log" Or $location is always empty on the destination side? Gabor Fabien Wernli <wernli@in2p3.fr<mailto:wernli@in2p3.fr>> ezt írta (időpont: 2022. márc. 23., Sze, 15:04): Hi Mark, It's really hard to tell what's happening without seeing your full configuration. Remember messages can go through multiple logpaths, some of which the variables are probably empty in. That being said, if I were you I'd use a default value for your macros in any case, much safer e.g.: destination d_default { file("`BASEPATH`/${location:-hidden}/$(lowercase ${HOST})/${app:-unknown}/$(lowercase ${HOST})_${app:-unknown}.log" create_dirs(yes) flags("threaded", "no-multi-line")); }; ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng<https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.balabit.hu%2Fmailman%2Flistinfo%2Fsyslog-ng&data=04%7C01%7Cmark.faine%40nasa.gov%7Ca118647407a341d07a4b08da0cde8599%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637836447117996806%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=H9Jf26xApIJN0ZMU2N26IytL5I79GdvyKzizpgPhsLM%3D&reserved=0> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng<https://gcc02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fsupport%2Fdocumentation%2F%3Fproduct%3Dsyslog-ng&data=04%7C01%7Cmark.faine%40nasa.gov%7Ca118647407a341d07a4b08da0cde8599%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637836447117996806%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=3gpPnIPgiHbCjQ%2F%2BXK86kv76yRJDcwW2CvernTCdgno%3D&reserved=0> FAQ: http://www.balabit.com/wiki/syslog-ng-faq<https://gcc02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fwiki%2Fsyslog-ng-faq&data=04%7C01%7Cmark.faine%40nasa.gov%7Ca118647407a341d07a4b08da0cde8599%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637836447117996806%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=R%2F6NGiYDh7ykOY7NG6PXRmihVb2%2BWZLv%2BiQdmsGa8vE%3D&reserved=0>
This should work: log { source(s_network); filter { filter(f_1) or filter(f_2) }; destination(d_syslog); }; Regards, Gabor ________________________________ From: syslog-ng <syslog-ng-bounces@lists.balabit.hu> on behalf of Faine, Mark R. (MSFC-IS40)[NICS] <mark.faine@nasa.gov> Sent: Wednesday, March 23, 2022 22:36 To: Syslog-ng users' and developers' mailing list <syslog-ng@lists.balabit.hu> Subject: Re: [syslog-ng] [EXTERNAL] Re: Value is dropped or unset in resolved destination template CAUTION: This email originated from outside of the organization. Do not follow guidance, click links, or open attachments unless you recognize the sender and know the content is safe. Can I OR filters when they are in the form filter(filter_name); such as filter(f_foo); or filter(f_bar); or does it have to be the long form: filter { message='foo' or message='bar' } The problem I'm having is that my filters are very large and I need to compare four of them for each message on the log path and so I don't want to write them inline inside of the log path. Thanks, -Mark From: syslog-ng <syslog-ng-bounces@lists.balabit.hu> On Behalf Of Nagy Gábor Sent: Wednesday, March 23, 2022 10:03 To: wernli@in2p3.fr; Syslog-ng users' and developers' mailing list <syslog-ng@lists.balabit.hu> Subject: [EXTERNAL] Re: [syslog-ng] Value is dropped or unset in resolved destination template Hi Mark, Fabien is right, you should have a default value. You are using the rewrite rules that set $location inside an if statement. Maybe what happens is that in some cases a log doesn't match which leads to an unset $location. Don't you have directories with $location's value too? So you have both: ("`BASEPATH`//$(lowercase ${HOST})/$app/$(lowercase ${HOST})_$app.log" ("`BASEPATH`/$location/$(lowercase ${HOST})/$app/$(lowercase ${HOST})_$app.log" Or $location is always empty on the destination side? Gabor Fabien Wernli <wernli@in2p3.fr<mailto:wernli@in2p3.fr>> ezt írta (időpont: 2022. márc. 23., Sze, 15:04): Hi Mark, It's really hard to tell what's happening without seeing your full configuration. Remember messages can go through multiple logpaths, some of which the variables are probably empty in. That being said, if I were you I'd use a default value for your macros in any case, much safer e.g.: destination d_default { file("`BASEPATH`/${location:-hidden}/$(lowercase ${HOST})/${app:-unknown}/$(lowercase ${HOST})_${app:-unknown}.log" create_dirs(yes) flags("threaded", "no-multi-line")); }; ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.balabit.hu%2Fmailman%2Flistinfo%2Fsyslog-ng&data=04%7C01%7Cgabor.nagy%40oneidentity.com%7C33b3db51f5524029fb2808da0d15243b%7C91c369b51c9e439c989c1867ec606603%7C0%7C1%7C637836681707286925%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=n5y0f6MjhGi%2FM1La56oY%2FoRyOcAqb5eM3TQg0Vs3ly4%3D&reserved=0> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng<https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fsupport%2Fdocumentation%2F%3Fproduct%3Dsyslog-ng&data=04%7C01%7Cgabor.nagy%40oneidentity.com%7C33b3db51f5524029fb2808da0d15243b%7C91c369b51c9e439c989c1867ec606603%7C0%7C1%7C637836681707286925%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=t8voTU8qjP0RrdGPOQqDOjLkMAtxfkhcv%2BY6AYBN0hI%3D&reserved=0> FAQ: http://www.balabit.com/wiki/syslog-ng-faq<https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fwiki%2Fsyslog-ng-faq&data=04%7C01%7Cgabor.nagy%40oneidentity.com%7C33b3db51f5524029fb2808da0d15243b%7C91c369b51c9e439c989c1867ec606603%7C0%7C1%7C637836681707286925%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=crU0bjlG3N%2Fsy2pJ0buOVz8LkTTsaaQ4HXUbIdqxIrc%3D&reserved=0>
Thanks, I understand. It's just hard sending this kind of stuff without potentially inadvertently sharing something I'm not supposed to. Either way, I think I have worked around the issue. -Mark -----Original Message----- From: syslog-ng <syslog-ng-bounces@lists.balabit.hu> On Behalf Of Fabien Wernli Sent: Wednesday, March 23, 2022 09:05 To: Syslog-ng users' and developers' mailing list <syslog-ng@lists.balabit.hu> Subject: [EXTERNAL] Re: [syslog-ng] Value is dropped or unset in resolved destination template Hi Mark, It's really hard to tell what's happening without seeing your full configuration. Remember messages can go through multiple logpaths, some of which the variables are probably empty in. That being said, if I were you I'd use a default value for your macros in any case, much safer e.g.: destination d_default { file("`BASEPATH`/${location:-hidden}/$(lowercase ${HOST})/${app:-unknown}/$(lowercase ${HOST})_${app:-unknown}.log" create_dirs(yes) flags("threaded", "no-multi-line")); }; ______________________________________________________________________________ Member info: https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.balabit.hu%2Fmailman%2Flistinfo%2Fsyslog-ng&data=04%7C01%7Cmark.faine%40nasa.gov%7Cdf3b7016170246b9e56f08da0cd649bf%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637836411758844542%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=ALqH3MyXmGYAeqms%2FZusFf1CATjfyvt6sqwVc7Hta5E%3D&reserved=0 Documentation: https://gcc02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fsupport%2Fdocumentation%2F%3Fproduct%3Dsyslog-ng&data=04%7C01%7Cmark.faine%40nasa.gov%7Cdf3b7016170246b9e56f08da0cd649bf%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637836411758844542%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=9ytDQzSFwkdc1QeC%2B8Y3tTCQoZLdgeh586YrHscdhVQ%3D&reserved=0 FAQ: https://gcc02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fwiki%2Fsyslog-ng-faq&data=04%7C01%7Cmark.faine%40nasa.gov%7Cdf3b7016170246b9e56f08da0cd649bf%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637836411758844542%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=fD7jsZOiRMyM8vJpr3GcWbcNVdR8Guieerhuepjy3Ss%3D&reserved=0
participants (4)
-
Fabien Wernli
-
Faine, Mark R. (MSFC-IS40)[NICS]
-
Gabor Nagy (gnagy)
-
Nagy Gábor