Help with embedded log paths
I need this configuration to do as the comments say but I can't figure out how to make it happen. Any pointers would be appreciated. Thanks, Mark log { source(default); # One of these first four should always match, if not $location=unknown log { filter(f_arc); rewrite(r_arc); }; log { filter(f_gsfc); rewrite(r_gsfc); }; log { filter(f_jsc); rewrite(r_jsc); }; log { filter(f_msfc); rewrite(r_msfc); }; ## The first one of these to match writes to disk and stops processing further log paths ## I don't know how this can be done since I can't add flags(final) inside of an embedded log path # Log Path for asa log { filter(f_asa); destination { file("/var/log/remote/data/${location:-unknown}/$(lowercase asa/${HOST}/${HOST}-asa.${UNIXTIME}.log)" create_dirs(yes) flags("threaded", "no-multi-line")); }; }; # Log Path for fmg log { filter(f_fmg); destination { file("/var/log/remote/data/${location:-unknown}/$(lowercase fmg/${HOST}/${HOST}-fmg.${UNIXTIME}.log)" create_dirs(yes) flags("threaded", "no-multi-line")); }; }; # Log Path for faz log { filter(f_faz); destination { file("/var/log/remote/data/${location:-unknown}/$(lowercase faz/${HOST}/${HOST}-faz.${UNIXTIME}.log)" create_dirs(yes) flags("threaded", "no-multi-line")); }; }; # Log Path for fw log { filter(f_fw); destination { file("/var/log/remote/data/${location:-unknown}/$(lowercase fw/${HOST}/${HOST}-fw.${UNIXTIME}.log)" create_dirs(yes) flags("threaded", "no-multi-line")); }; }; # Log Path for fw_block log { filter(f_fw_block); destination { file("/var/log/remote/data/${location:-unknown}/$(lowercase fw_block/${HOST}/${HOST}-fw_block.${UNIXTIME}.log)" create_dirs(yes) flags("threaded", "no-multi-line")); }; }; # Log Path for fortigate log { filter(f_fortigate); destination { file("/var/log/remote/data/${location:-unknown}/$(lowercase fortigate/${HOST}/${HOST}-fortigate.${UNIXTIME}.log)" create_dirs(yes) flags("threaded", "no-multi-line")); }; }; # Log Path for ips log { filter(f_ips); destination { file("/var/log/remote/data/${location:-unknown}/$(lowercase ips/${HOST}/${HOST}-ips.${UNIXTIME}.log)" create_dirs(yes) flags("threaded", "no-multi-line")); }; }; # Log Path for psa log { filter(f_psa); destination { file("/var/log/remote/data/${location:-unknown}/$(lowercase psa/${HOST}/${HOST}-psa.${UNIXTIME}.log)" create_dirs(yes) flags("threaded", "no-multi-line")); }; }; # Log Path for pubfw log { filter(f_pubfw); destination { file("/var/log/remote/data/${location:-unknown}/$(lowercase pubfw/${HOST}/${HOST}-pubfw.${UNIXTIME}.log)" create_dirs(yes) flags("threaded", "no-multi-line")); }; }; # Log Path for utm_other log { filter(f_utm_other); destination { file("/var/log/remote/data/${location:-unknown}/$(lowercase utm_other/${HOST}/${HOST}-utm_other.${UNIXTIME}.log)" create_dirs(yes) flags("threaded", "no-multi-line")); }; }; # Log Path for waf log { filter(f_waf); destination { file("/var/log/remote/data/${location:-unknown}/$(lowercase waf/${HOST}/${HOST}-waf.${UNIXTIME}.log)" create_dirs(yes) flags("threaded", "no-multi-line")); }; }; # Log Path for webfilter log { filter(f_webfilter); destination { file("/var/log/remote/data/${location:-unknown}/$(lowercase webfilter/${HOST}/${HOST}-webfilter.${UNIXTIME}.log)" create_dirs(yes) flags("threaded", "no-multi-line")); }; }; flags(final); };
Hi Mark, Seeing your two recent messages, I think you'd be better off converting your embedded logpath config to a conditional based one (with if-else), like described here: https://www.syslog-ng.com/technical-documents/doc/syslog-ng-open-source-edit... It's a much easier way to understannd what's happening, compared to the "legacy" way of doing with flags(final) etc.
Thank! That's essentially what I ended up doing. I need some sort of unique identifier that I can put on the file name in a destination that is static and not tied to the message or anything else that changes. It should be created when the file is created and not changed. I've looked over the macros but I don't see anything, does something like this exist? Thanks, -Mark -----Original Message----- From: syslog-ng <syslog-ng-bounces@lists.balabit.hu> On Behalf Of Fabien Wernli Sent: Wednesday, March 23, 2022 20:30 To: Syslog-ng users' and developers' mailing list <syslog-ng@lists.balabit.hu> Subject: [EXTERNAL] Re: [syslog-ng] Help with embedded log paths Hi Mark, Seeing your two recent messages, I think you'd be better off converting your embedded logpath config to a conditional based one (with if-else), like described here: https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.syslog-ng.com%2Ftechnical-documents%2Fdoc%2Fsyslog-ng-open-source-edition%2F3.33%2Fadministration-guide%2F58%23TOPIC-1663378&data=04%7C01%7Cmark.faine%40nasa.gov%7C2e7d5ce6aab74ece418608da0d360984%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637836823001406276%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=pMf9iuNbar5XV4vctRVGUzbVCiXK0bMK%2BhX394R6tnw%3D&reserved=0 It's a much easier way to understannd what's happening, compared to the "legacy" way of doing with flags(final) etc. ______________________________________________________________________________ Member info: https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.balabit.hu%2Fmailman%2Flistinfo%2Fsyslog-ng&data=04%7C01%7Cmark.faine%40nasa.gov%7C2e7d5ce6aab74ece418608da0d360984%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637836823001406276%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=cuu7WCWt6GokAY3ZuQZf86xuGut1lgxqODq9L0sVEfg%3D&reserved=0 Documentation: https://gcc02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fsupport%2Fdocumentation%2F%3Fproduct%3Dsyslog-ng&data=04%7C01%7Cmark.faine%40nasa.gov%7C2e7d5ce6aab74ece418608da0d360984%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637836823001406276%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=Kk%2BbNBHgkrPQGqBR%2Fqyv124vyXO9nEkN8sAROpSghy8%3D&reserved=0 FAQ: https://gcc02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fwiki%2Fsyslog-ng-faq&data=04%7C01%7Cmark.faine%40nasa.gov%7C2e7d5ce6aab74ece418608da0d360984%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637836823001406276%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=Z28icBg1tLi0hlhMcZIWlBVmhLQjdrYS8hTjxfnqxDE%3D&reserved=0
Hi Mark, Good to hear the if/else model suits you! On Thu, Mar 24, 2022 at 07:36:58PM +0000, Faine, Mark R. (MSFC-IS40)[NICS] wrote:
I need some sort of unique identifier that I can put on the file name in a destination that is static and not tied to the message or anything else that changes. It should be created when the file is created and not changed.
I've looked over the macros but I don't see anything, does something like this exist?
All the macros are tied to each and every message, and that's by design: routing is done per message. If you want something tied to a destination, all I can suggest is to set it as close as possible to that destination. For instance, you can set it *in* the destination as follows: destination d_foo { channel { rewrite { set("my_static_vaule", value("MY_STATIC_MACRO")}; }; destination { file("/path/to/dest" ...}; }; }; };
There is a ${FILE_NAME} macro: https://www.syslog-ng.com/technical-documents/doc/syslog-ng-open-source-edit... Or you could add a tag() to the source statement with a string literal, so only messages from that source will have that string. Gabor ________________________________ From: syslog-ng <syslog-ng-bounces@lists.balabit.hu> on behalf of Fabien Wernli <wernli@in2p3.fr> Sent: Friday, March 25, 2022 8:49 To: Faine, Mark R. (MSFC-IS40)[NICS] <mark.faine@nasa.gov> Cc: Syslog-ng users' and developers' mailing list <syslog-ng@lists.balabit.hu> Subject: Re: [syslog-ng] [EXTERNAL] Re: Help with embedded log paths CAUTION: This email originated from outside of the organization. Do not follow guidance, click links, or open attachments unless you recognize the sender and know the content is safe. Hi Mark, Good to hear the if/else model suits you! On Thu, Mar 24, 2022 at 07:36:58PM +0000, Faine, Mark R. (MSFC-IS40)[NICS] wrote:
I need some sort of unique identifier that I can put on the file name in a destination that is static and not tied to the message or anything else that changes. It should be created when the file is created and not changed.
I've looked over the macros but I don't see anything, does something like this exist?
All the macros are tied to each and every message, and that's by design: routing is done per message. If you want something tied to a destination, all I can suggest is to set it as close as possible to that destination. For instance, you can set it *in* the destination as follows: destination d_foo { channel { rewrite { set("my_static_vaule", value("MY_STATIC_MACRO")}; }; destination { file("/path/to/dest" ...}; }; }; }; ______________________________________________________________________________ Member info: https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.balabit.hu%2Fmailman%2Flistinfo%2Fsyslog-ng&data=04%7C01%7Cgabor.nagy%40oneidentity.com%7C281263658a24442eda3f08da0e3400b7%7C91c369b51c9e439c989c1867ec606603%7C0%7C1%7C637837913805954664%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=JwlArTRL1rTCbUHyADaEBOYtBhKkcJ1fQh%2FcU9qyANg%3D&reserved=0 Documentation: https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fsupport%2Fdocumentation%2F%3Fproduct%3Dsyslog-ng&data=04%7C01%7Cgabor.nagy%40oneidentity.com%7C281263658a24442eda3f08da0e3400b7%7C91c369b51c9e439c989c1867ec606603%7C0%7C1%7C637837913805954664%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=4lHfSUwrKwRGMGgQLWxNRNphYddMPfJrusHuZZ5BHME%3D&reserved=0 FAQ: https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fwiki%2Fsyslog-ng-faq&data=04%7C01%7Cgabor.nagy%40oneidentity.com%7C281263658a24442eda3f08da0e3400b7%7C91c369b51c9e439c989c1867ec606603%7C0%7C1%7C637837913805954664%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=s94ztS49SqoXTnM4MhPFaThvHSGFJQu94yOx8DkMwW4%3D&reserved=0
participants (3)
-
Fabien Wernli
-
Faine, Mark R. (MSFC-IS40)[NICS]
-
Gabor Nagy (gnagy)