[syslog-ng] [EXTERNAL] Re: using VARARGS correctly
Attila Szakacs (aszakacs)
Attila.Szakacs at oneidentity.com
Thu May 20 08:03:16 UTC 2021
Hi Mark,
I'm testing it with 3.19.1 and the following config:
@version: 3.19
block destination default_file(
basepath('/tmp')
location("")
app("")
name("")
file_path_args("")
) {
file(
"`basepath`/`location`/`app`/${HOST}/${HOST}_`app``file_path_args`.log"
persist-name(`name`)
create_dirs(yes)
flags("threaded", "no-multi-line")
);
};
destination d_default {
default_file(
basepath(`BASEPATH`)
location("my_location")
app("my_app")
name('d_default_udp')
file_path_args("${session}${some_other_arg}")
);
};
destination d_default_2 {
default_file(
basepath(`BASEPATH`)
location("my_location")
app("my_app")
name('d_default_udp_2')
# file_path_args("${session}${some_other_arg}")
);
};
log {
source { example-msg-generator(); };
destination(d_default);
destination(d_default_2);
rewrite { set("my_session" value("session")); };
destination(d_default);
};
These are the files generated:
[09:59][ /tmp/my_location/my_app/alltilla-Precision-5530 ] $ ls -algh
total 16K
drwx------ 2 alltilla 4,0K máj 20 09:55 .
drwx------ 3 alltilla 4,0K máj 20 09:35 ..
-rw------- 1 alltilla 2,1K máj 20 09:57 alltilla-Precision-5530_my_app.log
-rw------- 1 alltilla 910 máj 20 09:57 alltilla-Precision-5530_my_appmy_session.log
And the syslog-ng -Fedtv output:
[2021-05-20T09:57:55.503759] Setting value; name='MESSAGE', value='-- Generated message. --', msg='0x557306766d60'
[2021-05-20T09:57:55.503808] Incoming generated message; msg='-- Generated message. --'
[2021-05-20T09:57:55.503840] >>>>>> Source side message processing begin; instance='internal', location='/home/alltilla/Work/install/OSE-3.19/etc/syslog-ng.conf:39:12', msg='0x557306766d60'
[2021-05-20T09:57:55.503864] Setting value; name='HOST_FROM', value='alltilla-Precision-5530', msg='0x557306766d60'
[2021-05-20T09:57:55.503879] Setting value; name='HOST', value='alltilla-Precision-5530', msg='0x557306766d60'
[2021-05-20T09:57:55.503901] Setting value; name='SOURCE', value='#anon-source0', msg='0x557306766d60'
[2021-05-20T09:57:55.503986] Initializing destination file writer; template='/tmp/my_location/my_app/${HOST}/${HOST}_my_app${session}${some_other_arg}.log', filename='/tmp/my_location/my_app/alltilla-Precision-5530/alltilla-Precision-5530_my_app.log'
[2021-05-20T09:57:55.504102] affile_open_file; path='/tmp/my_location/my_app/alltilla-Precision-5530/alltilla-Precision-5530_my_app.log', fd='12'
[2021-05-20T09:57:55.504193] Initializing destination file writer; template='/tmp/my_location/my_app/${HOST}/${HOST}_my_app.log', filename='/tmp/my_location/my_app/alltilla-Precision-5530/alltilla-Precision-5530_my_app.log'
[2021-05-20T09:57:55.504243] affile_open_file; path='/tmp/my_location/my_app/alltilla-Precision-5530/alltilla-Precision-5530_my_app.log', fd='13'
[2021-05-20T09:57:55.504277] >>>>>> rewrite rule evaluation begin; rule='#anon-rewrite0', location='/home/alltilla/Work/install/OSE-3.19/etc/syslog-ng.conf:42:13', msg='0x557306766d60'
[2021-05-20T09:57:55.504295] Message was cloned; original_msg='0x557306766d60', new_msg='0x55730675ec00'
[2021-05-20T09:57:55.504306] Setting value; name='session', value='my_session', msg='0x55730675ec00'
[2021-05-20T09:57:55.504322] <<<<<< rewrite rule evaluation finished; rule='#anon-rewrite0', location='/home/alltilla/Work/install/OSE-3.19/etc/syslog-ng.conf:42:13', msg='0x55730675ec00'
[2021-05-20T09:57:55.504350] Initializing destination file writer; template='/tmp/my_location/my_app/${HOST}/${HOST}_my_app${session}${some_other_arg}.log', filename='/tmp/my_location/my_app/alltilla-Precision-5530/alltilla-Precision-5530_my_appmy_session.log'
[2021-05-20T09:57:55.504395] affile_open_file; path='/tmp/my_location/my_app/alltilla-Precision-5530/alltilla-Precision-5530_my_appmy_session.log', fd='14'
[2021-05-20T09:57:55.504421] <<<<<< Source side message processing finish; instance='internal', location='/home/alltilla/Work/install/OSE-3.19/etc/syslog-ng.conf:39:12', msg='0x557306766d60'
[2021-05-20T09:57:55.504903] Outgoing message; message='May 20 09:57:55 alltilla-Precision-5530 -- Generated message. --\x0a'
[2021-05-20T09:57:55.504984] Outgoing message; message='May 20 09:57:55 alltilla-Precision-5530 -- Generated message. --\x0a'
[2021-05-20T09:57:55.505017] Outgoing message; message='May 20 09:57:55 alltilla-Precision-5530 -- Generated message. --\x0a'
I do not have double quotes appended. Can you send me a similar config, which reproduces your issue?
Thanks!
Attila
________________________________
From: syslog-ng <syslog-ng-bounces at lists.balabit.hu> on behalf of Faine, Mark R. (MSFC-IS40)[NICS] <mark.faine at nasa.gov>
Sent: Wednesday, May 19, 2021 2:30 PM
To: Syslog-ng users' and developers' mailing list <syslog-ng at lists.balabit.hu>
Subject: Re: [syslog-ng] [EXTERNAL] Re: using VARARGS correctly
CAUTION: This email originated from outside of the organization. Do not follow guidance, click links, or open attachments unless you recognize the sender and know the content is safe.
Thanks, I didn't see anything about the "…" in the documentation. I'll take another look. Your suggestion is what I tried originally, however, the problem is that sometimes the argument is not wanted and I was trying to avoid multiple log paths with separate destinations. If I have a destination in which I sometimes do not add the session argument I get "" appended to my path.
So, in circumstances where I do pass a session it works fine and the session number is appended, however, in cases where the session is not passed I get "" appended to the end of the log file name. I was trying to use VARARGS only as a way to work around that problem.
Thanks,
-Mark
Mark Faine
System Administrator
SAIC/NICS
215 Wynn Dr. 5065
Huntsville, AL 35805
256-961-1295 (Desk)
256-617-4861 (Work Cell)
From: syslog-ng <syslog-ng-bounces at lists.balabit.hu> On Behalf Of Attila Szakacs (aszakacs)
Sent: Wednesday, May 19, 2021 01:40
To: Syslog-ng users' and developers' mailing list <syslog-ng at lists.balabit.hu>
Subject: [EXTERNAL] Re: [syslog-ng] using VARARGS correctly
Hi Mark,
VARARGS is used to pass unknown amount of options to the underlying driver through the block.
For example:
block destination ewmm(ip('127.0.0.1') transport(tcp) port(514) ...) {
network("`ip`" transport(`transport`) port(`port`)
template("$(format-ewmm)")
frac-digits(3)
`__VARARGS__`
);
};
"..." and "__VARARGS__" must be used together.
In the example above, any option given to the ewmm destination other than ip(), transport() and port(), is passed to the underlying network() destination.
For your use case, I think a single option would suffice:
block destination default_file(
basepath('/var/log/remote/backup')
location("")
app("")
name("")
file_path_args("")) {
file(
"`basepath`/`location`/`app`/${HOST}/${HOST}_`app``file_path_args`.log"
persist-name(`name`)
create_dirs(yes)
flags("threaded", "no-multi-line")
);
};
destination d_default {
default_file(
basepath(`BASEPATH`)
location("$location")
app("$app")
name('d_default_udp')
file_path_args("${session}${some_other_arg}")
);
};
With the file_path_args() option you can set any number of optionally available macros in the order you like. If a macro is not available, it will resolve to empty string.
Does this take care of your needs?
Cheers,
Attila
________________________________
From: syslog-ng <syslog-ng-bounces at lists.balabit.hu<mailto:syslog-ng-bounces at lists.balabit.hu>> on behalf of Faine, Mark R. (MSFC-IS40)[NICS] <mark.faine at nasa.gov<mailto:mark.faine at nasa.gov>>
Sent: Tuesday, May 18, 2021 8:00 PM
To: Syslog-ng users' and developers' mailing list <syslog-ng at lists.balabit.hu<mailto:syslog-ng at lists.balabit.hu>>
Subject: [syslog-ng] using VARARGS correctly
CAUTION: This email originated from outside of the organization. Do not follow guidance, click links, or open attachments unless you recognize the sender and know the content is safe.
Syslog-ng 3.19
block destination d_default(basepath('/var/log/remote/backup') location("") app("") name("")) {
file("`basepath`/`location`/`app`/${HOST}/${HOST}_`app``__VARARGS__`.log"
persist-name(`name`) create_dirs(yes)
flags("threaded", "no-multi-line"));
};
However, when I call it like so:
d_default(basepath(`BASEPATH`) location("$location") app("$app") name('d_default_udp') "$session");
I get a syntax error when checking with --syntax-only
Error parsing block reference, syntax error, unexpected LL_STRING, expecting ')' in /etc/syslog-ng/conf.d/splunk.conf:
23 categorize_loc();
24 categorize_app();
25 };
26 };
27 destination {
28----> d_default(basepath(`BASEPATH`) location("$location") app("$app") name('d_default_udp') "$session");
28----> ^^^^^^^^^^
From looking at the documentation, it looks like it's a valid way to use it. $session here is just a number from 0-9 that is set from rewriting a user defined macro from the message's sessionid field. It's only going to exist for a certain kind of message.
Thanks,
-Mark
______________________________________________________________________________
Member info: https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.balabit.hu%2Fmailman%2Flistinfo%2Fsyslog-ng&data=04%7C01%7CAttila.Szakacs%40oneidentity.com%7Cd76c6bdac4c64ec8a0fd08d91a26d9ae%7C91c369b51c9e439c989c1867ec606603%7C0%7C1%7C637569576443135501%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=sPCS%2BJNfaqQRl0lODAcJx9vKtYk9W2nCvLhh73%2BS3Rg%3D&reserved=0<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.balabit.hu%2Fmailman%2Flistinfo%2Fsyslog-ng&data=04%7C01%7Cattila.szakacs%40oneidentity.com%7C4bd6b9fe203b4c924fd408d91ac1d6f3%7C91c369b51c9e439c989c1867ec606603%7C0%7C1%7C637570242110426974%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=OJ1VrDSpxZ3cWMzp3TSj%2FD5AbPrDGTu3%2BjYi3mNtug8%3D&reserved=0>
Documentation: https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fsupport%2Fdocumentation%2F%3Fproduct%3Dsyslog-ng&data=04%7C01%7CAttila.Szakacs%40oneidentity.com%7Cd76c6bdac4c64ec8a0fd08d91a26d9ae%7C91c369b51c9e439c989c1867ec606603%7C0%7C1%7C637569576443135501%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=ZntsaJqT%2Bh5EaUoQf8gb%2BF%2BdXy6LUIgkuLQzxKANitY%3D&reserved=0<https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fsupport%2Fdocumentation%2F%3Fproduct%3Dsyslog-ng&data=04%7C01%7Cattila.szakacs%40oneidentity.com%7C4bd6b9fe203b4c924fd408d91ac1d6f3%7C91c369b51c9e439c989c1867ec606603%7C0%7C1%7C637570242110436963%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=9qI6ydL0hL36r5JhPlcQ4OfKUZFCw0WstgsazYfV6kY%3D&reserved=0>
FAQ: https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fwiki%2Fsyslog-ng-faq&data=04%7C01%7CAttila.Szakacs%40oneidentity.com%7Cd76c6bdac4c64ec8a0fd08d91a26d9ae%7C91c369b51c9e439c989c1867ec606603%7C0%7C1%7C637569576443135501%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=EvHK7TAM%2B8QsfR8pZBCYNiJ%2BolHT7b2wo3UMwPoorGM%3D&reserved=0<https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fwiki%2Fsyslog-ng-faq&data=04%7C01%7Cattila.szakacs%40oneidentity.com%7C4bd6b9fe203b4c924fd408d91ac1d6f3%7C91c369b51c9e439c989c1867ec606603%7C0%7C1%7C637570242110436963%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=N9W7jBvRDdmivJPsAiSAh7GhTHM0Ryylj%2Ft7ZEZW%2F3k%3D&reserved=0>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20210520/6c8dc348/attachment-0001.html>
More information about the syslog-ng
mailing list