Hi, I'm investigating using the EWMM forwarding model. Consider the following setup: Linux hosts collect logs using `system()` send them over using `syslog-ng()` destination to a remote host that collects them using `default-network-drivers()` source. It seems to me that the sudo app parsing is fired up twice: 1. On the sender side because `system()` expands to something including the `sudo-parser()` SCL 2. On the receiver side because `default-network-drivers()` expands to something involving the `app-parser()` This happens also when using `syslog()` source on the sender side, which is why I noticed this behaviour. So my question is, is there something wrong with that model ?
Hi, the syslog-ng() destination should use a special format which transfers all name-value pairs to the server side automatically. On the server side this should be handled specially in which case no sudo parser should run. This is how this is implemented in the server side: Traditional syslog: ``` channel { source { network(transport(tcp) port(`tcp-port`) max-connections(`max-connections`) log-msg-size(`log-msg-size`) flags(no-parse, `flags`) hook-commands(`hook-commands`)); network(transport(udp) port(`udp-port`) log-msg-size(`log-msg-size`) flags(no-parse, `flags`)); }; if { parser { app-parser(topic(syslog-raw) `__VARARGS__`); }; } else { parser { syslog-parser(flags(syslog-protocol)); }; if { parser { ewmm-parser(); }; } elif { parser { app-parser(topic(syslog) `__VARARGS__`); }; } else {}; }; }; ``` on port 514, we check if the message is matched by the any parsers registered on the syslog-raw topic. The sudo parser is not registered here, rather it is registered to "syslog": ``` application sudo[syslog] { filter { program("sudo" type(string)); }; parser { sudo-parser(); }; }; ``` In the else branch, we process the incoming stream with the syslog-parser(). ewmm is encapsulated into the syslog format, so it should be successful here. Once that finishes we apply ewmm-parser() which should again succeed, causing the elif branch not to run. Checking it in practice. I have this config on the client side: ``` @version: 3.28 @include "scl.conf" log { source { system(); }; destination { syslog-ng(server("127.0.0.1") port(2000)); }; }; ``` I invoked sudo, which triggered this message to be sent to the server: $ nc -l -p 2000 <85>1 2020-08-06T12:41:08.000+00:00 bzorp @syslog-ng - - [meta sequenceId="1"] {"PROGRAM":"sudo","PID":"243","MESSAGE":" bazsi : TTY=console ; PWD=/install ; USER=root ; COMMAND=/bin/ls","HOST_FROM":"bzorp","HOST":"bzorp",".unix":{"uid":"0","pid":"243","gid":"1000","cmdline":"sudo ls "},".sudo":{"USER":"root","TTY":"console","SUBJECT":"bazsi","PWD":"/install","COMMAND":"/bin/ls"},".app":{"name":"sudo"},"._TAGS":".app.sudo,.source.#anon-source0"} Please note the ".sudo" key value pairs above. I am then sending this to a default-network-drivers() source: config: ``` @version: 3.28 @include "scl.conf" log { source { default-network-drivers(tcp-port(2514) udp-port(2514) rfc5424-tls-port(6514) rfc5424-tcp-port(2601)); }; destination { file("/install/foobar.log" template("$(format-json --leave-initial-dot --scope all-nv-pairs)\n")); }; }; ``` output in foobar.log: {"SOURCE":"#anon-source0","PROGRAM":"sudo","PID":"243","MESSAGE":" bazsi : TTY=console ; PWD=/install ; USER=root ; COMMAND=/bin/ls","HOST_FROM":"bzorp","HOST":"bzorp",".unix":{"uid":"0","pid":"243","gid":"1000","cmdline":"sudo ls "},".sudo":{"USER":"root","TTY":"console","SUBJECT":"bazsi","PWD":"/install","COMMAND":"/bin/ls"},".app":{"name":"sudo"},".SDATA":{"meta":{"sequenceId":"1"}}} As you can see the ".sudo" top-level key is there, listing sudo related name-value pairs as extracted on the client. I also checked the debug/trace logs on the server and confirmed that only ewmm parsing was done, On Thu, Aug 6, 2020 at 11:55 AM Fabien Wernli <wernli@in2p3.fr> wrote:
Hi,
I'm investigating using the EWMM forwarding model. Consider the following setup: Linux hosts collect logs using `system()` send them over using `syslog-ng()` destination to a remote host that collects them using `default-network-drivers()` source.
It seems to me that the sudo app parsing is fired up twice:
1. On the sender side because `system()` expands to something including the `sudo-parser()` SCL 2. On the receiver side because `default-network-drivers()` expands to something involving the `app-parser()`
This happens also when using `syslog()` source on the sender side, which is why I noticed this behaviour.
So my question is, is there something wrong with that model ?
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
-- Bazsi
Hi Bazsi! On Thu, Aug 06, 2020 at 03:45:19PM +0200, Balazs Scheidler wrote:
As you can see the ".sudo" top-level key is there, listing sudo related name-value pairs as extracted on the client. I also checked the debug/trace logs on the server and confirmed that only ewmm parsing was done,
Thanks for your thorough investigation ! Thus I understand that when using syslog (not -ng) destination, and default-network-drivers, sudo parsing will happen twice.
that's actually true. Once it happens on the client, but the extracted information is NOT conveyed to the server, thus it must do it again. Bazsi On Thu, Aug 6, 2020 at 3:56 PM Fabien Wernli <wernli@in2p3.fr> wrote:
Hi Bazsi!
On Thu, Aug 06, 2020 at 03:45:19PM +0200, Balazs Scheidler wrote:
As you can see the ".sudo" top-level key is there, listing sudo related name-value pairs as extracted on the client. I also checked the debug/trace logs on the server and confirmed that only ewmm parsing was done,
Thanks for your thorough investigation !
Thus I understand that when using syslog (not -ng) destination, and default-network-drivers, sudo parsing will happen twice.
-- Bazsi
On Thu, Aug 06, 2020 at 04:00:12PM +0200, Balazs Scheidler wrote:
that's actually true. Once it happens on the client, but the extracted information is NOT conveyed to the server, thus it must do it again.
I'm relieved : I had a hard time to figure out why the sudo macros were on the remote host, I thought they were sent over, while they were actually generated on both ends :-)
participants (2)
-
Balazs Scheidler
-
Fabien Wernli