Syslog-ng setup for both RFC3164 and RFC5124
Hi all, Is there a way to configure syslog-ng to process properly both RFC3164 and RFC5124 on the same listening port ? The scenario is a bunch of devices sending traffic to one syslog server port (both udp + tcp) with the senders typically not knowing what protocol they are sending. We are running syslog-ng 3.13 with this setup: source s_syslog { udp(ip(0.0.0.0) port(514)) ; tcp(ip(0.0.0.0) port(514)); } If needed we could upgrade syslog-ng to 3.19.1 but having checked the doc for 3.19, it seems that the solution would be to create 2 source entries, 1 for RFC3164 with network() and 1 for RFC5124 with syslog(). Neverthless, these 2 sources would have to listen on *different* ports and that is the problem for us. Note that we also have an identical issue with cisco traffic, since it's not RFC compliant, syslog-ng adds automatically a header with timestamp and hostname. Thank you.
Yes, there is a flag "syslog-protocol" that will allow this. The rfc5124 only applies to TCP, so the flag is only on the tcp source. Our configuration for the source is source s_network_udp { network(localip(1.2.3.4) port(514) so_rcvbuf(33554432) log_fetch_limit(20000) log_iw_size(1000000) transport("udp") tags("unix_network") flags(no-multi-line) ); }; source s_network_tcp { network(localip(1.2.3.4) port(514) max_connections(5000) log_fetch_limit(20000) log_iw_size(1000000) transport("tcp") flags(no-multi-line,syslog-protocol) tags("unix_network") ); }; Hope that helps. Evan. On 2/23/19 5:07 PM, Carlan Philippe wrote:
Hi all,
Is there a way to configure syslog-ng to process properly both RFC3164 and RFC5124 on the same listening port ?
The scenario is a bunch of devices sending traffic to one syslog server port (both udp + tcp) with the senders typically not knowing what protocol they are sending.
We are running syslog-ng 3.13 with this setup:
source s_syslog { udp(ip(0.0.0.0) port(514)) ; tcp(ip(0.0.0.0) port(514)); }
If needed we could upgrade syslog-ng to 3.19.1 but having checked the doc for 3.19, it seems that the solution would be to create 2 source entries, 1 for RFC3164 with network() and 1 for RFC5124 with syslog(). Neverthless, these 2 sources would have to listen on *different* ports and that is the problem for us.
Note that we also have an identical issue with cisco traffic, since it's not RFC compliant, syslog-ng adds automatically a header with timestamp and hostname.
Also, check out the default-network-drivers() logic. You might not need the whole stuff, but it's good to get ideas from. It opens all relevant ports and processes both rfc3164/rfc5424 and cisco traffic properly. It is here: https://github.com/balabit/syslog-ng/blob/master/scl/default-network-drivers... default-network-drivers() can be extended using parsers automatically, so that you only have a static configuration, and application adapters that get deployed later on. Bazsi On Sun, Feb 24, 2019 at 4:30 PM Evan Rempel <erempel@uvic.ca> wrote:
Yes, there is a flag "syslog-protocol" that will allow this. The rfc5124 only applies to TCP, so the flag is only on the tcp source.
Our configuration for the source is
source s_network_udp { network(localip(1.2.3.4) port(514) so_rcvbuf(33554432) log_fetch_limit(20000) log_iw_size(1000000) transport("udp") tags("unix_network") flags(no-multi-line) ); }; source s_network_tcp { network(localip(1.2.3.4) port(514) max_connections(5000) log_fetch_limit(20000) log_iw_size(1000000) transport("tcp") flags(no-multi-line,syslog-protocol) tags("unix_network") ); };
Hope that helps.
Evan.
On 2/23/19 5:07 PM, Carlan Philippe wrote:
Hi all,
Is there a way to configure syslog-ng to process properly both RFC3164 and RFC5124 on the same listening port ?
The scenario is a bunch of devices sending traffic to one syslog server port (both udp + tcp) with the senders typically not knowing what protocol they are sending.
We are running syslog-ng 3.13 with this setup:
source s_syslog { udp(ip(0.0.0.0) port(514)) ; tcp(ip(0.0.0.0) port(514)); }
If needed we could upgrade syslog-ng to 3.19.1 but having checked the doc for 3.19, it seems that the solution would be to create 2 source entries, 1 for RFC3164 with network() and 1 for RFC5124 with syslog(). Neverthless, these 2 sources would have to listen on *different* ports and that is the problem for us.
Note that we also have an identical issue with cisco traffic, since it's not RFC compliant, syslog-ng adds automatically a header with timestamp and hostname.
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
Hi Carlan, If it helps, here’s what some of our users use for receiving on a local host and forwarding out to a docker container on that same host. Also, I’m curious about “identical issue with cisco traffic, since it's not RFC compliant” – why do you say that? (well, I agree that some of their stuff isn’t, but routers, switches and firewalls are as far as I know) # Local forwarding to LogZilla NEO containers # Generated on Thu Jan 3 17:49:27 UTC 2019 options { chain_hostnames(off); flush_lines(10000); threaded(yes); use_dns(yes); # This should be set to no in high scale environments use_fqdn(no); keep_hostname(yes); dns-cache-size(2000); dns-cache-expire(87600); use-dns(persist_only); dns-cache-hosts(/etc/hosts); owner("root"); group("root"); perm(0640); stats_freq(0); time_reopen(5); }; source s_local { system(); internal(); }; source s_rfc3164 { network( transport("tcp") port(514) log-iw-size(20000) ); network( transport("udp") so_rcvbuf(1048576) flags("no-multi-line") port(514) ); }; source s_rfc5424 { network( transport("tcp") flags(syslog-protocol) port(601) ); }; destination d_rfc3164 { tcp("localhost" port(32514)); }; destination d_rfc5424 { tcp("localhost" port(32601)); }; log { source(s_rfc3164); destination(d_rfc3164); }; log { source(s_rfc5424); destination(d_rfc5424); }; From: syslog-ng <syslog-ng-bounces@lists.balabit.hu> on behalf of Carlan Philippe <philrmls@yahoo.fr> Reply-To: Syslog-ng users' and developers' mailing list <syslog-ng@lists.balabit.hu> Date: Saturday, February 23, 2019 at 8:09 PM To: Syslog-ng users' and developers' mailing list <syslog-ng@lists.balabit.hu> Subject: [syslog-ng] Syslog-ng setup for both RFC3164 and RFC5124 Hi all, Is there a way to configure syslog-ng to process properly both RFC3164 and RFC5124 on the same listening port ? The scenario is a bunch of devices sending traffic to one syslog server port (both udp + tcp) with the senders typically not knowing what protocol they are sending. We are running syslog-ng 3.13 with this setup: source s_syslog { udp(ip(0.0.0.0) port(514)) ; tcp(ip(0.0.0.0) port(514)); } If needed we could upgrade syslog-ng to 3.19.1 but having checked the doc for 3.19, it seems that the solution would be to create 2 source entries, 1 for RFC3164 with network() and 1 for RFC5124 with syslog(). Neverthless, these 2 sources would have to listen on *different* ports and that is the problem for us. Note that we also have an identical issue with cisco traffic, since it's not RFC compliant, syslog-ng adds automatically a header with timestamp and hostname. Thank you.
participants (4)
-
Carlan Philippe
-
Clayton Dukes
-
Evan Rempel
-
Scheidler, Balázs