I’m quickly finding out that I need to divide my syslog traffic to different ES indexes or I’m missing a better way. I already have filters by subnet but can you have multiple ES destinations defined ? How do you recommend breaking up syslog-ng traffic for ES destinations ? destination d_os { elasticsearch2( index(“openstack_${YEAR}.${MONTH}.${DAY}") type("syslog") # Description: The type of the index. For example, type("test") template("$(format-json --scope rfc3164 --scope nv-pairs --exclude R_DATE --key ISODATE)\n”) …. } destination d_es { elasticsearch2( index(“syslog-ng_${YEAR}.${MONTH}.${DAY}") type("syslog") # Description: The type of the index. For example, type("test") template("$(format-json --scope rfc3164 --scope nv-pairs --exclude R_DATE --key ISODATE)\n”) …. } filter f_192_168_1_0 { netmask(192.168.1.0/24);}; filter f_192_168_4_0 { netmask(192.168.4.0/29);}; filter f_192_168_4_8 { netmask(192.168.4.8/29);}; log { source(s_net); parser(pattern_db); filter(f_192_168_1_0); filter(f_192_168_4_0); destination (d_es); }; log { source(s_net); parser(pattern_db); filter(f_192_168_4_8); destination (d_os); };
Hi Scot! We use a single destination but set the ${__es_index} macro using rewrite rules: elasticsearch2(index("${__es_index:-syslog}-$YEAR.$MONTH.$DAY"));
On Thu, Oct 13, 2016 at 07:48:36AM +0200, Fabien Wernli wrote:
Hi Scot!
We use a single destination but set the ${__es_index} macro using rewrite rules:
elasticsearch2(index("${__es_index:-syslog}-$YEAR.$MONTH.$DAY"));
so for instance: destination d_elastic { elasticsearch2( ... index("${__es_index:-syslog}-$YEAR.$MONTH.$DAY") ); }; log { ... junction { channel { filter(f_foo); rewrite { set("foo", value("__es_index")); }; flags(final); }; channel { filter(f_bar); rewrite { set("bar", value("__es_index")); }; flags(final); }; }; destination(d_elastic); };
participants (2)
-
Fabien Wernli
-
Scot Needy