Re: [syslog-ng] 3.11 Multiple elastic destinations.

14 Sep 2017

      Hi Scot,

On Wed, Sep 13, 2017 at 04:30:26PM -0400, Scot wrote:
...
Has anyone had success/failure using multiple ES destinations in syslog-ng.
I am want to direct traffic to different indexes based on syslog-ng filters
It is possible, but in your case not necessary: use a macro in
the index name! Here's an example:

    destination d_es {
      elasticsearch2(
        ...
        index("${__es_index:-syslog}-${YEAR}.${MONTH}.${DAY}")
        ...
        template("$(format-json ... -x __* ...)")
        ...
      );
    };

The template variable "${__es_index}" is set as usual using filters,
channels and rewrite rules:

    filter f_syslog {
      ...
    };

    filter f_network {
      ...
    };

    rewrite r_syslog {
        set(
            "syslog",
            value("__es_index")
        );
    };

    rewrite r_network {
        set(
            "network",
            value("__es_index")
        );
    };

    log {
      source(...);
      junction {
        channel {
          filter(f_syslog);
          rewrite(r_syslog);
        };
        channel {
          filter(f_network);
          rewrite(r_network);
        };
        channel {
          flags(fallback);
        }
      };
      destination(d_es);
    };

If you prefer having multiple destinations, it also works, but make sure you
also explicitly set the persist-name:

    destination d_es_1 {
      elasticsearch2(
        ...
        persist-name('es_1')
        ...
      );
    };

    destination d_es_2 {
      elasticsearch2(
        ...
        persist-name('es_2')
        ...
      );
    };

Cheers

Re: [syslog-ng] 3.11 Multiple elastic destinations.

Fabien Wernli