Re: [syslog-ng] Help with filtering at destination based on custom macro value match

30 Jan 2021

      The MSG_TAG should not include the : or the space when applying the filter on the destination.

Evan Rempel
University of Victoria
On Jan. 29, 2021, at 7:36 p.m., Akshay Joshi <auj89in@gmail.com<mailto:auj89in@gmail.com>> wrote:
Notice: This message was sent from outside the University of Victoria email system. Please be cautious with links and sensitive information.

My client is sending logs and it has the following config :

template t_global {template("<${PRI}>${LOGHOST} ${MSG_TAG}${MSGHDR}${MSG}\n"); };# Global logging remote destination:
#-----------------------------------
destination d_global_remote {
    tcp("proxy.dc.nuagedemo.net<http://proxy.dc.nuagedemo.net>" port(10514)
        template(t_global)
        tls(peer-verify(required-untrusted)
            ca-dir('/etc/default/bootstrap/keys')
            cert_file('/etc/default/bootstrap/keys/cert.pem')
            key-file('/etc/default/bootstrap/keys/key.pem')
        )
        flags("threaded")
    );
};
source s_nuageDiag {
    file("/home/user/nuage/nuage_diagnostics_daemon.log"
         follow-freq(10) default-facility(local1) default-priority(info) tags("nuageDiag"));
};
rewrite w_nuageDiag { set("nuage-diag: ", value("MSG_TAG") condition(tags("nuageDiag"))); };

On the destination, I have this :

source s_network {
    tcp(
        port(10514)
        max-connections(1000)
        tls(
            peer-verify(required-untrusted)
            key-file("/opt/proxy/config/keys/proxy-Key.pem")
            cert_file("/opt/proxy/config/keys/proxyCert.pem")
            ca-dir("/opt/proxy/config/keys/proxy-CA.pem")
        )
    );
};filter nsg_diag {
    match("nuage-diag: " value("MSG_TAG"));
};
filter f_messages { (level(info..warn) and filter (nsg_diag)); };destination d_logs {
        file(
            "/var/log/syslog-ng/logs.txt"
            owner("root")
            group("root")
            perm(0777)
            );
};
log { source(s_sys); source(s_network); filter(f_messages); destination(d_logs); };

I can write logs locally without the filtering. But with filtering, it does not match "nuage-diag: " macro.
This "MSG_TAG" does not seem to be a standard header but a custom one. I couldn't find many straightforward examples on forums etc.. as well. I am missing a trick or two config-wise for sure.

Any pointers / help will be much appreciated.