The standard syslog destinations do not send all name value pairs automatically.
E.g. if you have MSG_TAG on the client, it will only be available there and not on the server, unless the transport you are using to deliver it to another syslog-ng instance does this for you.
You can roll your own template () on the client side which can be parsed on the server, but again this parsing does not happen automatically (but please read on as an alternative). I can see that you are using a custom template (t_global) which indeed sends the value of MSG_TAG) as a prefix to the normal message. When syslog-ng parses this on the server, it will put this value into the PROGRAM name-value pair (and not MSG_TAG that you want to filter on), since that's the spot the client side template inserted this value.
An alternative to all of this is to use the syslog-ng () destination driver, which will use a JSON based format to include all client-side name value pairs.
The server automatically processes this if you used the default-network-drivers() as source on the server (this opens all relevant network ports and enabled automatic parsing of incoming messages).
If you don't want to use the whole of default-network-drivers (), you can stick to a simpler source and then apply parsing of the ewmm() format, using the ewmm-parser().
With that all name-value pairs would automatically make it to the server, where you can trivially continue filtering on any fields that have already been extracted.
Hope this helps,
Bazsi
My client is sending logs and it has the following config :
template t_global {template("<${PRI}>${LOGHOST} ${MSG_TAG}${MSGHDR}${MSG}\n"); };# Global logging remote destination:
#-----------------------------------
destination d_global_remote {
tcp("proxy.dc.nuagedemo.net" port(10514)
template(t_global)
tls(peer-verify(required-untrusted)
ca-dir('/etc/default/bootstrap/keys')
cert_file('/etc/default/bootstrap/keys/cert.pem')
key-file('/etc/default/bootstrap/keys/key.pem')
)
flags("threaded")
);
};
source s_nuageDiag {
file("/home/user/nuage/nuage_diagnostics_daemon.log"
follow-freq(10) default-facility(local1) default-priority(info) tags("nuageDiag"));
};
rewrite w_nuageDiag { set("nuage-diag: ", value("MSG_TAG") condition(tags("nuageDiag"))); };On the destination, I have this :
source s_network {
tcp(
port(10514)
max-connections(1000)
tls(
peer-verify(required-untrusted)
key-file("/opt/proxy/config/keys/proxy-Key.pem")
cert_file("/opt/proxy/config/keys/proxyCert.pem")
ca-dir("/opt/proxy/config/keys/proxy-CA.pem")
)
);
};filter nsg_diag {
match("nuage-diag: " value("MSG_TAG"));
};
filter f_messages { (level(info..warn) and filter (nsg_diag)); };destination d_logs {
file(
"/var/log/syslog-ng/logs.txt"
owner("root")
group("root")
perm(0777)
);
};
log { source(s_sys); source(s_network); filter(f_messages); destination(d_logs); };
I can write logs locally without the filtering. But with filtering, it does not match "nuage-diag: " macro.
This "MSG_TAG" does not seem to be a standard header but a custom one. I couldn't find many straightforward examples on forums etc.. as well. I am missing a trick or two config-wise for sure.
Any pointers / help will be much appreciated.
--
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq