[syslog-ng] Suppress UTF-8 BOM in incoming network messages

23 Sep 2024

      I'm trying to use Syslog-NG to receive logs from our anti-virus web console
(ESET Protect).  I've got the logs coming in successfully, but they seem to
be adding a UTF-8 BOM as part of the message:

[2024-09-23T19:03:36.709950] Incoming log entry; input='<15>1
2024-09-23T19:03:36.455Z b35f2351-8f78-4414-b964-415939a7b0fb ERAServer 43
- - <feff>{"event_type" : "Audit_Event","ipv4" : "172.17.0.11","ipv6" :
"","hostname" : "b35f2351-8f78-4414-b964-415939a7b0fb","source_uuid" :
"b35f2351-8f78-4414-b964-415939a7b0fb","os_name" : "","occured" :
"23-Sep-2024 19:03:36","group_name" : "","group_description" :
"","severity" : "Information","domain" : "Mapped account","action" :
"Logout","target" : "35396c68-5017-4ea9-b268-d209f8040de9","detail" :
"Logging out mapped account \'Jeremy Utley\'.","user" : "Jeremy
Utley","result" : "Success"}', msg='0x7f6020022150', rcptid='0'

Because I only care about the JSON data being provided, I'm setting the
"flags(no-parse)" option in my network source, and a rewrite filter to
remove the initial data as follows:

source s_network {
        network(
                port(6514)
                transport("tls")
                flags(no-parse)
                tls(
                        key-file("/etc/syslog-ng/certs/syslog.key")
                        cert-file("/etc/syslog-ng/certs/syslog.crt")
                        peer-verify(optional-untrusted)
                )
        );
};

rewrite r_trimjson {
  subst("^(.*)- - ", "", value("MESSAGE"));
};

destination d_json {
    file("/var/log/json_data.log" template("${MESSAGE}\n"));
};

But I can't seem to eliminate the BOM from the output, and that is messing
with JSON parsing, since that is added to the beginning of every line (but
the first) in my json_data.log file.  I tried adding "\xfe\xff" to the
rewrite in an attempt to eliminate it, but that didn't work - the rewrite
does not even match at that point.

Anyone have any suggestions on how I can deal with this?

Thanks!

Jeremy Utley