[syslog-ng] UTF-8 handling

[Robin Blanchard]

Hi,

I've got syslog-ng-3.7.3-3.el7.centos.x86_64 feeding logstash-2.2.4-1.noarch.

I've had flags(sanitize-utf8) enabled on the relevant syslog-ng listener, yet LS continues to complain with ':expected_charset=>"UTF-8"', to the point of LS ultimately choking.  The underlying messages are indeed garbage, eg:

$ pcregrep -n "[\x80-\xFF]" garbage.txt |tail -1
299:\\xc2\u0003\u0003\\xe0\u0001|\\x9d\\x9bX\\xa3wt}\\xe4\u001e\\xd6g\u0013j\\x9c!j\\xd9#*\\x95#\\xf8������

Should I instead be employing flags(validate-utf8) ?

Ultimately, I'd like for syslog-ng to either re-encode messages into proper UTF8, or -- if not possible -- just squelch/drop them….


Thanks in advance




--------------------------------------------------------------------------------------------------------------------------
This email has been sent to you on behalf of Nephila Advisors LLC (“Advisors”). Advisors provides consultancy services to Nephila Capital Ltd. (“Capital”), an investment advisor managed and carrying on business in Bermuda. Advisors and its employees do not act as agents for Capital or the funds it advises and do not have the authority to bind Capital or such funds to any transaction or agreement.

The information in this e-mail, and any attachment therein, is confidential and for use by the addressee only. Any use, disclosure, reproduction, modification or distribution of the contents of this e-mail, or any part thereof, other than by the intended recipient, is strictly prohibited. If you are not the intended recipient, please return the e-mail to the sender and delete it from your computer. This email is for information purposes only, nothing contained herein constitutes an offer to sell or buy securities, as such an offer may only be made from a properly authorized offering document. Although Nephila attempts to sweep e-mail and attachments for viruses, it does not guarantee that either are virus-free and accepts no liability for any damage sustained as a result of viruses.
--------------------------------------------------------------------------------------------------------------------------