I think I havecome across a bug in the elasticsearch destination where log lines with UTF8 characters result in a shortend message length attribute which results in a slightly truncated json object being sent to elasticsearch. and here is the source syslog line at our syslog server. This is where the json object is created. 2015-10-02T10:22:47-07:00 local@sandtiger.comp.uvic.ca/sandtiger.comp.uvic.ca mail.warning mimedefang.pl[10880]: t92HMkGW028396: Allowing attachment named OutlookEmoji-<U+1F633>.png, ext=.png, type=image/png, RELAY=mail-bn1on0131.outbound.protection.outlook.com [157.56.110.131], FROM=<Holly.Richardson@Dal.Ca>, TO=<cobyt@uvic.ca> Here is the json object as logged to a file destination on the same host that is rujnning the elasticsearch destination. This is just looging $MESSAGE since the payload is already JSON. {"flare":{"profile":"DCS"},"cfgmgrrole":"INFRA","cfgmgrosFull":"Redhat 5_64","cfgmgros":"unix","cfgmgrmodel":"ESX 5","cfgmgrlocation":"ESX-PROD","cfgmgrenvironment":"Prod","cfgmgrassetType":"Virtual Server","SOURCEHOST":"sandtiger.comp.uvic.ca","SHORTHOST":"sandtiger","PROGRAM":"mimedefang.pl","PRIORITY":"warning","PID":"10880","PATTERNID":"377","MESSAGE":"t92HMkGW028396: Allowing attachment named OutlookEmoji-<U+1F633>.png, ext=.png, type=image/png, RELAY=mail-bn1on0131.outbound.protection.outlook.com [157.56.110.131], FROM=<Holly.Richardson@Dal.Ca>, TO=<cobyt@uvic.ca>","ISODATE":"2015-10-02T10:22:47-07:00","HOST":"sandtiger.comp.uvic.ca","FACILITY":"mail"} This is the same conent that is sent to the elasticsearch destination -- option("message-template", "$MESSAGE\n") and here is the failed message from the elasticsearch server [2015-10-02 10:22:48,630][DEBUG][action.bulk ] [sponge] [flare-2015.10.02.17][2] failed to execute bulk item (index) index {[flare-2015.10.02.17][test][AVApk-CyhIyyHCO_k_bc], source[{"flare":{"profile":"DCS"},"cfgmgrrole":"INFRA","cfgmgrosFull":"Redhat 5_64","cfgmgros":"unix","cfgmgrmodel":"ESX 5","cfgmgrlocation":"ESX-PROD","cfgmgrenvironment":"Prod","cfgmg rassetType":"Virtual Server","SOURCEHOST":"sandtiger.comp.uvic.ca","SHORTHOST":"sandtiger","PROGRAM":"mimedefang.pl","PRIORITY":"warning","PID":"10880","PATTERNID":"377","MESSAGE":"t92HMkGW028396: Allowing attachment named OutlookEmoji-ð<U+009F><U+0098>³.png, ext=.png, type=image/png, RELAY=mail-bn1on0131.outbound.protection.outlook.com [157.56.110.131], FROM=<Holly.Rich ardson@Dal.Ca>, TO=<cobyt@uvic.ca>","ISODATE":"2015-10-02T10:22:47-07:00","HOST":"sandtiger.comp.uvic.ca","FACILITY":"mail]} Note that the source has unicde data as <U+1F633> The elasticsearch destination is sent <U+1F633> but the elastisearch server logs ð<U+009F><U+0098>³ The elasticsearch server also seems to end the message with the text "FACILITY":"mail when it should end with "FACILITY":"mail"} so it is missing two characters. Does anyone want to guess at what is happening? Should I post to the elasticsearch group with the reasoning that the source (syslog-ng) and the destination (elasticsearch) need to be configured with the same unicode settings? Thanks, -- Evan Rempel erempel@uvic.ca Senior Systems Administrator 250.721.7691 Data Centre Services, University Systems, University of Victoria