Hello,<div><br></div><div>We have a Python process that logs into a file. Included in the file are tracebacks. We would like to send these log entries through syslog-ng. To solve the problem of newlines in tracebacks, our plan was to modify the Python log formatter to replace all newlines with the Unicode character for line separator, U+2028. Syslog-ng then tails the log file and sends the entries to the central logging server through tcp, which is setup to replace the Unicode character with a newline and record it in a file.</div>
<div><br></div><div>The problem with this approach is that the Unicode character does not seem to be persisted in the file. A lot of rewrite rules were tried to replace the Unicode character. These are:</div><div> rewrite r_newlines { subst("\p{Zl}", " test ", value("MESSAGE") type("pcre") flags("utf8" "global")); };</div>
<div><div> rewrite r_newlines { subst("\x{2028}", " test ", value("MESSAGE") type("pcre") flags("utf8" "global")); };</div></div><div><div><div> rewrite r_newlines { subst("\u2028", " test ", value("MESSAGE") flags("global")); };</div>
</div></div><div><br></div><div>but none of them worked. The file source was also set to have UTF-8 encoding, as well as the tcp source (from the view of the central logging server) but these didn't work either. The problem seems to be either the rewrite rule or the Unicode character is not persisted in the log file. We also tried logging to syslog directly from Python (through /dev/log) but that also failed.</div>
<div><br></div><div>Replacing newlines with a string such as " [NEWLINE] " and using syslog-ng to rewrite that to newlines worked, so it seems to be a problem with the Unicode character. This solution is a last resort though since it could be problematic.</div>
<div><br></div><div>Thanks in advance.</div>