[syslog-ng] How to send tracebacks through syslog-ng?

[Luis Pugoy]

Hello,

We have a Python process that logs into a file. Included in the file are
tracebacks. We would like to send these log entries through syslog-ng. To
solve the problem of newlines in tracebacks, our plan was to modify the
Python log formatter to replace all newlines with the Unicode character for
line separator, U+2028. Syslog-ng then tails the log file and sends the
entries to the central logging server through tcp, which is setup to replace
the Unicode character with a newline and record it in a file.

The problem with this approach is that the Unicode character does not seem
to be persisted in the file. A lot of rewrite rules were tried to replace
the Unicode character. These are:
  rewrite r_newlines { subst("\p{Zl}", " test ", value("MESSAGE")
type("pcre") flags("utf8" "global")); };
  rewrite r_newlines { subst("\x{2028}", " test ", value("MESSAGE")
type("pcre") flags("utf8" "global")); };
  rewrite r_newlines { subst("\u2028", " test ", value("MESSAGE")
flags("global")); };

but none of them worked. The file source was also set to have UTF-8
encoding, as well as the tcp source (from the view of the central logging
server) but these didn't work either. The problem seems to be either the
rewrite rule or the Unicode character is not persisted in the log file. We
also tried logging to syslog directly from Python (through /dev/log) but
that also failed.

Replacing newlines with a string such as " [NEWLINE] " and using syslog-ng
to rewrite that to newlines worked, so it seems to be a problem with the
Unicode character. This solution is a last resort though since it could be
problematic.

Thanks in advance.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.balabit.hu/pipermail/syslog-ng/attachments/20110923/5c0ae38a/attachment.htm