Balazs Scheidler wrote:
On Thu, 2008-07-03 at 11:17 +0200, Balazs Scheidler wrote:
On Wed, 2008-07-02 at 18:31 -0400, Enigma wrote:
I have a host that sends mult-line messages (embedded newlines, not separate syslog messages) from what I can tell syslog-ng strips out all the newlines and replaces them with a space.
Is there anyway to disable or modify (replace them with something else that can be easily s/// in post-processing) this functionality without changing the code and compiling from source?
I have been through the syslog-ng manual and asked Mr. Google and I cannot find anything on this topic.
Newlines and stuff are incompatible with TCP transport. If you are using UDP, it could work, however as you point out syslog-ng removes all NLs from log messages in order not to ruin your logfiles.
With my development snapshot the new syslog-protocol drafts are implemented, that too allows embedded NLs.
This crude patch deletes the part that removes NLs from messages:
diff --git a/src/logmsg.c b/src/logmsg.c index 139fb3a..adb9f2d 100644 --- a/src/logmsg.c +++ b/src/logmsg.c @@ -522,11 +522,6 @@ log_msg_parse(LogMessage *self, gchar *data, gint length, guint flags, regex_t * self->stamp.time.tv_sec = now; }
- for (oldsrc = src, oldleft = left; oldleft >= 0; oldleft--, oldsrc++) - { - if (*oldsrc == '\n' || *oldsrc == '\r') - *oldsrc = ' '; - } g_string_assign_len(&self->msg, src, left); }
I might add something more sophisticated if you or anyone else can help me with finding out a good idea how to handle NLs when they are written to logfiles.
E.g. you have a template like this:
template("$DATE $HOST $MSG\n");
If there's an NL in $MSG it'd probably break a lot of log parsers. If syslog-ng would repeat the syslog header
sorry, sent too early. Would it be enough if syslog-ng would be capable of repeating the $DATE $HOST part for each line produced because of NLs in MSG?
How do you want to use multi-line messages?
Basically I want to preserve the logs as they come from the source. They are delivered to syslog-ng (via UDP so the TCP issue isn't a factor) with embedded newlines and we need them to be forwarded (again via UDP) and written to the log files with those embedded newlines. This is kind of a corner case because we know it will not affect any log parsing. Unfortunately patching the code will not work since this syslog-ng server resides on client prem and will not patch/compile from source on a production machine. The only other option would be to replace the newlines with some other char (with a regex) that we can so a substitute with a newline on the backend before writing to the log file or forwarding the message. I haven't been able to find this functionality either (quite possibly a PEBCAK situation). Is this the rewrite stuff you were talking about in your 'migrate over to PCRE' thread?