[syslog-ng] Multi-line Logs

[Balazs Scheidler]

On Thu, 2008-07-03 at 11:17 +0200, Balazs Scheidler wrote:
> On Wed, 2008-07-02 at 18:31 -0400, Enigma wrote:
> > I have a host that sends mult-line messages (embedded newlines, not 
> > separate syslog messages) from what I can tell syslog-ng strips out all 
> > the newlines and replaces them with a space.
> > 
> > Is there anyway to disable or modify (replace them with something else 
> > that can be easily s/// in post-processing) this functionality without 
> > changing the code and compiling from source?
> > 
> > I have been through the syslog-ng manual and asked Mr. Google and I 
> > cannot find anything on this topic.
> 
> Newlines and stuff are incompatible with TCP transport. If you are using
> UDP, it could work, however as you point out syslog-ng removes all NLs
> from log messages in order not to ruin your logfiles.
> 
> With my development snapshot the new syslog-protocol drafts are
> implemented, that too allows embedded NLs.
> 
> This crude patch deletes the part that removes NLs from messages:
> 
> diff --git a/src/logmsg.c b/src/logmsg.c
> index 139fb3a..adb9f2d 100644
> --- a/src/logmsg.c
> +++ b/src/logmsg.c
> @@ -522,11 +522,6 @@ log_msg_parse(LogMessage *self, gchar *data, gint length, guint flags, regex_t *
>        self->stamp.time.tv_sec = now;
>      }
>  
> -  for (oldsrc = src, oldleft = left; oldleft >= 0; oldleft--, oldsrc++)
> -    {
> -      if (*oldsrc == '\n' || *oldsrc == '\r')
> -       *oldsrc = ' ';
> -    }
>    g_string_assign_len(&self->msg, src, left);
>  }
>  
> 
> I might add something more sophisticated if you or anyone else can help me with finding out a 
> good idea how to handle NLs when they are written to logfiles.
> 
> E.g. you have a template like this:
> 
> template("$DATE $HOST $MSG\n");
> 
> If there's an NL in $MSG it'd probably break a lot of log parsers. If syslog-ng would repeat the syslog header
> 

sorry, sent too early. Would it be enough if syslog-ng would be capable
of repeating the $DATE $HOST part for each line produced because of NLs
in MSG?

How do you want to use multi-line messages?

-- 
Bazsi