[syslog-ng] Syslog-NG, log4j, tabs, whitespace, indent

[Balazs Scheidler]

On Thu, 2008-12-11 at 10:37 -0600, John Skopis wrote:
> > -----Original Message-----
> > From: syslog-ng-bounces at lists.balabit.hu [mailto:syslog-ng-
> > bounces at lists.balabit.hu] On Behalf Of Balazs Scheidler
> > Sent: Thursday, December 11, 2008 7:21 AM
> > To: Syslog-ng users' and developers' mailing list
> > Subject: Re: [syslog-ng] Syslog-NG, log4j, tabs, whitespace, indent
> > 
> > On Wed, 2008-12-10 at 16:55 -0600, John Skopis wrote:
> > > Bazsi,
> > >
> > > Strace:
> > > recvfrom(5, "<147>    at
> > org.jgroups.util.TimeScheduler$Loop.run(TimeScheduler.java:141)", 8192,
> > 0, {sa_family=AF_INET, sin_port=htons(41277),
> > sin_addr=inet_addr("1.2.3.4")}, [16]) = 75
> > > write(14, "Dec 10 17:52:04 1.2.3.4 at
> > org.jgroups.util.TimeScheduler$Loop.run(TimeScheduler.java:141)\n", 97)
> > = 97
> > > recvfrom(5, "<147>    at
> > org.jgroups.util.TimeScheduler$Loop.run(TimeScheduler.java:141)", 8192,
> > 0, {sa_family=AF_INET, sin_port=htons(41277),
> > sin_addr=inet_addr("1.2.3.4")}, [16]) = 75
> > > write(14, "Dec 10 17:52:04 1.2.3.4 at
> > org.jgroups.util.TimeScheduler$Loop.run(TimeScheduler.java:141)\n", 97)
> > = 97
> > >
> > > Let me know if you need tcpdump still (I started the cap but would
> > like to anonymize it and/or send it to you off-list).
> > >
> > 
> > Ah, this means that there's no tab at the beginning of the message, but
> > rather it is a space. syslog-ng skips spaces if the syslog header is
> > not
> > complete.
> > 
> > Try enabling setHeader(true) on your SyslogAppender class. This will
> > probably make log4j emit a complete syslog header, in which case
> > syslog-ng will not drop the preceding spaces.
> > 
> > Please let me know if this works for you.
> 
> Well, I tried setting Header=true but it did nothing. Actually, I think that it *is* setting the 
> header for the first line of output, however as a stack trace is a log message spanning multiple 
> lines it only sets the header for the first line.

hmm is log4j using TCP or UDP to send the message? I assume it is TCP.

In TCP mode NL characters are the record terminator characters, so as it
seems log4j generates a multiline message with only one header and
several embedded NL characters. syslog-ng interprets this as a series of
log messages, the first one with a syslog header the rest without one.

Therefore space removal kicks in, the result is what you see.

Skipping the white-space removal is a band-aid solution, the real
problem is that log4j uses multiline messages which syslog-ng does not
support.

The syslog-ng 3.0 branch already supports multi-line messages if the
underlying transport supports those: either UDP or the new
syslog-protocol. Are these real alternatives for you? I guess UDP is out
of question because it may easily lose messages. The new syslog-protocol
is not yet supported by log4j, however I know someone who implemented
initial support for that.

Would that be ok for you to switch over to use syslog-ng 3.0 and
syslog-protocol?

-- 
Bazsi