[syslog-ng] [Bug 215] syslog-ng v3 - tcp() does not support no-multi-line as docs reference.

[bugzilla at bugzilla.balabit.com]

https://bugzilla.balabit.com/show_bug.cgi?id=215





--- Comment #1 from Balazs Scheidler <bazsi at balabit.hu>  2013-01-02 20:19:46 ---
(In reply to comment #0)
> So after doing some test cases, I've noticed that on no-multi-line is not stripping newlines on v3.3.7. I've not tested previous versions yet.
> 
> Here is my syslog-ng config:
> 
> source s_net_tcp {
>         tcp(ip(0.0.0.0) port(514) flags(no-multi-line));
> };
> 
> destination all { file("/var/log/all"); };
> 
> log {
>         source(s_net_tcp);
>         destination(all);
>         flags(final);
> };
> 
> I've also attempted to do this with no-parse flags passed in to no avail. 

no-multi-line is a transport independent flag and it is applied to the message as 
decoded by the underlying transport protocol. The udp transport merely encloses 
messages into their own UDP frame, embedded newlines are thus possible, and 
no-multi-line removes them by replacing them with a space.

> When using the udp() source, it works just fine.
> 
> 
> My test case:
> 
> # cat blah
> line1
> line2
> line3
> 
> # cat blah | nc -r -n myhost 514
> 
> And on "myhost"
> Jan  2 12:25:16 192.2.0.1 line1
> Jan  2 12:25:16 192.2.0.1 line2
> Jan  2 12:25:16 192.2.0.1 line3

this is the expected behaviour. If you want embedded newlines in messages you need to use a
transport that supports those. Such is rfc5424 style messages used by the syslog driver.

What do you want to accomplish exactly?

> 
> So I assume this has something to do with it:
> 
> https://bugzilla.balabit.com/show_bug.cgi?id=74
> 
> Particularly about newlines being message terminators over tcp(), 

exactly.

> however, why do the docs even say tcp() supports the no-multi-line flag?

yup, it supports it, but it fails to find an actual newline there. but the same 
functionality is used when regardless of the transport.

> 
> Maybe there is a way to adjust the terminator if this is the case? 

it's not possible right now, I'm afraid. 

> I must admit, I was curious what syslog-ng would do if the message size exceeded the mtu of
> the network. With UDP, I assume the end of the packet is the end of the message so any messages large enough to span two udp packets would actually be two log
> entries. 

This is not true. IP level fragmentation happens below the UDP level, and UDP genuinely 
supports 64k messages. However, if fragmentation becomes necessary, you need to 
receive each and every frame in order to reconstruct the whole message. Therefore it 
is not recommended to send very large messages over udp. RFC3164 even limits the 
size in 1024 octets, this is not enforced by syslog-ng though.

> I assumed it would also be the case with TCP, even if it is stream based.

With TCP, the underlying packet boundaries are not available to the application, thus it
cannot be used to infer the message framing solely from the tcp.

> 
> So the potential problems I see are:
> 1. tcp() doesn't really support no-multi-line (bug) or

it does, but it does nothing.

> 2. the docs should be updated to not say it supports no-multi-line (bug) or

well, this should be clarified there, I agree.

> 3. I've overlooked the doc references or
> 4. I'm missing some option (terminator?) in order to get tcp() to work with no-multi-line. (env)
> 

Algernon has already experimented with support for multiple lines over stream 
like transport, but it's not yet integrated. It used MIME-style line continuations, 
e.g. the first character of subsequent must be white space. It's used on the 
modern /dev/kmsg interface of the linux kernel, but it'll probably work for tcp too.

What I couldn't decipher exactly, what you'd like to accomplish. Can you elaborate on that?
Thanks.


-- 
Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching all bug changes.