RFC5424 sending - dest=unix-stream. Framing?
I have syslog-ng pumping data into a unix-stream destination socketfilething. Data going into this domain socket has lots of adventures, and then emerges as a source unix-stream on another host. Both have flags(syslog-protocol), but the data gets framing errors at the receiver. I think it's become a bytestream (instead of a message stream) enroute, and the recv() is relying on only getting one message at a time, because it can't know the message length any other way, as unix-stream gets messages without framing(?). Is there a way to salvage this? (I noticed TLS might add framing, but since the tunnel is already TLS ...) -- Declan White
If the new lines at the end are ok, and the messages dont contain newlines themselves, you should be fine. If both destination and source is unix-STREAM(). Don't you hapoen to use unix-dgram() somewhere? That would take the datagram fragments as individual messages. Can you show me the framing errors you mention? On Feb 1, 2018 22:33, "Declan White" <declanw@is.bbc.co.uk> wrote: I have syslog-ng pumping data into a unix-stream destination socketfilething. Data going into this domain socket has lots of adventures, and then emerges as a source unix-stream on another host. Both have flags(syslog-protocol), but the data gets framing errors at the receiver. I think it's become a bytestream (instead of a message stream) enroute, and the recv() is relying on only getting one message at a time, because it can't know the message length any other way, as unix-stream gets messages without framing(?). Is there a way to salvage this? (I noticed TLS might add framing, but since the tunnel is already TLS ...) -- Declan White ____________________________________________________________ __________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/? product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
Yeah, disregard me. Sorry. Looks like the tunnel I'm using is unclean.. On Fri, Feb 02, 2018 at 06:48:09AM +0100, Scheidler, Bal?zs wrote:
If the new lines at the end are ok, and the messages dont contain newlines themselves, you should be fine. If both destination and source is unix-STREAM(). Don't you hapoen to use unix-dgram() somewhere? That would take the datagram fragments as individual messages.
Can you show me the framing errors you mention?
On Feb 1, 2018 22:33, "Declan White" <declanw@is.bbc.co.uk> wrote:
I have syslog-ng pumping data into a unix-stream destination socketfilething.
Data going into this domain socket has lots of adventures, and then emerges as a source unix-stream on another host. Both have flags(syslog-protocol), but the data gets framing errors at the receiver.
I think it's become a bytestream (instead of a message stream) enroute, and the recv() is relying on only getting one message at a time, because it can't know the message length any other way, as unix-stream gets messages without framing(?).
Is there a way to salvage this? (I noticed TLS might add framing, but since the tunnel is already TLS ...)
-- Declan White
On Fri, Feb 02, 2018 at 07:01:55PM +0000, Declan White wrote:
Yeah, disregard me. Sorry. Looks like the tunnel I'm using is unclean..
Well, I fixed the tunnel, but it turned out to be a red herring. It seems I've got two different problems in syslog-ng (on Solaris) : 1) If I read from a unix-stream->unix-stream source, recvmsg() reliably EFAULT's the second read of a new connection. Unless I attach a library call tracer.. and then it won't. Heisenbug... 10109/11: write(2, 0x00059898, 58) = 58 10109/11: S e t t i n g v a l u e ; m s g = ' 4 7 b 1 0 ' , n a m e 10109/11: = ' S O U R C E ' , v a l u e = ' i n _ t e s t ' 10109/11: write(2, "\n", 1) = 1 10109/11: recvmsg(15, 0xFE66B4B4, 0) Err#14 EFAULT I think my build env may have eaten something bad. I'm recompiling all the dependant libs with GCC, and dumping libwrap, to see if that changes anything. Is any of this compiler noise relevant? : ../source/syslog-ng-3.12.1/modules/afsocket/socket-options-inet.c:47:8: warning: unused variable 'on' [-Wunused-variable] gint on = 1; ../source/syslog-ng-3.12.1/modules/afsocket/transport-unix-socket.c:52:35: warning: format '%d' expects argument of type 'int', but argument 4 has type 'pid_t {aka long int}' [-Wformat=] g_snprintf(buf, buflen, "/proc/%d/%s", pid, proc_file); ../source/syslog-ng-3.12.1/modules/afsocket/transport-unix-socket.c:138:1: warning: '_add_nv_pair_proc_readlink' defined but not used [-Wunused-function] ../source/syslog-ng-3.12.1/modules/afsocket/transport-unix-socket.c:118:1: warning: '_add_nv_pair_proc_read_argv' defined but not used [-Wunused-function ../source/syslog-ng-3.12.1/modules/afsocket/transport-unix-socket.c:104:1: warning: '_add_nv_pair_proc_read_unless_unset' defined but not used [-Wunused-function] ../source/syslog-ng-3.12.1/modules/afsocket/transport-unix-socket.c:38:1: warning: '_add_nv_pair_int' defined but not used [-Wunused-function] ../source/syslog-ng-3.12.1/modules/afstreams/afstreams.c: In function 'log_transport_streams_new': ../source/syslog-ng-3.12.1/modules/afstreams/afstreams.c:97:14: warning: assignment from incompatible pointer type [-Wincompatible-pointer-types] self->read = log_transport_streams_read; ../source/syslog-ng-3.12.1/modules/afstreams/afstreams.c: In function 'afstreams_init_door': ../source/syslog-ng-3.12.1/modules/afstreams/afstreams.c:133:11: warning: implicit declaration of function 'close'; did you mean 'pclose'? [-Wimplicit-function-declaration] close(fd); 2) If I avoid that unix-stream source and socat the client from a unix domain socket destination into a syslog port source, syslog-ng now has a mismatch of a source sending no framing, and a destination expecting it. syslog() write(5, 0x000721E8, 60) = 60 5 7 < 1 3 > 1 2 0 1 8 - 0 2 - 0 3 T 0 3 : 1 1 : 2 0 + 0 0 : 0 0 o p s t e s t 6 a - - - - t e s t 0 8\n unix-stream() read(6, 0x00072B00, 8192) = 57 < 1 3 > 1 2 0 1 8 - 0 2 - 0 3 T 0 2 : 0 5 : 0 5 + 0 0 : 0 0 o p s t e s t 6 a - - - - t e s t 0 7\n I take it that if I did something evil like faking up a length header with length() in a template() I might get unix-stream -> syslog working?
On Fri, Feb 02, 2018 at 06:48:09AM +0100, Scheidler, Bal?zs wrote:
If the new lines at the end are ok, and the messages dont contain newlines themselves, you should be fine. If both destination and source is unix-STREAM(). Don't you hapoen to use unix-dgram() somewhere? That would take the datagram fragments as individual messages.
Can you show me the framing errors you mention?
On Feb 1, 2018 22:33, "Declan White" <declanw@is.bbc.co.uk> wrote:
I have syslog-ng pumping data into a unix-stream destination socketfilething.
Data going into this domain socket has lots of adventures, and then emerges as a source unix-stream on another host. Both have flags(syslog-protocol), but the data gets framing errors at the receiver.
I think it's become a bytestream (instead of a message stream) enroute, and the recv() is relying on only getting one message at a time, because it can't know the message length any other way, as unix-stream gets messages without framing(?).
Is there a way to salvage this? (I noticed TLS might add framing, but since the tunnel is already TLS ...)
You could use tcp(flags(syslog-protocol)) to send rfc5424 without framing. On Feb 3, 2018 06:16, "Declan White" <declanw@is.bbc.co.uk> wrote:
On Fri, Feb 02, 2018 at 07:01:55PM +0000, Declan White wrote:
Yeah, disregard me. Sorry. Looks like the tunnel I'm using is unclean..
Well, I fixed the tunnel, but it turned out to be a red herring. It seems I've got two different problems in syslog-ng (on Solaris) :
1) If I read from a unix-stream->unix-stream source, recvmsg() reliably EFAULT's the second read of a new connection. Unless I attach a library call tracer.. and then it won't. Heisenbug... 10109/11: write(2, 0x00059898, 58) = 58 10109/11: S e t t i n g v a l u e ; m s g = ' 4 7 b 1 0 ' , n a m e 10109/11: = ' S O U R C E ' , v a l u e = ' i n _ t e s t ' 10109/11: write(2, "\n", 1) = 1 10109/11: recvmsg(15, 0xFE66B4B4, 0) Err#14 EFAULT
I think my build env may have eaten something bad. I'm recompiling all the dependant libs with GCC, and dumping libwrap, to see if that changes anything.
Is any of this compiler noise relevant? : ../source/syslog-ng-3.12.1/modules/afsocket/socket-options-inet.c:47:8: warning: unused variable 'on' [-Wunused-variable] gint on = 1; ../source/syslog-ng-3.12.1/modules/afsocket/transport-unix-socket.c:52:35: warning: format '%d' expects argument of type 'int', but argument 4 has type 'pid_t {aka long int}' [-Wformat=] g_snprintf(buf, buflen, "/proc/%d/%s", pid, proc_file); ../source/syslog-ng-3.12.1/modules/afsocket/transport-unix-socket.c:138:1: warning: '_add_nv_pair_proc_readlink' defined but not used [-Wunused-function] ../source/syslog-ng-3.12.1/modules/afsocket/transport-unix-socket.c:118:1: warning: '_add_nv_pair_proc_read_argv' defined but not used [-Wunused-function ../source/syslog-ng-3.12.1/modules/afsocket/transport-unix-socket.c:104:1: warning: '_add_nv_pair_proc_read_unless_unset' defined but not used [-Wunused-function] ../source/syslog-ng-3.12.1/modules/afsocket/transport-unix-socket.c:38:1: warning: '_add_nv_pair_int' defined but not used [-Wunused-function] ../source/syslog-ng-3.12.1/modules/afstreams/afstreams.c: In function 'log_transport_streams_new': ../source/syslog-ng-3.12.1/modules/afstreams/afstreams.c:97:14: warning: assignment from incompatible pointer type [-Wincompatible-pointer-types] self->read = log_transport_streams_read; ../source/syslog-ng-3.12.1/modules/afstreams/afstreams.c: In function 'afstreams_init_door': ../source/syslog-ng-3.12.1/modules/afstreams/afstreams.c:133:11: warning: implicit declaration of function 'close'; did you mean 'pclose'? [-Wimplicit-function-declaration] close(fd);
2) If I avoid that unix-stream source and socat the client from a unix domain socket destination into a syslog port source, syslog-ng now has a mismatch of a source sending no framing, and a destination expecting it.
syslog() write(5, 0x000721E8, 60) = 60 5 7 < 1 3 > 1 2 0 1 8 - 0 2 - 0 3 T 0 3 : 1 1 : 2 0 + 0 0 : 0 0 o p s t e s t 6 a - - - - t e s t 0 8\n
unix-stream() read(6, 0x00072B00, 8192) = 57 < 1 3 > 1 2 0 1 8 - 0 2 - 0 3 T 0 2 : 0 5 : 0 5 + 0 0 : 0 0 o p s t e s t 6 a - - - - t e s t 0 7\n
I take it that if I did something evil like faking up a length header with length() in a template() I might get unix-stream -> syslog working?
On Fri, Feb 02, 2018 at 06:48:09AM +0100, Scheidler, Bal?zs wrote:
If the new lines at the end are ok, and the messages dont contain newlines themselves, you should be fine. If both destination and source is unix-STREAM(). Don't you hapoen to use unix-dgram() somewhere? That would take the datagram fragments as individual messages.
Can you show me the framing errors you mention?
On Feb 1, 2018 22:33, "Declan White" <declanw@is.bbc.co.uk> wrote:
I have syslog-ng pumping data into a unix-stream destination socketfilething.
Data going into this domain socket has lots of adventures, and then emerges as a source unix-stream on another host. Both have flags(syslog-protocol), but the data gets framing errors at the receiver.
I think it's become a bytestream (instead of a message stream) enroute, and the recv() is relying on only getting one message at a time, because it can't know the message length any other way, as unix-stream gets messages without framing(?).
Is there a way to salvage this? (I noticed TLS might add framing, but since the tunnel is already TLS ...)
__________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/? product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
On Sat, Feb 03, 2018 at 03:23:06PM +0100, Scheidler, Bal?zs wrote:
You could use tcp(flags(syslog-protocol)) to send rfc5424 without framing.
I was already sending without framing but, now you've mentioned it, I've gone over the differences between network(flags(syslog-protocol)) and syslog() again. I'd thought they were synonymous/equivalent - I was mistaken! EUREKA! Switching the receiver from syslog() to network(flags(syslog-protocol)) gets everything working. Thanks! :) - Declan PS: the recompile didn't kill the Solaris heisenbug, but I'll happily take this escape route instead.
On Feb 3, 2018 06:16, "Declan White" <declanw@is.bbc.co.uk> wrote:
On Fri, Feb 02, 2018 at 07:01:55PM +0000, Declan White wrote:
Yeah, disregard me. Sorry. Looks like the tunnel I'm using is unclean..
Well, I fixed the tunnel, but it turned out to be a red herring. It seems I've got two different problems in syslog-ng (on Solaris) :
1) If I read from a unix-stream->unix-stream source, recvmsg() reliably EFAULT's the second read of a new connection. Unless I attach a library call tracer.. and then it won't. Heisenbug... 10109/11: write(2, 0x00059898, 58) = 58 10109/11: S e t t i n g v a l u e ; m s g = ' 4 7 b 1 0 ' , n a m e 10109/11: = ' S O U R C E ' , v a l u e = ' i n _ t e s t ' 10109/11: write(2, "\n", 1) = 1 10109/11: recvmsg(15, 0xFE66B4B4, 0) Err#14 EFAULT
I think my build env may have eaten something bad. I'm recompiling all the dependant libs with GCC, and dumping libwrap, to see if that changes anything.
Is any of this compiler noise relevant? : ../source/syslog-ng-3.12.1/modules/afsocket/socket-options-inet.c:47:8: warning: unused variable 'on' [-Wunused-variable] gint on = 1; ../source/syslog-ng-3.12.1/modules/afsocket/transport-unix-socket.c:52:35: warning: format '%d' expects argument of type 'int', but argument 4 has type 'pid_t {aka long int}' [-Wformat=] g_snprintf(buf, buflen, "/proc/%d/%s", pid, proc_file); ../source/syslog-ng-3.12.1/modules/afsocket/transport-unix-socket.c:138:1: warning: '_add_nv_pair_proc_readlink' defined but not used [-Wunused-function] ../source/syslog-ng-3.12.1/modules/afsocket/transport-unix-socket.c:118:1: warning: '_add_nv_pair_proc_read_argv' defined but not used [-Wunused-function ../source/syslog-ng-3.12.1/modules/afsocket/transport-unix-socket.c:104:1: warning: '_add_nv_pair_proc_read_unless_unset' defined but not used [-Wunused-function] ../source/syslog-ng-3.12.1/modules/afsocket/transport-unix-socket.c:38:1: warning: '_add_nv_pair_int' defined but not used [-Wunused-function] ../source/syslog-ng-3.12.1/modules/afstreams/afstreams.c: In function 'log_transport_streams_new': ../source/syslog-ng-3.12.1/modules/afstreams/afstreams.c:97:14: warning: assignment from incompatible pointer type [-Wincompatible-pointer-types] self->read = log_transport_streams_read; ../source/syslog-ng-3.12.1/modules/afstreams/afstreams.c: In function 'afstreams_init_door': ../source/syslog-ng-3.12.1/modules/afstreams/afstreams.c:133:11: warning: implicit declaration of function 'close'; did you mean 'pclose'? [-Wimplicit-function-declaration] close(fd);
2) If I avoid that unix-stream source and socat the client from a unix domain socket destination into a syslog port source, syslog-ng now has a mismatch of a source sending no framing, and a destination expecting it.
syslog() write(5, 0x000721E8, 60) = 60 5 7 < 1 3 > 1 2 0 1 8 - 0 2 - 0 3 T 0 3 : 1 1 : 2 0 + 0 0 : 0 0 o p s t e s t 6 a - - - - t e s t 0 8\n
unix-stream() read(6, 0x00072B00, 8192) = 57 < 1 3 > 1 2 0 1 8 - 0 2 - 0 3 T 0 2 : 0 5 : 0 5 + 0 0 : 0 0 o p s t e s t 6 a - - - - t e s t 0 7\n
I take it that if I did something evil like faking up a length header with length() in a template() I might get unix-stream -> syslog working?
On Fri, Feb 02, 2018 at 06:48:09AM +0100, Scheidler, Bal?zs wrote:
If the new lines at the end are ok, and the messages dont contain newlines themselves, you should be fine. If both destination and source is unix-STREAM(). Don't you hapoen to use unix-dgram() somewhere? That would take the datagram fragments as individual messages.
Can you show me the framing errors you mention?
On Feb 1, 2018 22:33, "Declan White" <declanw@is.bbc.co.uk> wrote:
I have syslog-ng pumping data into a unix-stream destination socketfilething.
Data going into this domain socket has lots of adventures, and then emerges as a source unix-stream on another host. Both have flags(syslog-protocol), but the data gets framing errors at the receiver.
I think it's become a bytestream (instead of a message stream) enroute, and the recv() is relying on only getting one message at a time, because it can't know the message length any other way, as unix-stream gets messages without framing(?).
Is there a way to salvage this? (I noticed TLS might add framing, but since the tunnel is already TLS ...)
-- Declan White
participants (2)
-
Declan White
-
Scheidler, Balázs