[syslog-ng] Issue with Juniper SRX structured stream logs getting parsed by syslog-ng

[Brad Riemann]

Hello,

I've been banging my head against the wall for a while with this
issue, I have a Juniper SRX that sends traffic logs to syslog-ng
(centos 6, 6, version 3.13.2-2), the key being they are structured so
the look like the following.

[2018-06-12T19:03:54.384014] Incoming log entry; line='<14>1
2018-06-12T19:03:54.125Z corefw-site-0 RT_FLOW - RT_FLOW_SESSION_CLOSE
[junos at 2636.1.1.1.2.40 reason="idle Timeout" source-address="10.1.1.1"
source-port="56998" destination-address="0.0.0.0"
destination-port="443" service-name="junos-https"
nat-source-address="10.100.1.1" nat-source-port="39155"
nat-destination-address="0.0.0.0" nat-destination-port="443"
src-nat-rule-type="source rule" src-nat-rule-name="test-servers"
dst-nat-rule-type="N/A" dst-nat-rule-name="N/A" protocol-id="6"
policy-name="allow-outbound" source-zone-name="test-zone"
destination-zone-name="untrust" session-id-32="499236"
packets-from-client="3" bytes-from-client="180"
packets-from-server="0" bytes-from-server="0" elapsed-time="19"
application="UNKNOWN" nested-application="UNKNOWN" username="N/A"
roles="N/A" packet-incoming-interface="reth0.1" encrypted="UNKNOWN"]'

I can confirm that syslog-ng is seeing the whole message as part of
the incoming log entry, however it's just not showing up.

I've dug through RFC 5424 and these logs DO conform but my confusion
still remains. When I check the logs that were written the line looks
like the following, which seems to just trunk everything after the
RT_FLOW of the message header.

2018-06-12T19:03:54.125+00:00 corefw-site-0 RT_FLOW:

I've run syslog-ng in debug mode, which was how I was able to confirm
the outgoing message matches the prior line as well as the incoming
message matching the expected data from my srx.

[2018-06-12T19:03:54.385505] Outgoing message;
message='2018-06-12T19:03:54.125+00:00 corefw-site-0 RT_FLOW: \x0a'

Here is a snippet of my syslog config, it isn't that complex (at least
I think so..) and i'm running out of hair to pull, any input would be
GREATLY appreciated.

https://gist.github.com/robotman321/217a69d456e543831936b1209b8bf501

-Brad