<div dir="ltr"><div>Dear Brad,</div><div><br></div><div>You should try adding something like this</div><div><br></div><pre class="gmail-prettyprint gmail-synopsis gmail-prettyprinted" style="display:table-cell"><span class="gmail-pln"></span><span class="gmail-kwd">template</span><span class="gmail-pun">(</span><span class="gmail-str">"${ISODATE} ${HOST} ${SDATA} ${MESSAGE}\n"</span><span class="gmail-pun">)</span></pre><div><br></div><div>to the file destination's definition.</div><div><br></div><div>Thus it would become:</div><div><br></div><div><pre class="gmail-prettyprint gmail-synopsis gmail-prettyprinted" style="display:table-cell"><span class="gmail-pun"></span><span class="gmail-str">destination syslog_log {<br>file("/var/log/syslog/$HOST/$FACILITY/$YEAR$MONTH/$YEAR$MONTH$DAY-$FACILITY.log"<br><pre class="gmail-prettyprint gmail-synopsis gmail-prettyprinted" style="display:table-cell"><span class="gmail-pln"></span><span class="gmail-kwd">template</span><span class="gmail-pun">(</span><span class="gmail-str">"${ISODATE} ${HOST} ${SDATA} ${MESSAGE}\n"</span><span class="gmail-pun">)</span></pre><br>owner(root)<br>group(root)<br>perm(0600)<br>dir_perm(0700)<br>create_dirs(yes)<br>);</span><span class="gmail-pun"></span></pre><br></div><div><br></div><div>Regards,</div><div>János</div><div><br></div><div><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr">--</div><div dir="ltr">Janos SZIGETVARI<br><span>RHCE, License no. <a href="https://www.redhat.com/rhtapps/verify/?certId=150-053-692" target="_blank">150-053-692</a></span><br></div><div dir="ltr"><span></span><br>__@__˚V˚<br>Make the switch to open (source) applications, protocols, formats now:<br>- windows -> Linux, iexplore -> Firefox, msoffice -> LibreOffice<br>- msn -> jabber protocol (Pidgin, Google Talk)<br>- mp3 -> ogg, wmv -> ogg, jpg -> png, doc/xls/ppt -> odt/ods/odp</div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div><br></div><br><div class="gmail_quote"><div dir="ltr">Brad Riemann <<a href="mailto:briemann@nmi.com">briemann@nmi.com</a>> ezt írta (időpont: 2018. jún. 13., Sze, 16:59):<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">János,<br>
<br>
I did, it should be in the gist i sent over:<br>
<a href="https://gist.github.com/robotman321/217a69d456e543831936b1209b8bf501" rel="noreferrer" target="_blank">https://gist.github.com/robotman321/217a69d456e543831936b1209b8bf501</a><br>
<br>
<br>
Fabian,<br>
<br>
I tried adding  flags(syslog-protocol) to the syslog() object however<br>
there were no changes in output.<br>
<br>
Thanks guys.<br>
-Brad<br>
<br>
On Wed, Jun 13, 2018 at 3:01 AM SZIGETVÁRI János <<a href="mailto:jszigetvari@gmail.com" target="_blank">jszigetvari@gmail.com</a>> wrote:<br>
><br>
> Hi Brad,<br>
><br>
> Could you please share with us the destination configuration?<br>
> I have a hunch that the problem may be that the SRX sends the log data totally as SDATA (with the message part being empty), and by default SDATA is not written into file destinations, so you only get an empty message part written to your file destination.<br>
> So in order to write the data contained in the SDATA fields to a file, you will have to edit the destination's template to (explicitly) include those fields as well.<br>
><br>
> Please let me know if my assumptions were correct.<br>
> Thanks!<br>
><br>
> Regards,<br>
> János<br>
> --<br>
> Janos SZIGETVARI<br>
> RHCE, License no. 150-053-692<br>
><br>
> LinkedIn: <a href="http://linkedin.com/in/janosszigetvari" rel="noreferrer" target="_blank">linkedin.com/in/janosszigetvari</a><br>
> E-mail: <a href="mailto:janos@szigetvari.com" target="_blank">janos@szigetvari.com</a>, <a href="mailto:jszigetvari@gmail.com" target="_blank">jszigetvari@gmail.com</a><br>
> Phone: +36209440412 (Hungary)<br>
><br>
> __@__˚V˚<br>
> Make the switch to open (source) applications, protocols, formats now:<br>
> - windows -> Linux, iexplore -> Firefox, msoffice -> LibreOffice<br>
> - msn -> jabber protocol (Pidgin, Google Talk)<br>
> - mp3 -> ogg, wmv -> ogg, jpg -> png, doc/xls/ppt -> odt/ods/odp<br>
><br>
><br>
> Brad Riemann <<a href="mailto:briemann@nmi.com" target="_blank">briemann@nmi.com</a>> ezt írta (időpont: 2018. jún. 13., Sze, 0:10):<br>
>><br>
>> Hello,<br>
>><br>
>> I've been banging my head against the wall for a while with this<br>
>> issue, I have a Juniper SRX that sends traffic logs to syslog-ng<br>
>> (centos 6, 6, version 3.13.2-2), the key being they are structured so<br>
>> the look like the following.<br>
>><br>
>> [2018-06-12T19:03:54.384014] Incoming log entry; line='<14>1<br>
>> 2018-06-12T19:03:54.125Z corefw-site-0 RT_FLOW - RT_FLOW_SESSION_CLOSE<br>
>> [junos@2636.1.1.1.2.40 reason="idle Timeout" source-address="10.1.1.1"<br>
>> source-port="56998" destination-address="0.0.0.0"<br>
>> destination-port="443" service-name="junos-https"<br>
>> nat-source-address="10.100.1.1" nat-source-port="39155"<br>
>> nat-destination-address="0.0.0.0" nat-destination-port="443"<br>
>> src-nat-rule-type="source rule" src-nat-rule-name="test-servers"<br>
>> dst-nat-rule-type="N/A" dst-nat-rule-name="N/A" protocol-id="6"<br>
>> policy-name="allow-outbound" source-zone-name="test-zone"<br>
>> destination-zone-name="untrust" session-id-32="499236"<br>
>> packets-from-client="3" bytes-from-client="180"<br>
>> packets-from-server="0" bytes-from-server="0" elapsed-time="19"<br>
>> application="UNKNOWN" nested-application="UNKNOWN" username="N/A"<br>
>> roles="N/A" packet-incoming-interface="reth0.1" encrypted="UNKNOWN"]'<br>
>><br>
>> I can confirm that syslog-ng is seeing the whole message as part of<br>
>> the incoming log entry, however it's just not showing up.<br>
>><br>
>> I've dug through RFC 5424 and these logs DO conform but my confusion<br>
>> still remains. When I check the logs that were written the line looks<br>
>> like the following, which seems to just trunk everything after the<br>
>> RT_FLOW of the message header.<br>
>><br>
>> 2018-06-12T19:03:54.125+00:00 corefw-site-0 RT_FLOW:<br>
>><br>
>> I've run syslog-ng in debug mode, which was how I was able to confirm<br>
>> the outgoing message matches the prior line as well as the incoming<br>
>> message matching the expected data from my srx.<br>
>><br>
>> [2018-06-12T19:03:54.385505] Outgoing message;<br>
>> message='2018-06-12T19:03:54.125+00:00 corefw-site-0 RT_FLOW: \x0a'<br>
>><br>
>> Here is a snippet of my syslog config, it isn't that complex (at least<br>
>> I think so..) and i'm running out of hair to pull, any input would be<br>
>> GREATLY appreciated.<br>
>><br>
>> <a href="https://gist.github.com/robotman321/217a69d456e543831936b1209b8bf501" rel="noreferrer" target="_blank">https://gist.github.com/robotman321/217a69d456e543831936b1209b8bf501</a><br>
>><br>
>> -Brad<br>
>> ______________________________________________________________________________<br>
>> Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" rel="noreferrer" target="_blank">https://lists.balabit.hu/mailman/listinfo/syslog-ng</a><br>
>> Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" rel="noreferrer" target="_blank">http://www.balabit.com/support/documentation/?product=syslog-ng</a><br>
>> FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" rel="noreferrer" target="_blank">http://www.balabit.com/wiki/syslog-ng-faq</a><br>
>><br>
> ______________________________________________________________________________<br>
> Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" rel="noreferrer" target="_blank">https://lists.balabit.hu/mailman/listinfo/syslog-ng</a><br>
> Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" rel="noreferrer" target="_blank">http://www.balabit.com/support/documentation/?product=syslog-ng</a><br>
> FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" rel="noreferrer" target="_blank">http://www.balabit.com/wiki/syslog-ng-faq</a><br>
><br>
______________________________________________________________________________<br>
Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" rel="noreferrer" target="_blank">https://lists.balabit.hu/mailman/listinfo/syslog-ng</a><br>
Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" rel="noreferrer" target="_blank">http://www.balabit.com/support/documentation/?product=syslog-ng</a><br>
FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" rel="noreferrer" target="_blank">http://www.balabit.com/wiki/syslog-ng-faq</a><br>
<br>
</blockquote></div>