<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
Thank you <span style="font-size:12pt;background-color:rgba(0, 0, 0, 0);display:inline !important">Balázs,</span></div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
<span style="font-size:12pt;background-color:rgba(0, 0, 0, 0);display:inline !important"><br>
</span></div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
<span style="font-size:12pt;background-color:rgba(0, 0, 0, 0);display:inline !important">I have done some more digging and I don't believe it is syslog that is the issue, I think the traffic is being forwarded to me from another syslog platform and that is
where the issue lies.</span></div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
<span style="font-size:12pt;background-color:rgba(0, 0, 0, 0);display:inline !important"><br>
</span></div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
<span style="font-size:12pt;background-color:rgba(0, 0, 0, 0);display:inline !important">A packet capture is showing individual events with the correct line breaks, but then it also shows a number of events being sent as one large packet. Unless I can get syslog-ng
to linebreak on regex when the data comes in, I need to go back to the source and get them to address this.</span></div>
<div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
<br>
</div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
PS the template mismatch below was a typo.</div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
<br>
</div>
<div id="Signature">
<div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
</div>
<div></div>
<div></div>
<div></div>
<div></div>
<div></div>
<div></div>
<div name="divtagdefaultwrapper" style="margin: 0px;">
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;"><span>Kind Regards</span></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;"><span><br>
</span></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;"><span>John</span></div>
</div>
</div>
</div>
</div>
<div id="signature_bookmark"></div>
<div id="appendonsend"></div>
<div style="font-family:Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> syslog-ng <syslog-ng-bounces@lists.balabit.hu> on behalf of Balazs Scheidler <bazsi77@gmail.com><br>
<b>Sent:</b> 20 August 2022 06:33<br>
<b>To:</b> Syslog-ng users' and developers' mailing list <syslog-ng@lists.balabit.hu><br>
<b>Subject:</b> Re: [syslog-ng] Potential issue with line breaking not working correctly when writing logs to disk</font>
<div> </div>
</div>
<div>
<div dir="auto">
<div>Hi,</div>
<div dir="auto"><br>
</div>
<div dir="auto">Sorry I have trouble matching up the config with the log output. Please find some comments inline.</div>
<div dir="auto"><br>
</div>
<div dir="auto" class="elementToProof">Balázs<br>
<br>
<div class="x_gmail_quote" dir="auto">
<div dir="ltr" class="x_gmail_attr">On Fri, Aug 19, 2022, 16:47 John Law <<a href="mailto:JohnLaw@tfl.gov.uk" data-auth="NotApplicable">JohnLaw@tfl.gov.uk</a>> wrote:<br>
</div>
<blockquote class="x_gmail_quote" style="margin:0 0 0 .8ex; border-left:1px #ccc solid; padding-left:1ex">
<div dir="ltr">
<div>
<div class="x_MsoNormal" style="margin:0px 0cm; font-size:11pt; font-family:Calibri,sans-serif">
Hi,
<div><br>
</div>
<div>We have a number of network devices sending syslog traffic to syslog-ng OSE 3.25 installed on RHEL 7.9.</div>
<div><br>
</div>
<div>We are having an issue where multiple events are being written as the same line.</div>
<div><br>
</div>
<div>The config file is </div>
<div><br>
</div>
<div>template-function adm1 "${HOST} message-length=$(length \"${MSG}\") ${MESSAGE}\n";</div>
<div>source s_adm1 {</div>
<div> udp(ip(0.0.0.0) port(5527) );</div>
<div> tcp(ip(0.0.0.0) port(5527) );</div>
<div>};</div>
<div>destination d_adm1 {</div>
<div> file(</div>
<div> "/Data/syslog_data/$LOGHOST/$R_YEAR-$R_MONTH-$R_DAY/adm1/$HOST_FROM/$HOST/$FACILITY.local"</div>
<div> template("$(adm1)\n")</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
<div dir="auto"><br>
</div>
<div dir="auto">You only mention $(adm1) as a template function but this one refers to $(adm1-function)</div>
<div dir="auto"><br>
</div>
<div dir="auto">
<div class="x_gmail_quote" dir="auto">
<blockquote class="x_gmail_quote" style="margin:0 0 0 .8ex; border-left:1px #ccc solid; padding-left:1ex">
<div dir="ltr">
<div>
<div class="x_MsoNormal" style="margin:0px 0cm; font-size:11pt; font-family:Calibri,sans-serif">
<div> );</div>
<div>};</div>
<div>log { source(s_adm1); destination(d_adm1); flags(final); };</div>
<div><br>
</div>
<div><br>
</div>
<div>A small extract of one of the files is, naturally I have changed the IPs</div>
<div><br>
</div>
<div><br>
</div>
<div><a href="http://webdefence.global.blackspider.com/urlwrap/?q=AXicY3BnUGFmYAgUZWQoyqk0NMjQKy4q08tNzMxJzs8rKcrP0UvOz2UwMvc2jYhMNDU0MDO1MGfIys_Iy0ksdyhJy9FLzy_TK81myCgpKbDS1ze0NNIzBEF9Y0NDC0NdBgYGLR0GBgCskBrr&Z" data-auth="NotApplicable">192.1.1.1/31181-</a>><a href="http://webdefence.global.blackspider.com/urlwrap/?q=AXicY3BnUGFmYAgUZWQoyqk0NMjQKy4q08tNzMxJzs8rKcrP0UvOz2UwMvc2jYhMNDU0MDO1MGfIys_Iy0ksdyhJy9FLzy_TK81myCgpKbDS1ze0NNIz1DPWM9Q3NDZlYGDQ0mFgAABZfBpb&Z" data-auth="NotApplicable">192.1.3.1/135</a>
0x0 source rule r12 N/A N/A 6 ACME-CNB010 ACME CNB 1727847 N/A(N/A) reth3.860 UNKNOWN UNKNOWN UNKNOWN275 <14>Aug 19 11:38:50 sunny-fwl29 RT_FLOW: RT_FLOW_SESSION_CREATE: session created
<a href="http://webdefence.global.blackspider.com/urlwrap/?q=AXicBcG9DoIwFAbQz_gMvIGTkduf2Bac3Byc3GBsSLTKhUtqxfj2noMLdlvgVm2Q-Wd0ondeaYpPHmQuWZgGmWDD1XV9dEZ71wS8JM0cv-dyZ3rISp8RqZTlpJRpLXmymow6But0DWB_AP7JSxsl&Z" data-auth="NotApplicable">
192.6.20.1/47250-</a>><a href="http://webdefence.global.blackspider.com/urlwrap/?q=AXicY3BnUGFmYAgUZWQoyqk0NMjQKy4q08tNzMxJzs8rKcrP0UvOz2UwMvc2jYhMNDU0MDO1MGfIys_Iy0ksdyhJy9FLzy_TK81myCgpKbDS1ze0NNIzNDDRMzLQM9Q3MTFmYGDQ0mFgAACs0Brw&Z" data-auth="NotApplicable">192.104.20.1/443</a>
0x0 junos-https <a href="http://webdefence.global.blackspider.com/urlwrap/?q=AXicBcG9DoIwFAbQz_gMvIGTkduf2Bac3Byc3GBsSLTKhUtqxfj2noMLdlvgVm2Q-Wd0ondeaYpPHmQuWZgGmWDD1XV9dEZ71wS8JM0cv-dyZ3rISp8RqZTlpJRpLXmymow6But0DWB_AP7JSxsl&Z" data-auth="NotApplicable">
192.6.20.1/47250-</a>><a href="http://webdefence.global.blackspider.com/urlwrap/?q=AXicY3BnUGFmYAgUZWQoyqk0NMjQKy4q08tNzMxJzs8rKcrP0UvOz2UwMvc2jYhMNDU0MDO1MGfIys_Iy0ksdyhJy9FLzy_TK81myCgpKbDS1ze0NNIzNDDRMzLQM9Q3MTFmYGDQ0mFgAACs0Brw&Z" data-auth="NotApplicable">192.104.20.1/443</a>
0x0 N/A N/A N/A N/A 6 ACME-BWC002 ACME BWC_EDESIX_AWS 917879 N/A(N/A) reth3.860 UNKNOWN UNKNOWN UNKNOWN275 <14>Aug 19 11:38:50 sunny-fwl29 RT_FLOW: RT_FLOW_SESSION_CREATE: session created</div>
<div dir="auto"></div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
<div dir="auto"><br>
</div>
<div dir="auto">Can you show me the lines intermixed here? Again this does not seem to have been formatted with $(adm1) at least as defined above.</div>
<div dir="auto"><br>
</div>
<div dir="auto"><br>
</div>
<div dir="auto">
<div class="x_gmail_quote" dir="auto">
<blockquote class="x_gmail_quote" style="margin:0 0 0 .8ex; border-left:1px #ccc solid; padding-left:1ex">
<div dir="ltr">
<div>
<div class="x_MsoNormal" style="margin:0px 0cm; font-size:11pt; font-family:Calibri,sans-serif">
<div dir="auto"> <a href="http://webdefence.global.blackspider.com/urlwrap/?q=AXicBcGxDoIwEAbg3_gMvoGT0Wt7sRSZ3ByY3HRsSLTqwZFaML4934cLtmvgulkhy9_ZRN88Ux9f0ulQsgp12oND62_36J2tfB3w1jRI_J3LQ-ipM00fpFLGxhh3YqqILTlzDOz5AGC3BxbJXRsn&Z" data-auth="NotApplicable">192.6.20.1/47252-</a>><a href="http://webdefence.global.blackspider.com/urlwrap/?q=AXicY3BnUGFmYAgUZWQoyqk0NMjQKy4q08tNzMxJzs8rKcrP0UvOz2UwMvc2jYhMNDU0MDO1MGfIys_Iy0ksdyhJy9FLzy_TK81myCgpKbDS1ze0NNIzNDDRMzLQM9Q3MTFmYGDQ0mFgAACs0Brw&Z" data-auth="NotApplicable">192.104.20.1/443</a>
0x0 junos-https <a href="http://webdefence.global.blackspider.com/urlwrap/?q=AXicBcGxDoIwEAbg3_gMvoGT0Wt7sRSZ3ByY3HRsSLTqwZFaML4934cLtmvgulkhy9_ZRN88Ux9f0ulQsgp12oND62_36J2tfB3w1jRI_J3LQ-ipM00fpFLGxhh3YqqILTlzDOz5AGC3BxbJXRsn&Z" data-auth="NotApplicable">
192.6.20.1/47252-</a>><a href="http://webdefence.global.blackspider.com/urlwrap/?q=AXicY3BnUGFmYAgUZWQoyqk0NMjQKy4q08tNzMxJzs8rKcrP0UvOz2UwMvc2jYhMNDU0MDO1MGfIys_Iy0ksdyhJy9FLzy_TK81myCgpKbDS1ze0NNIzNDDRMzLQM9Q3MTFmYGDQ0mFgAACs0Brw&Z" data-auth="NotApplicable">192.104.20.1/443</a>
0x0 N/A N/A N/A N/A 6 ACME-BWC002 ACME BWC_EDESIX_AWS 595069 N/A(N/A) reth3.860 UNKNOWN UNKNOWN UNKNOWN285 <14>Aug 19 11:38:50 sunny-fwl29 RT_FLOW: RT_FLOW_SESSION_CREATE: session created
<a href="http://webdefence.global.blackspider.com/urlwrap/?q=AXicBcHBDgExEAbgXzyDN3ASpp3GKHtyc3By49hsQpfZHanuirf3fThhOQcuixmK_thn-pSJ-tRpa0MtptRajxDPcr0lYb-TfcTT8qDpe6x3pYdNNL6Qa303zvEhEJNEYifb4GUDYLUG_sl4Gyg&Z" data-auth="NotApplicable">
192.1.57.1/54205-</a>><a href="http://webdefence.global.blackspider.com/urlwrap/?q=AXicY3BnUGFmYAgUZWQoyqk0NMjQKy4q08tNzMxJzs8rKcrP0UvOz2UwMvc2jYhMNDU0MDO1MGfIys_Iy0ksdyhJy9FLzy_TK81myCgpKbDS1ze0NNIz1DPXM9Q3NDZnYGDQ0mFgAABZxBph&Z" data-auth="NotApplicable">192.1.7.1/137</a>
0x0 junos-nbname <a href="http://webdefence.global.blackspider.com/urlwrap/?q=AXicBcHBDgExEAbgXzyDN3ASpp3GKHtyc3By49hsQpfZHanuirf3fThhOQcuixmK_thn-pSJ-tRpa0MtptRajxDPcr0lYb-TfcTT8qDpe6x3pYdNNL6Qa303zvEhEJNEYifb4GUDYLUG_sl4Gyg&Z" data-auth="NotApplicable">192.1.57.1/54205-</a>><a href="http://webdefence.global.blackspider.com/urlwrap/?q=AXicY3BnUGFmYAgUZWQoyqk0NMjQKy4q08tNzMxJzs8rKcrP0UvOz2UwMvc2jYhMNDU0MDO1MGfIys_Iy0ksdyhJy9FLzy_TK81myCgpKbDS1ze0NNIz1DPXM9Q3NDZnYGDQ0mFgAABZxBph&Z" data-auth="NotApplicable">192.1.7.1/137</a>
0x0 N/A N/A N/A N/A 17 ACME-COMP007 ACME CCA 1735324 N/A(N/A) reth3.860 UNKNOWN UNKNOWN UNKNOWN288 <14>Aug 19 11:38:50 sunny-fwl29 RT_FLOW: RT_FLOW_SESSION_CREATE: session created
<a href="http://webdefence.global.blackspider.com/urlwrap/?q=AXicBcGxDoIwEAbgn_gMvoGT0SvX2Bad3ByY2HRsSLTowZlaMb4934cLNiugW1fI8uc60SfPNMZBep1KVqFeR9jQuustOq69awKemiaJv3O5Cz10pu8LqZT3yRg-WmJygdh4e2j8HsB2ByzJ3Bsy&Z" data-auth="NotApplicable">
192.1.57.1/62486-</a>><a href="http://webdefence.global.blackspider.com/urlwrap/?q=AXicY3BnUGFmYAgUZWQoyqk0NMjQKy4q08tNzMxJzs8rKcrP0UvOz2UwMvc2jYhMNDU0MDO1MGfIys_Iy0ksdyhJy9FLzy_TK81myCgpKbDS1ze0NNIz1DPXM9Q3NDZlYGDQ0mFgAABZtBpf&Z" data-auth="NotApplicable">192.1.7.1/135</a>
0x0 junos-ms-rpc-tcp <a href="http://webdefence.global.blackspider.com/urlwrap/?q=AXicBcGxDoIwEAbgn_gMvoGT0SvX2Bad3ByY2HRsSLTowZlaMb4934cLNiugW1fI8uc60SfPNMZBep1KVqFeR9jQuustOq69awKemiaJv3O5Cz10pu8LqZT3yRg-WmJygdh4e2j8HsB2ByzJ3Bsy&Z" data-auth="NotApplicable">192.1.57.1/62486-</a>><a href="http://webdefence.global.blackspider.com/urlwrap/?q=AXicY3BnUGFmYAgUZWQoyqk0NMjQKy4q08tNzMxJzs8rKcrP0UvOz2UwMvc2jYhMNDU0MDO1MGfIys_Iy0ksdyhJy9FLzy_TK81myCgpKbDS1ze0NNIz1DPXM9Q3NDZlYGDQ0mFgAABZtBpf&Z" data-auth="NotApplicable">192.1.7.1/135</a>
0x0 N/A N/A N/A N/A 6 ACME-COMP007 ACME CCA 1027474 N/A(N/A) reth3.860 UNKNOWN UNKNOWN UNKNOWN249</div>
<div><br>
</div>
I suspect it might be something to do with the amount of logs that are received in very quick succession, but I have no idea how to address this. One thing I have noticed is that it always appears to be the same device that is causing the issue. When I look
at other devices coming in on the same port, they appear to write out correctly.<br>
</div>
<div class="x_MsoNormal" style="margin:0px 0cm; font-size:11pt; font-family:Calibri,sans-serif">
<br>
</div>
<div class="x_MsoNormal" style="margin:0px 0cm; font-size:11pt; font-family:Calibri,sans-serif">
Any suggestion would be really welcome.</div>
<div class="x_MsoNormal" style="margin:0px 0cm; font-size:11pt; font-family:Calibri,sans-serif">
<br>
</div>
<div class="x_MsoNormal" style="margin:0px 0cm; font-size:11pt; font-family:Calibri,sans-serif">
Thanks</div>
<div class="x_MsoNormal" style="margin:0px 0cm; font-size:11pt; font-family:Calibri,sans-serif">
<br>
</div>
<div class="x_MsoNormal" style="margin:0px 0cm; font-size:11pt; font-family:Calibri,sans-serif">
John</div>
</div>
<br>
<br>
<p align="center"><font style="background-color:#ffffff">This message has been scanned for malware by Forcepoint.
</font><a href="http://www.forcepoint.com/" data-auth="NotApplicable"><font color="#000000" style="background-color:#ffffff">www.forcepoint.com</font></a></p>
</div>
______________________________________________________________________________<br>
Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" data-auth="NotApplicable">
https://lists.balabit.hu/mailman/listinfo/syslog-ng</a><br>
Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" data-auth="NotApplicable">
http://www.balabit.com/support/documentation/?product=syslog-ng</a><br>
FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" data-auth="NotApplicable">
http://www.balabit.com/wiki/syslog-ng-faq</a><br>
<br>
</blockquote>
</div>
</div>
</div>
</div>
</body>
</html>