<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">
Hi Evan,</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">
thank you for your reply.</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">
<br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">
I copied a part of the stored log file below. There is no line break between a lot of syslog messages and then just before (hashtags) this performance output it starts separating lines.</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">
And each performance stat is a new line. After that (right away after hashtags) it starts storing again without line breaks.</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">
This is mostly going on and on like this.</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">
<br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">
I didn't change windows size so it should be default. Currently there is only one source host sending syslog messages.</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">
<br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof ContentPasted6">
...<189>Jan 13 2023 13:29:58+02:00 hostname %%01SSH/5/SSHS_REKEY_STATUS(s):CID=xxx;SSH server key renegotiation with client. (SessionID=1, RekeyReason=Rekey timer timed out, Status=Success, UserAddress=x.y.w.z, LocalAddress=x.y.w.z, VPNInstanceName=_xxx_)<189>Jan
13 2023 13:29:59+02:00 hostname %%01SSH/5/SSHS_REKEY_STATUS(s):CID=xxx;SSH server key renegotiation with client. (SessionID=2, RekeyReason=Rekey timer timed out, Status=Begin, UserAddress=x.y.w.z, LocalAddress=x.y.w.z, VPNInstanceName=_xxx_)<189>Jan 13 2023
13:30:00+02:00 hostname %%01SSH/5/SSHS_REKEY_STATUS(s):CID=xxx;SSH server key renegotiation with client. (SessionID=2, RekeyReason=Rekey timer timed out, Status=Success, UserAddress=x.y.w.z, LocalAddress=x.y.w.z, VPNInstanceName=_xxx_)<190>Jan 13 2023 13:48:30+02:00
hostname %%01DEBUG/6/DBG_HEALTH(l):CID=0x80cc000d;Automatic record:
<div class="ContentPasted6">###########################################</div>
<div class="ContentPasted6">#Automatic record log end,current health information as follows:</div>
<div class="ContentPasted6"><190>Jan 13 2023 13:48:30+02:00 hostname %%01DEBUG/6/DBG_HEALTH(l):CID=xxx;Automatic record:</div>
<div class="ContentPasted6">Slot CPU Memory(Used/Total) Physical Memory Usage(Free/Total/Cache)</div>
<div class="ContentPasted6">--------------------------------------------------------------------------------------</div>
<div class="ContentPasted6">1 IPU(Master) 12% 26% 4006MB/15394MB 34% 10123MB/15396MB/2287MB</div>
<div class="ContentPasted6"> CPU0 17%</div>
<div class="ContentPasted6"> CPU1 12%</div>
<div class="ContentPasted6"> CPU2 13%</div>
<div class="ContentPasted6"> CPU3 14%</div>
<div class="ContentPasted6"> CPU4 7%</div>
<div class="ContentPasted6"> CPU5 11%</div>
<div class="ContentPasted6"> ProcessId CPU</div>
<div class="ContentPasted6"> 1019 1%</div>
<div class="ContentPasted6"> 1001 3%</div>
<div class="ContentPasted6"> 1005 0%</div>
<div class="ContentPasted6"> 1015 41%</div>
<div class="ContentPasted6"> 1012 0%</div>
<div class="ContentPasted6"> 3 1%</div>
<div class="ContentPasted6"> 10001 2%</div>
<div class="ContentPasted6"> 1006 1%</div>
<div class="ContentPasted6"> 1018 0%</div>
<div class="ContentPasted6"> 1000 0%</div>
<div class="ContentPasted6"> 1013 2%</div>
<div class="ContentPasted6"> 1003 1%</div>
<div class="ContentPasted6"> 1007 0%</div>
<div class="ContentPasted6"> 1010 1%</div>
<div class="ContentPasted6"> 1008 1%</div>
<div class="ContentPasted6"> 1016 201%</div>
<div class="ContentPasted6"> 1017 1%</div>
<div class="ContentPasted6"> 1014 0%</div>
<div class="ContentPasted6"> 1011 0%</div>
<div class="ContentPasted6"> .<190>Jan 13 2023 13:48:30+02:00 hostname %%01DEBUG/6/DBG_HEALTH(l):CID=xxxx;Automatic record:</div>
<div class="ContentPasted6">#DateTime Stamp:2023-01-13 13:48:30.673</div>
###########################################<188>Jan 13 2023 13:49:16+02:00 hostname %%01SNMP/4/SNMP_MIB_SET(s):CID=xxx;MIB node set. (UserName=xxx, SourceIP=x.y.w.z, DestIP=x.y.w.z, Version=v3, RequestId=xxx, hwCfgOperateType.89005=6,hwCfgOperateProtocol.89005=3,hwCfgOperateFileName.89005=[xxx(hex)],hwCfgOperateServerAddress.89005=x.y.w.z,hwCfgOperateUserName.89005=[xxx(hex)],hwCfgOperateUserPassword.89005=******,hwCfgOperateServerPort.89005=xxx,hwCfgOperateRowStatus.89005=xxx,
VPN=xxx)<188>Jan 13 2023 13:49:17+02:00 hostname %%01CONFIGURATION/4/CONFIGMIB_FILE_OPERATE_FINISH(l):CID=xxx;Configuration was copied. (OperationType=6, OptTime=94, OptState=2, OptEndTime=24674900)...<br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof ContentPasted6">
<br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof ContentPasted6">
<br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof ContentPasted6">
<br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof ContentPasted6">
<br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof ContentPasted6">
Br,</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof ContentPasted6">
Dragan</div>
<div id="appendonsend"></div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> syslog-ng <syslog-ng-bounces@lists.balabit.hu> on behalf of Evan Rempel <erempel@uvic.ca><br>
<b>Sent:</b> Thursday, February 2, 2023 2:07 PM<br>
<b>To:</b> syslog-ng@lists.balabit.hu <syslog-ng@lists.balabit.hu><br>
<b>Subject:</b> Re: [syslog-ng] Syslog messages not stored in separate lines</font>
<div> </div>
</div>
<div>
<div class="x_moz-cite-prefix"><br>
</div>
<div class="x_moz-cite-prefix">Is there a line break anywhere in the log file?</div>
<div class="x_moz-cite-prefix"><br>
</div>
<div class="x_moz-cite-prefix">If yes</div>
<div class="x_moz-cite-prefix"><br>
</div>
<div class="x_moz-cite-prefix">1. is the line break in the middle of a syslog line from this device?</div>
<div class="x_moz-cite-prefix">2. is the line break at the maximum message size?</div>
<div class="x_moz-cite-prefix">3. is the line break at the end of a log message from a different device that is logging to the same s_xxx_xxx source?</div>
<div class="x_moz-cite-prefix"><br>
</div>
<div class="x_moz-cite-prefix"><br>
</div>
<div class="x_moz-cite-prefix">What I am wondering is that the source of the log messages is logging the exact same stream of data to the TCP port that it would have over the UDP port (which would be an error). UDP messages are not terminated with a new line,
while the TCP messages are. If that were the case then syslog-ng would never see multiple messages, and would write a continuous stream on a single line until it reached the maximum message length, or it logged a correctly terminated message from a different
device.<br>
</div>
<div class="x_moz-cite-prefix"><br>
</div>
<div class="x_moz-cite-prefix">Evan.<br>
</div>
<div class="x_moz-cite-prefix"><br>
</div>
<div class="x_moz-cite-prefix"><br>
</div>
<div class="x_moz-cite-prefix">On 2023-02-02 05:55, Dragan Zecevic wrote:<br>
</div>
<blockquote type="cite"><style type="text/css" style="display:none">
<!--
p
{margin-top:0;
margin-bottom:0}
-->
</style><br>
<div>
<div class="x_elementToProof" style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0); background-color:rgb(255,255,255)">
Hi Balazs,</div>
<div class="x_elementToProof" style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0); background-color:rgb(255,255,255)">
<br>
</div>
<div class="x_elementToProof" style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0); background-color:rgb(255,255,255)">
this is how the whole config snippet looks like:</div>
<div class="x_elementToProof" style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0); background-color:rgb(255,255,255)">
<br>
</div>
<div class="x_elementToProof x_ContentPasted0" style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0); background-color:rgb(255,255,255)">
source s_xxx_xxx {
<div class="x_ContentPasted0"> network(</div>
<div class="x_ContentPasted0"> ip(0.0.0.0)</div>
<div class="x_ContentPasted0"> transport(tcp)</div>
<div class="x_ContentPasted0"> port(xxxx)</div>
<div class="x_ContentPasted0"> flags(store-raw-message)</div>
<div class="x_ContentPasted0"> );</div>
<div class="x_ContentPasted0">};</div>
<div><br class="x_ContentPasted0">
</div>
<div><br class="x_ContentPasted0">
</div>
<div><br class="x_ContentPasted0">
</div>
<div class="x_ContentPasted0">filter filter_xxx_xxx {</div>
<div><br class="x_ContentPasted0">
</div>
<div class="x_ContentPasted0"> host("xxx") or host("xxx") ...;</div>
<div class="x_ContentPasted0">};</div>
<div><br class="x_ContentPasted0">
</div>
<div><br class="x_ContentPasted0">
</div>
<div class="x_ContentPasted0">destination folder_xxx_xxx {</div>
<div><br class="x_ContentPasted0">
</div>
<div class="x_ContentPasted0"> file(</div>
<div class="x_ContentPasted0"> "/xxx/.../xxx/${R_YEAR}${R_MONTH}${R_DAY}/${SOURCEIP}_${HOST}_${R_HOUR}.log"</div>
<div class="x_ContentPasted0"> template("${RAWMSG}\n")</div>
<div class="x_ContentPasted0"> dir-group(xxx)</div>
<div class="x_ContentPasted0"> dir-perm(xxx)</div>
<div class="x_ContentPasted0"> group(xxx)</div>
<div><br class="x_ContentPasted0">
</div>
<div class="x_ContentPasted0"> );</div>
<div class="x_ContentPasted0">};</div>
<div><br class="x_ContentPasted0">
</div>
<div><br class="x_ContentPasted0">
</div>
<div class="x_ContentPasted0">log {</div>
<div class="x_ContentPasted0"> source(s_xxx_xxx); filter(filter_xxx_xxx); destination(folder_xxx_xxx);</div>
};<br>
</div>
<div class="x_elementToProof x_ContentPasted0" style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0); background-color:rgb(255,255,255)">
<br>
</div>
<div class="x_elementToProof x_ContentPasted0" style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0); background-color:rgb(255,255,255)">
Sorry for the xxx but I can't export real parameters in conversation like this. Also, I can't provide some pcap or tcpdump.</div>
<div class="x_elementToProof x_ContentPasted0" style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0); background-color:rgb(255,255,255)">
<br>
</div>
<div class="x_elementToProof x_ContentPasted0" style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0); background-color:rgb(255,255,255)">
I restarted syslog-ng multiple times because I also added some other sources and there were no error messages. And this is the only part of the configuration where either this destination or folder are used.</div>
<div class="x_elementToProof x_ContentPasted0" style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0); background-color:rgb(255,255,255)">
<br>
</div>
<div class="x_elementToProof x_ContentPasted0" style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0); background-color:rgb(255,255,255)">
I hope this info is helpful.</div>
<div class="x_elementToProof x_ContentPasted0" style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0); background-color:rgb(255,255,255)">
<br>
</div>
<div class="x_elementToProof x_ContentPasted0" style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0); background-color:rgb(255,255,255)">
Thanks,
<div>Dragan</div>
</div>
<div class="x_elementToProof">
<div id="x_Signature"><br>
</div>
</div>
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="x_divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> syslog-ng
<a class="x_moz-txt-link-rfc2396E" href="mailto:syslog-ng-bounces@lists.balabit.hu">
<syslog-ng-bounces@lists.balabit.hu></a> on behalf of Balazs Scheidler <a class="x_moz-txt-link-rfc2396E" href="mailto:bazsi77@gmail.com">
<bazsi77@gmail.com></a><br>
<b>Sent:</b> Tuesday, January 31, 2023 9:18 PM<br>
<b>To:</b> Syslog-ng users' and developers' mailing list <a class="x_moz-txt-link-rfc2396E" href="mailto:syslog-ng@lists.balabit.hu">
<syslog-ng@lists.balabit.hu></a><br>
<b>Subject:</b> Re: [syslog-ng] Syslog messages not stored in separate lines</font>
<div> </div>
</div>
<div>
<div dir="auto">
<div>If there's an initialization error with a config at reloading, syslog-ng can fall back to the old one. Can this happen? Or two destinations writing the same file?</div>
<div dir="auto"><br>
</div>
<div dir="auto">If the problem persists, can you create minimal example with complete with config and sample message that you send and which reproduces the issue?</div>
<div dir="auto"><br>
</div>
<div dir="auto">Thanks<br>
<br>
<div class="x_x_gmail_quote" dir="auto">
<div dir="ltr" class="x_x_gmail_attr">On Tue, Jan 31, 2023, 20:44 Dragan Zecevic <<a href="mailto:dragan.zecevic@live.com" class="x_moz-txt-link-freetext">dragan.zecevic@live.com</a>> wrote:<br>
</div>
<blockquote class="x_x_gmail_quote" style="margin:0 0 0
.8ex; border-left:1px #ccc solid; padding-left:1ex">
<div dir="ltr">
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0); background-color:rgb(255,255,255)">
<br>
</div>
<div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Hi Balazs,</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
thank you for your reply.</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Yes, I used config like this for other sources as well and restarted syslog-ng service.</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
I don't get it why it doesn't work in this case. I was thinking it is up to the input.</div>
<div id="x_x_m_4134358523529882524Signature"><br>
</div>
</div>
<hr style="display:inline-block; width:98%">
<div id="x_x_m_4134358523529882524divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> syslog-ng <<a href="mailto:syslog-ng-bounces@lists.balabit.hu" target="_blank" rel="noreferrer" class="x_moz-txt-link-freetext">syslog-ng-bounces@lists.balabit.hu</a>>
on behalf of Balazs Scheidler <<a href="mailto:bazsi77@gmail.com" target="_blank" rel="noreferrer" class="x_moz-txt-link-freetext">bazsi77@gmail.com</a>><br>
<b>Sent:</b> Monday, January 30, 2023 6:48 AM<br>
<b>To:</b> Syslog-ng users' and developers' mailing list <<a href="mailto:syslog-ng@lists.balabit.hu" target="_blank" rel="noreferrer" class="x_moz-txt-link-freetext">syslog-ng@lists.balabit.hu</a>><br>
<b>Subject:</b> Re: [syslog-ng] Syslog messages not stored in separate lines</font>
<div> </div>
</div>
<div>
<div dir="auto">
<div>This would be very strange indeed as the template of your file destination includes a newline character at the end of every message, so it should not depend on the input.</div>
<div dir="auto"><br>
</div>
<div dir="auto">You sure that this is the destination config that you quote here? Did you reload syslog-ng to use that config?<br>
<br>
<div dir="auto">
<div dir="ltr">On Sun, Jan 29, 2023, 13:55 Dragan Zecevic <<a href="mailto:dragan.zecevic@live.com" target="_blank" rel="noreferrer" class="x_moz-txt-link-freetext">dragan.zecevic@live.com</a>> wrote:<br>
</div>
<blockquote style="margin:0 0 0 .8ex; border-left:1px #ccc solid; padding-left:1ex">
<div dir="ltr">
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0); background-color:rgb(255,255,255)">
<br>
</div>
<div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Hi,</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
I am collecting logs from a network device. They configured syslog format on their source side to be RFC3164.</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
On syslog-ng side I am using source and destination like this:</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
source s_xxx {
<div> network(</div>
<div> ip(0.0.0.0)</div>
<div> transport(tcp)</div>
<div> port(xxx)</div>
<div> flags(store-raw-message)</div>
<div> );</div>
<div>};</div>
<div><br>
</div>
<div><br>
</div>
<div>destination folder_xxx {</div>
<div><br>
</div>
<div> file(</div>
<div> "/xxx/${R_YEAR}${R_MONTH}${R_DAY}/${SOURCEIP}_${HOST}_${R_HOUR}.log"</div>
<div> template("${RAWMSG}\n")</div>
<div> );</div>
};</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
syslog-ng version 3.34</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
CentOS Linux release 7.9.2009</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
The problem is that syslog messages are stored in raw format but not separated in different line. Parity bit of new message starts imidiatelly after previous line -without space or enter.</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
I have the same config for some other hosts and there log files are created with separate lines. Vendor says they can't change anything on source side.</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Do you have any idea what is the cause of this?</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Thank you.</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Br,</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Dragan</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<br>
</div>
</body>
</html>