<div dir="ltr">Hello Alain,<div><br></div><div>Thanks for sharing the debug output, it showed me where is the problem.</div><div>It seems that you don't have the "^M" as a special character in your message but 2 regular ascii characters '^' and 'M'.</div><div>In case of a carriage-return character syslog-ng would print this debug message (check the "\x0d" strings):</div><div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">[2017-12-06T10:08:57.374088] Incoming log entry; line='Dec 4 13:35:23 conf01 2017-12-04 13:35:23,561 Level="INFO" Name="support.sip" Message="Sending SIP response" Src-address="80.12.yy.xx" Src-port="5061" Dst-address="161.105.yy.xx" Dst-port="38509" Transport="TLS" Detail="\x0dMSIP/2.0 403 Forbidden\x0dMVia: SIP/2.0/TLS 161.105.150.12:38509;alias;branch=z9hG4bK.MEgSOM8O4;rport=38509;received=161.105.150.12\x0dMFrom: <sip:SondeSQS_001!@<a href="http://int.ovp.orange-business.com">int.ovp.orange-business.com</a>>;tag=5kYhVMAyi\x0dTo: sip:SondeSQS_001!@<a href="http://int.ovp.orange-business.com">int.ovp.orange-business.com</a>;tag=aynkBKUjt0pXHzNv\x0dCSeq: 25 REGISTER\x0dCall-ID: Vhihsb~BhQ\x0dAllow: INVITE,ACK,OPTIONS,CANCEL,BYE,REGISTER,INFO,SUBSCRIBE,NOTIFY,MESSAGE\x0dSupported: categoryList,adhoclist,sdp-anat,replaces\x0dContent-Length: 0\x0d\x0d"'</blockquote></div><div><br></div><div>You can remove these characters with subst rewrite rule, but please note that by default syslog-ng uses PCRE regular expressions, so you either escape the ^ or use literal strings as search pattern.</div><div>Also, when escaping I would recommend of the usage of single-quote instead of double-quotes, otherwise you need to double-escape special characters.</div><div>rewrite{ subst('\^M', ' ', flags("global")); };<br></div><div><div>rewrite{ subst("\\^M", ' ', flags("global")); };<br></div></div><div>or use literal strings instead of regex</div><div>rewrite { subst("^M", " ", type("string"), flags("substring")); };</div><div><br></div><div><br></div><div>Some notes about regular expressions(in case I forgot something):</div><div><a href="https://www.balabit.com/documents/syslog-ng-ose-latest-guides/en/syslog-ng-ose-guide-admin/html/regular-expressions.html">https://www.balabit.com/documents/syslog-ng-ose-latest-guides/en/syslog-ng-ose-guide-admin/html/regular-expressions.html</a><br></div><div><br></div><div>Best Regards,</div><div>Gabor</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Dec 5, 2017 at 2:03 PM, <span dir="ltr"><<a href="mailto:alain.villefranque@orange.com" target="_blank">alain.villefranque@orange.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div lang="FR" link="blue" vlink="purple">
<div class="m_-6955482930723540828WordSection1">
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black">Hi Gabor,<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black">Thanks for your help.<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black">I’ve tried in different way without success:<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black">rewrite r_rewrite_subst_CR {<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black"> subst('\r\n'," ",value("MESSAGE"), flags("global"));<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black"> subst('^M'," ",value("MESSAGE"), flags("global"));<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black"> subst("^M"," ",value("MESSAGE"), flags("global"));<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black"> subst("\r\n"," ",value("MESSAGE"), flags("global"));<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black"> subst('\r'," ",value("MESSAGE"), flags("global"));<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black"> subst("\r"," ",value("MESSAGE"), flags("global"));<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black">
</span><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black">subst("Detail","COUCOU",value(<wbr>"MESSAGE"), flags("global"));<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black">};<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black">You will see in the attached file that this rewrite rule has no effect on the received message specifically against the ^M character….<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black">Do you have any other advice to provide me?<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black">Thanks,<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black">Regards<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black">Alain<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black"><u></u> <u></u></span></p>
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">De :</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> syslog-ng [mailto:<a href="mailto:syslog-ng-bounces@lists.balabit.hu" target="_blank">syslog-ng-bounces@<wbr>lists.balabit.hu</a>]
<b>De la part de</b> Nagy, Gábor<br>
<b>Envoyé :</b> lundi 4 décembre 2017 16:30<br>
<b>À :</b> Syslog-ng users' and developers' mailing list<br>
<b>Cc :</b> RAMBERT Christophe IMT/OLS<br>
<b>Objet :</b> Re: [syslog-ng] Syslog-ng relay: how to delete or modify special character ^M?<u></u><u></u></span></p><div><div class="h5">
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal">Hi Alain!<u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">You can replace unwanted special characters by using rewrite rules is syslog-ng.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><a href="https://www.balabit.com/documents/syslog-ng-ose-latest-guides/en/syslog-ng-ose-guide-admin/html/rewrite-replace.html" target="_blank">https://www.balabit.com/<wbr>documents/syslog-ng-ose-<wbr>latest-guides/en/syslog-ng-<wbr>ose-guide-admin/html/rewrite-<wbr>replace.html</a><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Rewrite rules accept regular expressions as search pattern.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Please check regular expression options (e.g. use global flags to replace all instances).<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Example:<u></u><u></u></p>
</div>
<div>
<div>
<p class="MsoNormal"> rewrite{ subst("\r", " ", flags("global")); };<u></u><u></u></p>
</div>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal"><a href="https://www.balabit.com/documents/syslog-ng-ose-latest-guides/en/syslog-ng-ose-guide-admin/html/regular-expressions.html" target="_blank">https://www.balabit.com/<wbr>documents/syslog-ng-ose-<wbr>latest-guides/en/syslog-ng-<wbr>ose-guide-admin/html/regular-<wbr>expressions.html</a><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Best regards,<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Gabor<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal">On Mon, Dec 4, 2017 at 3:33 PM, <<a href="mailto:alain.villefranque@orange.com" target="_blank">alain.villefranque@orange.com</a><wbr>> wrote:<u></u><u></u></p>
<div>
<div>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black">Hi all,</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black"> </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black">I have an issue with the character Carriage Return (^M) inserted by the Syslog source machine.</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black">I try to substitute or rewrite this special character with syslog-ng relay, but it is impossible
to modify it, it seems this char is invisible for the program.</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black">I’ve tried to modified it with either with ^M or 0xD char with no success.</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black"> </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black">Example of received message:</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black"> </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black">Dec 4 13:35:23 conf01 2017-12-04 13:35:23,561 Level="INFO" Name="support.sip" Message="Sending
SIP response" Src-address="80.12.yy.xx" Src-port="5061" Dst-address="161.105.yy.xx" Dst-port="38509" Transport="TLS"
</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black">Detail<b>="^M</b>SIP/2.0 403 Forbidden<b>^M</b>Via: SIP/2.0/TLS 161.105.150.12:38509;alias;<wbr>branch=z9hG4bK.MEgSOM8O4;<wbr>rport=38509;received=161.105.<wbr>150.12<b>^M</b>From:
<sip:SondeSQS_001!@<a href="http://int.ovp.orange-business.com" target="_blank">int.ovp.<wbr>orange-business.com</a>>;tag=<wbr>5kYhVMAyi^MTo: sip:SondeSQS_001!@<a href="http://int.ovp.orange-business.com" target="_blank">int.ovp.<wbr>orange-business.com</a>;tag=<wbr>aynkBKUjt0pXHzNv<b>^M</b>CSeq:
25 REGISTER<b>^M</b>Call-ID: Vhihsb~BhQ<b>^M</b>Allow: INVITE,ACK,OPTIONS,CANCEL,BYE,<wbr>REGISTER,INFO,SUBSCRIBE,<wbr>NOTIFY,MESSAGE<b>^M</b>Supported: categoryList,adhoclist,sdp-<wbr>anat,replaces^MContent-Length: 0<b>^M^M</b>"</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black"> </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black">I’d like to suppress
<b>^M</b> or replace it with a “space” char.</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black"> </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black">Is there any specific action to do in order to modify ^M special character ?</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black"> </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif"">Regards
</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif""> </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Arial","sans-serif"">Alain</span><u></u><u></u></p>
<p class="MsoNormal" style="line-height:15.0pt">
<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<pre>______________________________<wbr>______________________________<wbr>______________________________<wbr>______________________________<wbr>_<u></u><u></u></pre>
<pre><u></u> <u></u></pre>
<pre>Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc<u></u><u></u></pre>
<pre>pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler<u></u><u></u></pre>
<pre>a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,<u></u><u></u></pre>
<pre>Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.<u></u><u></u></pre>
<pre><u></u> <u></u></pre>
<pre>This message and its attachments may contain confidential or privileged information that may be protected by law;<u></u><u></u></pre>
<pre>they should not be distributed, used or copied without authorisation.<u></u><u></u></pre>
<pre>If you have received this email in error, please notify the sender and delete this message and its attachments.<u></u><u></u></pre>
<pre>As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.<u></u><u></u></pre>
<pre>Thank you.<u></u><u></u></pre>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
______________________________<wbr>______________________________<wbr>__________________<br>
Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" target="_blank">
https://lists.balabit.hu/<wbr>mailman/listinfo/syslog-ng</a><br>
Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" target="_blank">
http://www.balabit.com/<wbr>support/documentation/?<wbr>product=syslog-ng</a><br>
FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" target="_blank">http://www.balabit.com/wiki/<wbr>syslog-ng-faq</a><br>
<br>
<u></u><u></u></p>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div></div></div><div><div class="h5">
<pre>______________________________<wbr>______________________________<wbr>______________________________<wbr>______________________________<wbr>_
Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.
This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.
</pre></div></div></div>
<br>______________________________<wbr>______________________________<wbr>__________________<br>
Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" rel="noreferrer" target="_blank">https://lists.balabit.hu/<wbr>mailman/listinfo/syslog-ng</a><br>
Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" rel="noreferrer" target="_blank">http://www.balabit.com/<wbr>support/documentation/?<wbr>product=syslog-ng</a><br>
FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" rel="noreferrer" target="_blank">http://www.balabit.com/wiki/<wbr>syslog-ng-faq</a><br>
<br>
<br></blockquote></div><br></div>