[syslog-ng] Syslog-ng relay: how to delete or modify special character ^M?

Wed Dec 6 10:26:01 UTC 2017

Hello Alain,

Thanks for sharing the debug output, it showed me where is the problem.
It seems that you don't have the "^M" as a special character in your
message but 2 regular ascii characters '^' and 'M'.
In case of a carriage-return character syslog-ng would print this debug
message (check the "\x0d" strings):

> [2017-12-06T10:08:57.374088] Incoming log entry; line='Dec  4 13:35:23
> conf01 2017-12-04 13:35:23,561 Level="INFO" Name="support.sip"
> Message="Sending SIP response" Src-address="80.12.yy.xx" Src-port="5061"
> Dst-address="161.105.yy.xx" Dst-port="38509" Transport="TLS"
> Detail="\x0dMSIP/2.0 403 Forbidden\x0dMVia: SIP/2.0/TLS 161.105.150.12:38509;alias;branch=z9hG4bK.MEgSOM8O4;rport=38509;received=161.105.150.12\x0dMFrom:
> <sip:SondeSQS_001!@int.ovp.orange-business.com>;tag=5kYhVMAyi\x0dTo:
> sip:SondeSQS_001!@int.ovp.orange-business.com;tag=aynkBKUjt0pXHzNv\x0dCSeq:
> 25 REGISTER\x0dCall-ID: Vhihsb~BhQ\x0dAllow:
> INVITE,ACK,OPTIONS,CANCEL,BYE,REGISTER,INFO,SUBSCRIBE,NOTIFY,MESSAGE\x0dSupported:
> categoryList,adhoclist,sdp-anat,replaces\x0dContent-Length: 0\x0d\x0d"'

You can remove these characters with subst rewrite rule, but please note
that by default syslog-ng uses PCRE regular expressions, so you either
escape the ^ or use literal strings as search pattern.
Also, when escaping I would recommend of the usage of single-quote instead
of double-quotes, otherwise you need to double-escape special characters.
rewrite{ subst('\^M', ' ', flags("global"));  };
rewrite{ subst("\\^M", ' ', flags("global"));  };
or use literal strings instead of regex
rewrite { subst("^M", " ", type("string"), flags("substring")); };

Some notes about regular expressions(in case I forgot something):
https://www.balabit.com/documents/syslog-ng-ose-latest-guides/en/syslog-ng-ose-guide-admin/html/regular-expressions.html

Best Regards,
Gabor

On Tue, Dec 5, 2017 at 2:03 PM, <alain.villefranque at orange.com> wrote:

> Hi Gabor,
>
>
>
> Thanks for your help.
>
>
>
> I’ve tried in different way without success:
>
>
>
> rewrite r_rewrite_subst_CR {
>
>    subst('\r\n'," ",value("MESSAGE"), flags("global"));
>
>    subst('^M'," ",value("MESSAGE"), flags("global"));
>
>    subst("^M"," ",value("MESSAGE"), flags("global"));
>
>    subst("\r\n"," ",value("MESSAGE"), flags("global"));
>
>    subst('\r'," ",value("MESSAGE"), flags("global"));
>
>    subst("\r"," ",value("MESSAGE"), flags("global"));
>
>    subst("Detail","COUCOU",value("MESSAGE"), flags("global"));
>
> };
>
>
>
> You will see in the attached file that this rewrite rule has no effect on
> the received message specifically against the ^M character….
>
>
>
> Do you have any other advice to provide me?
>
>
>
> Thanks,
>
>
>
> Regards
>
>
>
> Alain
>
>
>
>
>
> *De :* syslog-ng [mailto:syslog-ng-bounces at lists.balabit.hu] *De la part
> de* Nagy, Gábor
> *Envoyé :* lundi 4 décembre 2017 16:30
> *À :* Syslog-ng users' and developers' mailing list
> *Cc :* RAMBERT Christophe IMT/OLS
> *Objet :* Re: [syslog-ng] Syslog-ng relay: how to delete or modify
> special character ^M?
>
>
>
> Hi Alain!
>
>
>
> You can replace unwanted special characters by using rewrite rules is
> syslog-ng.
>
> https://www.balabit.com/documents/syslog-ng-ose-
> latest-guides/en/syslog-ng-ose-guide-admin/html/rewrite-replace.html
>
> Rewrite rules accept regular expressions as search pattern.
>
> Please check regular expression options (e.g. use global flags to replace
> all instances).
>
> Example:
>
>     rewrite{ subst("\r", " ", flags("global"));  };
>
>
>
> https://www.balabit.com/documents/syslog-ng-ose-
> latest-guides/en/syslog-ng-ose-guide-admin/html/regular-expressions.html
>
>
>
> Best regards,
>
> Gabor
>
>
>
>
>
> On Mon, Dec 4, 2017 at 3:33 PM, <alain.villefranque at orange.com> wrote:
>
> Hi all,
>
>
>
> I have an issue with the character Carriage Return (^M) inserted by the
> Syslog source machine.
>
> I try to substitute or rewrite this special character with syslog-ng
> relay, but it is impossible to modify it, it seems this char is invisible
> for the program.
>
> I’ve tried to modified it with either with ^M or 0xD char with no success.
>
>
>
> Example of received message:
>
>
>
> Dec  4 13:35:23 conf01 2017-12-04 13:35:23,561 Level="INFO"
> Name="support.sip" Message="Sending SIP response" Src-address="80.12.yy.xx"
> Src-port="5061" Dst-address="161.105.yy.xx" Dst-port="38509"
> Transport="TLS"
>
> Detail*="^M*SIP/2.0 403 Forbidden*^M*Via: SIP/2.0/TLS 161.105.150.12:38509
> ;alias;branch=z9hG4bK.MEgSOM8O4;rport=38509;received=161.105.150.12*^M*From:
> <sip:SondeSQS_001!@int.ovp.orange-business.com>;tag=5kYhVMAyi^MTo:
> sip:SondeSQS_001!@int.ovp.orange-business.com;tag=aynkBKUjt0pXHzNv*^M*CSeq:
> 25 REGISTER*^M*Call-ID: Vhihsb~BhQ*^M*Allow:
> INVITE,ACK,OPTIONS,CANCEL,BYE,REGISTER,INFO,SUBSCRIBE,NOTIFY,MESSAGE*^M*Supported:
> categoryList,adhoclist,sdp-anat,replaces^MContent-Length: 0*^M^M*"
>
>
>
> I’d like to suppress *^M* or replace it with a “space” char.
>
>
>
> Is there any specific action to do in order to modify ^M special character
> ?
>
>
>
> Regards
>
>
>
> Alain
>
>
>
>
>
> _________________________________________________________________________________________________________________________
>
>
>
> Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
>
> pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
>
> a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
>
> Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.
>
>
>
> This message and its attachments may contain confidential or privileged information that may be protected by law;
>
> they should not be distributed, used or copied without authorisation.
>
> If you have received this email in error, please notify the sender and delete this message and its attachments.
>
> As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
>
> Thank you.
>
>
> ____________________________________________________________
> __________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?
> product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>
>
> _________________________________________________________________________________________________________________________
>
> Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
> pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
> a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
> Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.
>
> This message and its attachments may contain confidential or privileged information that may be protected by law;
> they should not be distributed, used or copied without authorisation.
> If you have received this email in error, please notify the sender and delete this message and its attachments.
> As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
> Thank you.
>
>
> ____________________________________________________________
> __________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?
> product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20171206/534db24a/attachment-0001.html>