[syslog-ng] Syslog-ng relay: how to delete or modify special character ^M?

Wed Dec 6 12:48:40 UTC 2017

Hi Gabor,

Thank you for your deep investigation.
The suggested rewrite{ subst("\\^M<file:///\\%5eM>", ' ', flags("global"));  }; works perfectly,

Thanks again,

Regards

Alain

De : syslog-ng [mailto:syslog-ng-bounces at lists.balabit.hu] De la part de Nagy, Gábor
Envoyé : mercredi 6 décembre 2017 11:26
À : Syslog-ng users' and developers' mailing list
Cc : RAMBERT Christophe IMT/OLS
Objet : Re: [syslog-ng] Syslog-ng relay: how to delete or modify special character ^M?

Hello Alain,

Thanks for sharing the debug output, it showed me where is the problem.
It seems that you don't have the "^M" as a special character in your message but 2 regular ascii characters '^' and 'M'.
In case of a carriage-return character syslog-ng would print this debug message (check the "\x0d" strings):
[2017-12-06T10:08:57.374088] Incoming log entry; line='Dec  4 13:35:23 conf01 2017-12-04 13:35:23,561 Level="INFO" Name="support.sip" Message="Sending SIP response" Src-address="80.12.yy.xx" Src-port="5061" Dst-address="161.105.yy.xx" Dst-port="38509" Transport="TLS" Detail="\x0dMSIP/2.0 403 Forbidden\x0dMVia: SIP/2.0/TLS 161.105.150.12:38509;alias;branch=z9hG4bK.MEgSOM8O4;rport=38509;received=161.105.150.12\x0dMFrom:  <sip:SondeSQS_001!@int.ovp.orange-business.com<http://int.ovp.orange-business.com>>;tag=5kYhVMAyi\x0dTo: sip:SondeSQS_001!@int.ovp.orange-business.com<http://int.ovp.orange-business.com>;tag=aynkBKUjt0pXHzNv\x0dCSeq: 25 REGISTER\x0dCall-ID: Vhihsb~BhQ\x0dAllow: INVITE,ACK,OPTIONS,CANCEL,BYE,REGISTER,INFO,SUBSCRIBE,NOTIFY,MESSAGE\x0dSupported: categoryList,adhoclist,sdp-anat,replaces\x0dContent-Length: 0\x0d\x0d"'

You can remove these characters with subst rewrite rule, but please note that by default syslog-ng uses PCRE regular expressions, so you either escape the ^ or use literal strings as search pattern.
Also, when escaping I would recommend of the usage of single-quote instead of double-quotes, otherwise you need to double-escape special characters.
rewrite{ subst('\^M', ' ', flags("global"));  };
rewrite{ subst("\\^M<file:///\\%5eM>", ' ', flags("global"));  };
or use literal strings instead of regex
rewrite { subst("^M", " ", type("string"), flags("substring")); };

Some notes about regular expressions(in case I forgot something):
https://www.balabit.com/documents/syslog-ng-ose-latest-guides/en/syslog-ng-ose-guide-admin/html/regular-expressions.html

Best Regards,
Gabor

On Tue, Dec 5, 2017 at 2:03 PM, <alain.villefranque at orange.com<mailto:alain.villefranque at orange.com>> wrote:
Hi Gabor,

Thanks for your help.

I’ve tried in different way without success:

rewrite r_rewrite_subst_CR {
   subst('\r\n'," ",value("MESSAGE"), flags("global"));
   subst('^M'," ",value("MESSAGE"), flags("global"));
   subst("^M"," ",value("MESSAGE"), flags("global"));
   subst("\r\n"," ",value("MESSAGE"), flags("global"));
   subst('\r'," ",value("MESSAGE"), flags("global"));
   subst("\r"," ",value("MESSAGE"), flags("global"));
   subst("Detail","COUCOU",value("MESSAGE"), flags("global"));
};

You will see in the attached file that this rewrite rule has no effect on the received message specifically against the ^M character….

Do you have any other advice to provide me?

Thanks,

Regards

Alain

De : syslog-ng [mailto:syslog-ng-bounces at lists.balabit.hu<mailto:syslog-ng-bounces at lists.balabit.hu>] De la part de Nagy, Gábor
Envoyé : lundi 4 décembre 2017 16:30
À : Syslog-ng users' and developers' mailing list
Cc : RAMBERT Christophe IMT/OLS
Objet : Re: [syslog-ng] Syslog-ng relay: how to delete or modify special character ^M?

Hi Alain!

You can replace unwanted special characters by using rewrite rules is syslog-ng.
https://www.balabit.com/documents/syslog-ng-ose-latest-guides/en/syslog-ng-ose-guide-admin/html/rewrite-replace.html
Rewrite rules accept regular expressions as search pattern.
Please check regular expression options (e.g. use global flags to replace all instances).
Example:
    rewrite{ subst("\r", " ", flags("global"));  };

https://www.balabit.com/documents/syslog-ng-ose-latest-guides/en/syslog-ng-ose-guide-admin/html/regular-expressions.html

Best regards,
Gabor

On Mon, Dec 4, 2017 at 3:33 PM, <alain.villefranque at orange.com<mailto:alain.villefranque at orange.com>> wrote:
Hi all,

I have an issue with the character Carriage Return (^M) inserted by the Syslog source machine.
I try to substitute or rewrite this special character with syslog-ng relay, but it is impossible to modify it, it seems this char is invisible for the program.
I’ve tried to modified it with either with ^M or 0xD char with no success.

Example of received message:

Dec  4 13:35:23 conf01 2017-12-04 13:35:23,561 Level="INFO" Name="support.sip" Message="Sending SIP response" Src-address="80.12.yy.xx" Src-port="5061" Dst-address="161.105.yy.xx" Dst-port="38509" Transport="TLS"
Detail="^MSIP/2.0 403 Forbidden^MVia: SIP/2.0/TLS 161.105.150.12:38509;alias;branch=z9hG4bK.MEgSOM8O4;rport=38509;received=161.105.150.12^MFrom:  <sip:SondeSQS_001!@int.ovp.orange-business.com<http://int.ovp.orange-business.com>>;tag=5kYhVMAyi^MTo: sip:SondeSQS_001!@int.ovp.orange-business.com<http://int.ovp.orange-business.com>;tag=aynkBKUjt0pXHzNv^MCSeq: 25 REGISTER^MCall-ID: Vhihsb~BhQ^MAllow: INVITE,ACK,OPTIONS,CANCEL,BYE,REGISTER,INFO,SUBSCRIBE,NOTIFY,MESSAGE^MSupported: categoryList,adhoclist,sdp-anat,replaces^MContent-Length: 0^M^M"

I’d like to suppress ^M or replace it with a “space” char.

Is there any specific action to do in order to modify ^M special character ?

Regards

Alain

_________________________________________________________________________________________________________________________

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc

pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler

a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,

Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;

they should not be distributed, used or copied without authorisation.

If you have received this email in error, please notify the sender and delete this message and its attachments.

As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.

Thank you.

______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq

_________________________________________________________________________________________________________________________

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc

pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler

a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,

Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;

they should not be distributed, used or copied without authorisation.

If you have received this email in error, please notify the sender and delete this message and its attachments.

As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.

Thank you.

______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq

_________________________________________________________________________________________________________________________

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20171206/54edb94b/attachment-0001.html>