syslog-ng forwarding and processing issue
Hi guys, I have following setup in place: remote server - 192.168.1.10 pseudomizer - syslog-ng PE in client mode - 192.168.2.10 SIEM - 192.168.3.10 So I am sending syslog logs from remote server to pseudomizer: source src { internal()}; destination dst { udp ("192.168.2.10) port (514);}; log { source(src); destination (dst); this is very old config from syslog v4 Then on pseudomizer - syslog-ng LTS 6.0.1 PE, I am collecting the logs, processing them - removing private data, putting pseudonyms instead and forwarding them to SIEM. source s_net_udp514 { syslog( ip(192.168.2.10) ip-protocol(4) transport("udp") so_rcvbuf(2097152) ); }; source src { internal(); unix-dgram("/dev/log"); system (); }; destination d_net_udp514 { syslog ( "192.168.3.10" port(514) transport(udp) spoof_source(yes) mark_mode(periodical));}; rewrite r_rewrite { subst("admin", "pseudonym000001", value("MESSAGE"), flags("global")); log { source(s_net_udp514); source (src); rewrite(r_rewrite); # do the pseudomizing destination(d_net_udp514); }; On SIEM device, I can see only pseudomizer internal logs (src), not processed logs from remote server. Any advice ? Many thanks. Denis
Hi, The destination on your remote server and the source on the pseudomizer host do not match: the first one uses the udp() driver (RFC3164 protocol), while the second uses the syslog() driver (RFC5424) protocol. Change the destination driver to syslog() on the remote server. (For more possibilities, see https://www.balabit.com/documents/syslog-ng-pe-latest-guides/en/syslog-ng-pe... ) HTH Robert On Wed, Sep 28, 2016 at 1:17 PM, Denis Dolinský <denis.dolinsky@gmail.com> wrote:
Hi guys,
I have following setup in place: remote server - 192.168.1.10 pseudomizer - syslog-ng PE in client mode - 192.168.2.10 SIEM - 192.168.3.10
So I am sending syslog logs from remote server to pseudomizer: source src { internal()}; destination dst { udp ("192.168.2.10) port (514);}; log { source(src); destination (dst);
this is very old config from syslog v4
Then on pseudomizer - syslog-ng LTS 6.0.1 PE, I am collecting the logs, processing them - removing private data, putting pseudonyms instead and forwarding them to SIEM.
source s_net_udp514 { syslog( ip(192.168.2.10) ip-protocol(4) transport("udp") so_rcvbuf(2097152) ); };
source src { internal(); unix-dgram("/dev/log"); system (); };
destination d_net_udp514 { syslog ( "192.168.3.10" port(514) transport(udp) spoof_source(yes) mark_mode(periodical));}; rewrite r_rewrite { subst("admin", "pseudonym000001", value("MESSAGE"), flags("global"));
log { source(s_net_udp514); source (src); rewrite(r_rewrite); # do the pseudomizing destination(d_net_udp514); };
On SIEM device, I can see only pseudomizer internal logs (src), not processed logs from remote server.
Any advice ?
Many thanks.
Denis
____________________________________________________________ __________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/? product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
Hi, In case of udp, the syslog source should handle receiving logs with old and the new version too. (But that is more an exception than the rule, so matching the receiver and the sender is a good idea generaly.) Before anything else I would check if the logs arrive to the anonymizer host or not. The statistics can help on this. Also, if there are parsing issue, the syslog-ng would tell this through its log. After that starting the syslog-ng with enabled debug logs can also help on discovering what happening with the received log. From: syslog-ng-bounces@lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Fekete, Róbert Sent: Wednesday, September 28, 2016 1:47 PM To: Syslog-ng users' and developers' mailing list Subject: Re: [syslog-ng] syslog-ng forwarding and processing issue Hi, The destination on your remote server and the source on the pseudomizer host do not match: the first one uses the udp() driver (RFC3164 protocol), while the second uses the syslog() driver (RFC5424) protocol. Change the destination driver to syslog() on the remote server. (For more possibilities, see https://www.balabit.com/documents/syslog-ng-pe-latest-guides/en/syslog-ng-pe... ) HTH Robert On Wed, Sep 28, 2016 at 1:17 PM, Denis Dolinský <denis.dolinsky@gmail.com<mailto:denis.dolinsky@gmail.com>> wrote: Hi guys, I have following setup in place: remote server - 192.168.1.10 pseudomizer - syslog-ng PE in client mode - 192.168.2.10 SIEM - 192.168.3.10 So I am sending syslog logs from remote server to pseudomizer: source src { internal()}; destination dst { udp ("192.168.2.10) port (514);}; log { source(src); destination (dst); this is very old config from syslog v4 Then on pseudomizer - syslog-ng LTS 6.0.1 PE, I am collecting the logs, processing them - removing private data, putting pseudonyms instead and forwarding them to SIEM. source s_net_udp514 { syslog( ip(192.168.2.10) ip-protocol(4) transport("udp") so_rcvbuf(2097152) ); }; source src { internal(); unix-dgram("/dev/log"); system (); }; destination d_net_udp514 { syslog ( "192.168.3.10" port(514) transport(udp) spoof_source(yes) mark_mode(periodical));}; rewrite r_rewrite { subst("admin", "pseudonym000001", value("MESSAGE"), flags("global")); log { source(s_net_udp514); source (src); rewrite(r_rewrite); # do the pseudomizing destination(d_net_udp514); }; On SIEM device, I can see only pseudomizer internal logs (src), not processed logs from remote server. Any advice ? Many thanks. Denis ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq ________________________________ NOTICE: Morgan Stanley is not acting as a municipal advisor and the opinions or views contained herein are not intended to be, and do not constitute, advice within the meaning of Section 975 of the Dodd-Frank Wall Street Reform and Consumer Protection Act. If you have received this communication in error, please destroy all electronic and paper copies and notify the sender immediately. Mistransmission is not intended to waive confidentiality or privilege. Morgan Stanley reserves the right, to the extent permitted under applicable law, to monitor electronic communications. This message is subject to terms available at the following link: http://www.morganstanley.com/disclaimers If you cannot access these links, please notify us by reply message and we will send the contents to you. By communicating with Morgan Stanley you consent to the foregoing and to the voice recording of conversations with personnel of Morgan Stanley.
hi guys, this is stats: destination;d_net_udp514;;a;processed;13 source;s_net_udp514;;a;processed;3 dst.syslog;d_net_udp514#0;udp,192.168.3.1:514;a;dropped;0 dst.syslog;d_net_udp514#0;udp,192.168.3.1:514;a;processed;10 dst.syslog;d_net_udp514#0;udp,192.168.3.1:514;a;stored;0 from debug: Incoming log entry; source='s_net_udp514#0', line='<78> remote_server /usr/sbin/cron[24934]: Can't bind hostname for the IP address, therefore using IP address as hostname; IP address='192.168.2.1' Do you see anything what I do not do ? Thanks. Denis 2016-09-28 14:02 GMT+02:00 Szalai, Attila <Attila.Szalai@morganstanley.com>:
Hi,
In case of udp, the syslog source should handle receiving logs with old and the new version too. (But that is more an exception than the rule, so matching the receiver and the sender is a good idea generaly.)
Before anything else I would check if the logs arrive to the anonymizer host or not. The statistics can help on this. Also, if there are parsing issue, the syslog-ng would tell this through its log.
After that starting the syslog-ng with enabled debug logs can also help on discovering what happening with the received log.
*From:* syslog-ng-bounces@lists.balabit.hu [mailto:syslog-ng-bounces@ lists.balabit.hu] *On Behalf Of *Fekete, Róbert *Sent:* Wednesday, September 28, 2016 1:47 PM *To:* Syslog-ng users' and developers' mailing list *Subject:* Re: [syslog-ng] syslog-ng forwarding and processing issue
Hi,
The destination on your remote server and the source on the pseudomizer host do not match: the first one uses the udp() driver (RFC3164 protocol), while the second uses the syslog() driver (RFC5424) protocol.
Change the destination driver to syslog() on the remote server. (For more possibilities, see https://www.balabit.com/documents/syslog-ng-pe-latest- guides/en/syslog-ng-pe-guide-admin/html/concepts-things-to-consider.html )
HTH
Robert
On Wed, Sep 28, 2016 at 1:17 PM, Denis Dolinský <denis.dolinsky@gmail.com> wrote:
Hi guys,
I have following setup in place:
remote server - 192.168.1.10
pseudomizer - syslog-ng PE in client mode - 192.168.2.10
SIEM - 192.168.3.10
So I am sending syslog logs from remote server to pseudomizer:
source src { internal()};
destination dst { udp ("192.168.2.10) port (514);};
log { source(src); destination (dst);
this is very old config from syslog v4
Then on pseudomizer - syslog-ng LTS 6.0.1 PE, I am collecting the logs, processing them - removing private data, putting pseudonyms instead and forwarding them to SIEM.
source s_net_udp514 { syslog( ip(192.168.2.10) ip-protocol(4) transport("udp") so_rcvbuf(2097152) ); };
source src { internal(); unix-dgram("/dev/log"); system (); };
destination d_net_udp514 { syslog ( "192.168.3.10" port(514) transport(udp) spoof_source(yes) mark_mode(periodical));}; rewrite r_rewrite { subst("admin", "pseudonym000001", value("MESSAGE"), flags("global"));
log { source(s_net_udp514); source (src); rewrite(r_rewrite); # do the pseudomizing destination(d_net_udp514); };
On SIEM device, I can see only pseudomizer internal logs (src), not processed logs from remote server.
Any advice ?
Many thanks.
Denis
____________________________________________________________ __________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/? product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
------------------------------
NOTICE: Morgan Stanley is not acting as a municipal advisor and the opinions or views contained herein are not intended to be, and do not constitute, advice within the meaning of Section 975 of the Dodd-Frank Wall Street Reform and Consumer Protection Act. If you have received this communication in error, please destroy all electronic and paper copies and notify the sender immediately. Mistransmission is not intended to waive confidentiality or privilege. Morgan Stanley reserves the right, to the extent permitted under applicable law, to monitor electronic communications. This message is subject to terms available at the following link: http://www.morganstanley.com/disclaimers If you cannot access these links, please notify us by reply message and we will send the contents to you. By communicating with Morgan Stanley you consent to the foregoing and to the voice recording of conversations with personnel of Morgan Stanley.
____________________________________________________________ __________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/? product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
-- Ing. Denis Dolinský denis.dolinsky@gmail.com private cell: *+421 907 530711*
Just a quick note. The warning message about the binding issue caused by the spoof_source option. Is that option necessary? From: syslog-ng-bounces@lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Denis Dolinský Sent: Wednesday, September 28, 2016 3:47 PM To: Syslog-ng users' and developers' mailing list Subject: Re: [syslog-ng] syslog-ng forwarding and processing issue hi guys, this is stats: destination;d_net_udp514;;a;processed;13 source;s_net_udp514;;a;processed;3 dst.syslog;d_net_udp514#0;udp,192.168.3.1:514;a;dropped;0 dst.syslog;d_net_udp514#0;udp,192.168.3.1:514;a;processed;10 dst.syslog;d_net_udp514#0;udp,192.168.3.1:514;a;stored;0 from debug: Incoming log entry; source='s_net_udp514#0', line='<78> remote_server /usr/sbin/cron[24934]: Can't bind hostname for the IP address, therefore using IP address as hostname; IP address='192.168.2.1' Do you see anything what I do not do ? Thanks. Denis 2016-09-28 14:02 GMT+02:00 Szalai, Attila <Attila.Szalai@morganstanley.com<mailto:Attila.Szalai@morganstanley.com>>: Hi, In case of udp, the syslog source should handle receiving logs with old and the new version too. (But that is more an exception than the rule, so matching the receiver and the sender is a good idea generaly.) Before anything else I would check if the logs arrive to the anonymizer host or not. The statistics can help on this. Also, if there are parsing issue, the syslog-ng would tell this through its log. After that starting the syslog-ng with enabled debug logs can also help on discovering what happening with the received log. From: syslog-ng-bounces@lists.balabit.hu<mailto:syslog-ng-bounces@lists.balabit.hu> [mailto:syslog-ng-bounces@lists.balabit.hu<mailto:syslog-ng-bounces@lists.balabit.hu>] On Behalf Of Fekete, Róbert Sent: Wednesday, September 28, 2016 1:47 PM To: Syslog-ng users' and developers' mailing list Subject: Re: [syslog-ng] syslog-ng forwarding and processing issue Hi, The destination on your remote server and the source on the pseudomizer host do not match: the first one uses the udp() driver (RFC3164 protocol), while the second uses the syslog() driver (RFC5424) protocol. Change the destination driver to syslog() on the remote server. (For more possibilities, see https://www.balabit.com/documents/syslog-ng-pe-latest-guides/en/syslog-ng-pe... ) HTH Robert On Wed, Sep 28, 2016 at 1:17 PM, Denis Dolinský <denis.dolinsky@gmail.com<mailto:denis.dolinsky@gmail.com>> wrote: Hi guys, I have following setup in place: remote server - 192.168.1.10 pseudomizer - syslog-ng PE in client mode - 192.168.2.10 SIEM - 192.168.3.10 So I am sending syslog logs from remote server to pseudomizer: source src { internal()}; destination dst { udp ("192.168.2.10) port (514);}; log { source(src); destination (dst); this is very old config from syslog v4 Then on pseudomizer - syslog-ng LTS 6.0.1 PE, I am collecting the logs, processing them - removing private data, putting pseudonyms instead and forwarding them to SIEM. source s_net_udp514 { syslog( ip(192.168.2.10) ip-protocol(4) transport("udp") so_rcvbuf(2097152) ); }; source src { internal(); unix-dgram("/dev/log"); system (); }; destination d_net_udp514 { syslog ( "192.168.3.10" port(514) transport(udp) spoof_source(yes) mark_mode(periodical));}; rewrite r_rewrite { subst("admin", "pseudonym000001", value("MESSAGE"), flags("global")); log { source(s_net_udp514); source (src); rewrite(r_rewrite); # do the pseudomizing destination(d_net_udp514); }; On SIEM device, I can see only pseudomizer internal logs (src), not processed logs from remote server. Any advice ? Many thanks. Denis ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq ________________________________ NOTICE: Morgan Stanley is not acting as a municipal advisor and the opinions or views contained herein are not intended to be, and do not constitute, advice within the meaning of Section 975 of the Dodd-Frank Wall Street Reform and Consumer Protection Act. If you have received this communication in error, please destroy all electronic and paper copies and notify the sender immediately. Mistransmission is not intended to waive confidentiality or privilege. Morgan Stanley reserves the right, to the extent permitted under applicable law, to monitor electronic communications. This message is subject to terms available at the following link: http://www.morganstanley.com/disclaimers If you cannot access these links, please notify us by reply message and we will send the contents to you. By communicating with Morgan Stanley you consent to the foregoing and to the voice recording of conversations with personnel of Morgan Stanley. ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq -- Ing. Denis Dolinský denis.dolinsky@gmail.com<mailto:denis.dolinsky@gmail.com> private cell: +421 907 530711 ________________________________ NOTICE: Morgan Stanley is not acting as a municipal advisor and the opinions or views contained herein are not intended to be, and do not constitute, advice within the meaning of Section 975 of the Dodd-Frank Wall Street Reform and Consumer Protection Act. If you have received this communication in error, please destroy all electronic and paper copies and notify the sender immediately. Mistransmission is not intended to waive confidentiality or privilege. Morgan Stanley reserves the right, to the extent permitted under applicable law, to monitor electronic communications. This message is subject to terms available at the following link: http://www.morganstanley.com/disclaimers If you cannot access these links, please notify us by reply message and we will send the contents to you. By communicating with Morgan Stanley you consent to the foregoing and to the voice recording of conversations with personnel of Morgan Stanley.
Hi, yes, I need spoof_source to be enabled for source identification ... Denis 2016-09-28 16:44 GMT+02:00 Szalai, Attila <Attila.Szalai@morganstanley.com>:
Just a quick note.
The warning message about the binding issue caused by the spoof_source option. Is that option necessary?
*From:* syslog-ng-bounces@lists.balabit.hu [mailto:syslog-ng-bounces@ lists.balabit.hu] *On Behalf Of *Denis Dolinský *Sent:* Wednesday, September 28, 2016 3:47 PM
*To:* Syslog-ng users' and developers' mailing list *Subject:* Re: [syslog-ng] syslog-ng forwarding and processing issue
hi guys,
this is stats:
destination;d_net_udp514;;a;processed;13 source;s_net_udp514;;a;processed;3 dst.syslog;d_net_udp514#0;udp,192.168.3.1:514;a;dropped;0 dst.syslog;d_net_udp514#0;udp,192.168.3.1:514;a;processed;10 dst.syslog;d_net_udp514#0;udp,192.168.3.1:514;a;stored;0
from debug:
Incoming log entry; source='s_net_udp514#0', line='<78> remote_server /usr/sbin/cron[24934]: Can't bind hostname for the IP address, therefore using IP address as hostname; IP address='192.168.2.1'
Do you see anything what I do not do ?
Thanks.
Denis
2016-09-28 14:02 GMT+02:00 Szalai, Attila <Attila.Szalai@morganstanley.com
:
Hi,
In case of udp, the syslog source should handle receiving logs with old and the new version too. (But that is more an exception than the rule, so matching the receiver and the sender is a good idea generaly.)
Before anything else I would check if the logs arrive to the anonymizer host or not. The statistics can help on this. Also, if there are parsing issue, the syslog-ng would tell this through its log.
After that starting the syslog-ng with enabled debug logs can also help on discovering what happening with the received log.
*From:* syslog-ng-bounces@lists.balabit.hu [mailto:syslog-ng-bounces@ lists.balabit.hu] *On Behalf Of *Fekete, Róbert *Sent:* Wednesday, September 28, 2016 1:47 PM *To:* Syslog-ng users' and developers' mailing list *Subject:* Re: [syslog-ng] syslog-ng forwarding and processing issue
Hi,
The destination on your remote server and the source on the pseudomizer host do not match: the first one uses the udp() driver (RFC3164 protocol), while the second uses the syslog() driver (RFC5424) protocol.
Change the destination driver to syslog() on the remote server. (For more possibilities, see https://www.balabit.com/documents/syslog-ng-pe-latest- guides/en/syslog-ng-pe-guide-admin/html/concepts-things-to-consider.html )
HTH
Robert
On Wed, Sep 28, 2016 at 1:17 PM, Denis Dolinský <denis.dolinsky@gmail.com> wrote:
Hi guys,
I have following setup in place:
remote server - 192.168.1.10
pseudomizer - syslog-ng PE in client mode - 192.168.2.10
SIEM - 192.168.3.10
So I am sending syslog logs from remote server to pseudomizer:
source src { internal()};
destination dst { udp ("192.168.2.10) port (514);};
log { source(src); destination (dst);
this is very old config from syslog v4
Then on pseudomizer - syslog-ng LTS 6.0.1 PE, I am collecting the logs, processing them - removing private data, putting pseudonyms instead and forwarding them to SIEM.
source s_net_udp514 { syslog( ip(192.168.2.10) ip-protocol(4) transport("udp") so_rcvbuf(2097152) ); };
source src { internal(); unix-dgram("/dev/log"); system (); };
destination d_net_udp514 { syslog ( "192.168.3.10" port(514) transport(udp) spoof_source(yes) mark_mode(periodical));}; rewrite r_rewrite { subst("admin", "pseudonym000001", value("MESSAGE"), flags("global"));
log { source(s_net_udp514); source (src); rewrite(r_rewrite); # do the pseudomizing destination(d_net_udp514); };
On SIEM device, I can see only pseudomizer internal logs (src), not processed logs from remote server.
Any advice ?
Many thanks.
Denis
____________________________________________________________ __________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/? product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
------------------------------
NOTICE: Morgan Stanley is not acting as a municipal advisor and the opinions or views contained herein are not intended to be, and do not constitute, advice within the meaning of Section 975 of the Dodd-Frank Wall Street Reform and Consumer Protection Act. If you have received this communication in error, please destroy all electronic and paper copies and notify the sender immediately. Mistransmission is not intended to waive confidentiality or privilege. Morgan Stanley reserves the right, to the extent permitted under applicable law, to monitor electronic communications. This message is subject to terms available at the following link: http://www.morganstanley.com/disclaimers If you cannot access these links, please notify us by reply message and we will send the contents to you. By communicating with Morgan Stanley you consent to the foregoing and to the voice recording of conversations with personnel of Morgan Stanley.
____________________________________________________________ __________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/? product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
--
Ing. Denis Dolinský denis.dolinsky@gmail.com private cell: *+421 907 530711 <%2B421%20907%20530711>*
------------------------------
NOTICE: Morgan Stanley is not acting as a municipal advisor and the opinions or views contained herein are not intended to be, and do not constitute, advice within the meaning of Section 975 of the Dodd-Frank Wall Street Reform and Consumer Protection Act. If you have received this communication in error, please destroy all electronic and paper copies and notify the sender immediately. Mistransmission is not intended to waive confidentiality or privilege. Morgan Stanley reserves the right, to the extent permitted under applicable law, to monitor electronic communications. This message is subject to terms available at the following link: http://www.morganstanley.com/disclaimers If you cannot access these links, please notify us by reply message and we will send the contents to you. By communicating with Morgan Stanley you consent to the foregoing and to the voice recording of conversations with personnel of Morgan Stanley.
____________________________________________________________ __________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/? product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
-- Ing. Denis Dolinský denis.dolinsky@gmail.com private cell: *+421 907 530711*
Hi, Source spoofing fakes the source IP address of the outgoing packets, as this can't work with connection-oriented protocols it is usable only with UDP datagrams. Actually source spoofing isn't needed in most cases and won't even work when spoofing protection is enabled in firewalls / routers and not all hosts are on the same subnet. Take a look at keep-hostname() and chain-hostnames() Regards, Sandor On 09/29/2016 09:29 AM, Denis Dolinský wrote:
Hi,
yes, I need spoof_source to be enabled for source identification ...
Denis
2016-09-28 16:44 GMT+02:00 Szalai, Attila <Attila.Szalai@morganstanley.com <mailto:Attila.Szalai@morganstanley.com>>:
Just a quick note.____
The warning message about the binding issue caused by the spoof_source option. Is that option necessary?____
__ __
*From:*syslog-ng-bounces@lists.balabit.hu <mailto:syslog-ng-bounces@lists.balabit.hu> [mailto:syslog-ng-bounces@lists.balabit.hu <mailto:syslog-ng-bounces@lists.balabit.hu>] *On Behalf Of *Denis Dolinský *Sent:* Wednesday, September 28, 2016 3:47 PM
*To:* Syslog-ng users' and developers' mailing list *Subject:* Re: [syslog-ng] syslog-ng forwarding and processing issue____
__ __
hi guys,____
__ __
this is stats:____
__ __
destination;d_net_udp514;;a;processed;13 source;s_net_udp514;;a;processed;3 dst.syslog;d_net_udp514#0;udp,192.168.3.1:514;a;dropped;0 dst.syslog;d_net_udp514#0;udp,192.168.3.1:514;a;processed;10 dst.syslog;d_net_udp514#0;udp,192.168.3.1:514;a;stored;0____
__ __
from debug:____
__ __
Incoming log entry; source='s_net_udp514#0', line='<78> remote_server /usr/sbin/cron[24934]: Can't bind hostname for the IP address, therefore using IP address as hostname; IP address='192.168.2.1'____
__ __
Do you see anything what I do not do ?____
__ __
Thanks.____
__ __
Denis____
__ __
2016-09-28 14:02 GMT+02:00 Szalai, Attila <Attila.Szalai@morganstanley.com <mailto:Attila.Szalai@morganstanley.com>>:____
Hi,____
____
In case of udp, the syslog source should handle receiving logs with old and the new version too. (But that is more an exception than the rule, so matching the receiver and the sender is a good idea generaly.)____
____
Before anything else I would check if the logs arrive to the anonymizer host or not. The statistics can help on this. Also, if there are parsing issue, the syslog-ng would tell this through its log.____
____
After that starting the syslog-ng with enabled debug logs can also help on discovering what happening with the received log.____
____
*From:*syslog-ng-bounces@lists.balabit.hu <mailto:syslog-ng-bounces@lists.balabit.hu> [mailto:syslog-ng-bounces@lists.balabit.hu <mailto:syslog-ng-bounces@lists.balabit.hu>] *On Behalf Of *Fekete, Róbert *Sent:* Wednesday, September 28, 2016 1:47 PM *To:* Syslog-ng users' and developers' mailing list *Subject:* Re: [syslog-ng] syslog-ng forwarding and processing issue____
____
Hi, ____
____
The destination on your remote server and the source on the pseudomizer host do not match: the first one uses the udp() driver (RFC3164 protocol), while the second uses the syslog() driver (RFC5424) protocol. ____
____
Change the destination driver to syslog() on the remote server. (For more possibilities, see https://www.balabit.com/documents/syslog-ng-pe-latest-guides/en/syslog-ng-pe... <https://www.balabit.com/documents/syslog-ng-pe-latest-guides/en/syslog-ng-pe-guide-admin/html/concepts-things-to-consider.html> )____
____
HTH____
____
Robert____
____
On Wed, Sep 28, 2016 at 1:17 PM, Denis Dolinský <denis.dolinsky@gmail.com <mailto:denis.dolinsky@gmail.com>> wrote:____
Hi guys,____
I have following setup in place:____
remote server - 192.168.1.10____
pseudomizer - syslog-ng PE in client mode - 192.168.2.10____
SIEM - 192.168.3.10____
So I am sending syslog logs from remote server to pseudomizer:____
source src { internal()};____
destination dst { udp ("192.168.2.10) port (514);};____
log { source(src); destination (dst);____
this is very old config from syslog v4____
Then on pseudomizer - syslog-ng LTS 6.0.1 PE, I am collecting the logs, processing them - removing private data, putting pseudonyms instead and forwarding them to SIEM.
source s_net_udp514 { syslog( ip(192.168.2.10) ip-protocol(4) transport("udp") so_rcvbuf(2097152) ); };
source src { internal(); unix-dgram("/dev/log"); system (); };
destination d_net_udp514 { syslog ( "192.168.3.10" port(514) transport(udp) spoof_source(yes) mark_mode(periodical));}; rewrite r_rewrite { subst("admin", "pseudonym000001", value("MESSAGE"), flags("global"));
log { source(s_net_udp514); source (src); rewrite(r_rewrite); # do the pseudomizing destination(d_net_udp514); };____
On SIEM device, I can see only pseudomizer internal logs (src), not processed logs from remote server.____
Any advice ?____
Many thanks.____
Denis____
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng <https://lists.balabit.hu/mailman/listinfo/syslog-ng> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng <http://www.balabit.com/support/documentation/?product=syslog-ng> FAQ: http://www.balabit.com/wiki/syslog-ng-faq <http://www.balabit.com/wiki/syslog-ng-faq>____
____
__ __
------------------------------------------------------------------------
NOTICE: Morgan Stanley is not acting as a municipal advisor and the opinions or views contained herein are not intended to be, and do not constitute, advice within the meaning of Section 975 of the Dodd-Frank Wall Street Reform and Consumer Protection Act. If you have received this communication in error, please destroy all electronic and paper copies and notify the sender immediately. Mistransmission is not intended to waive confidentiality or privilege. Morgan Stanley reserves the right, to the extent permitted under applicable law, to monitor electronic communications. This message is subject to terms available at the following link: http://www.morganstanley.com/disclaimers <http://www.morganstanley.com/disclaimers> If you cannot access these links, please notify us by reply message and we will send the contents to you. By communicating with Morgan Stanley you consent to the foregoing and to the voice recording of conversations with personnel of Morgan Stanley.____
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng <https://lists.balabit.hu/mailman/listinfo/syslog-ng> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng <http://www.balabit.com/support/documentation/?product=syslog-ng> FAQ: http://www.balabit.com/wiki/syslog-ng-faq <http://www.balabit.com/wiki/syslog-ng-faq>
____
-- ____
Ing. Denis Dolinský denis.dolinsky@gmail.com <mailto:denis.dolinsky@gmail.com> private cell: _+421 907 530711 <tel:%2B421%20907%20530711>_____
------------------------------------------------------------------------
NOTICE: Morgan Stanley is not acting as a municipal advisor and the opinions or views contained herein are not intended to be, and do not constitute, advice within the meaning of Section 975 of the Dodd-Frank Wall Street Reform and Consumer Protection Act. If you have received this communication in error, please destroy all electronic and paper copies and notify the sender immediately. Mistransmission is not intended to waive confidentiality or privilege. Morgan Stanley reserves the right, to the extent permitted under applicable law, to monitor electronic communications. This message is subject to terms available at the following link: http://www.morganstanley.com/disclaimers <http://www.morganstanley.com/disclaimers> If you cannot access these links, please notify us by reply message and we will send the contents to you. By communicating with Morgan Stanley you consent to the foregoing and to the voice recording of conversations with personnel of Morgan Stanley.
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng <https://lists.balabit.hu/mailman/listinfo/syslog-ng> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng <http://www.balabit.com/support/documentation/?product=syslog-ng> FAQ: http://www.balabit.com/wiki/syslog-ng-faq <http://www.balabit.com/wiki/syslog-ng-faq>
-- Ing. Denis Dolinský denis.dolinsky@gmail.com <mailto:denis.dolinsky@gmail.com> private cell: _+421 907 530711_
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
hi Sandor, these are my global options: # # Global options. # options { chain_hostnames(yes); keep_hostname(yes); keep_timestamp(yes); flush_lines(0); perm(0640); stats_freq(3600); }; so spoof source is not necessary here ? Thanks. Denis 2016-09-29 9:49 GMT+02:00 Sandor Geller <sandor.geller@ericsson.com>:
Hi,
Source spoofing fakes the source IP address of the outgoing packets, as this can't work with connection-oriented protocols it is usable only with UDP datagrams.
Actually source spoofing isn't needed in most cases and won't even work when spoofing protection is enabled in firewalls / routers and not all hosts are on the same subnet.
Take a look at keep-hostname() and chain-hostnames()
Regards,
Sandor
On 09/29/2016 09:29 AM, Denis Dolinský wrote:
Hi,
yes, I need spoof_source to be enabled for source identification ...
Denis
2016-09-28 16:44 GMT+02:00 Szalai, Attila <Attila.Szalai@morganstanley.com <mailto:Attila.Szalai@morganstanley.com
:
Just a quick note.____
The warning message about the binding issue caused by the spoof_source option. Is that option necessary?____
__ __
*From:*syslog-ng-bounces@lists.balabit.hu <mailto:syslog-ng-bounces@lists.balabit.hu> [mailto:syslog-ng-bounces@lists.balabit.hu <mailto:syslog-ng-bounces@lists.balabit.hu>] *On Behalf Of *Denis Dolinský *Sent:* Wednesday, September 28, 2016 3:47 PM
*To:* Syslog-ng users' and developers' mailing list *Subject:* Re: [syslog-ng] syslog-ng forwarding and processing issue____
__ __
hi guys,____
__ __
this is stats:____
__ __
destination;d_net_udp514;;a;processed;13 source;s_net_udp514;;a;processed;3 dst.syslog;d_net_udp514#0;udp,192.168.3.1:514;a;dropped;0 dst.syslog;d_net_udp514#0;udp,192.168.3.1:514;a;processed;10 dst.syslog;d_net_udp514#0;udp,192.168.3.1:514;a;stored;0____
__ __
from debug:____
__ __
Incoming log entry; source='s_net_udp514#0', line='<78> remote_server /usr/sbin/cron[24934]: Can't bind hostname for the IP address, therefore using IP address as hostname; IP address='192.168.2.1'____
__ __
Do you see anything what I do not do ?____
__ __
Thanks.____
__ __
Denis____
__ __
2016-09-28 14:02 GMT+02:00 Szalai, Attila <Attila.Szalai@morganstanley.com <mailto:Attila.Szalai@morganstanley.com>>:____
Hi,____
____
In case of udp, the syslog source should handle receiving logs with old and the new version too. (But that is more an exception than the rule, so matching the receiver and the sender is a good idea generaly.)____
____
Before anything else I would check if the logs arrive to the anonymizer host or not. The statistics can help on this. Also, if there are parsing issue, the syslog-ng would tell this through its log.____
____
After that starting the syslog-ng with enabled debug logs can also help on discovering what happening with the received log.____
____
*From:*syslog-ng-bounces@lists.balabit.hu <mailto:syslog-ng-bounces@lists.balabit.hu> [mailto:syslog-ng-bounces@lists.balabit.hu <mailto:syslog-ng-bounces@lists.balabit.hu>] *On Behalf Of *Fekete, Róbert *Sent:* Wednesday, September 28, 2016 1:47 PM *To:* Syslog-ng users' and developers' mailing list *Subject:* Re: [syslog-ng] syslog-ng forwarding and processing issue____
____
Hi, ____
____
The destination on your remote server and the source on the pseudomizer host do not match: the first one uses the udp() driver (RFC3164 protocol), while the second uses the syslog() driver (RFC5424) protocol. ____
____
Change the destination driver to syslog() on the remote server. (For more possibilities, see https://www.balabit.com/documents/syslog-ng-pe-latest- guides/en/syslog-ng-pe-guide-admin/html/concepts-things-to-consider.html <https://www.balabit.com/documents/syslog-ng-pe-latest- guides/en/syslog-ng-pe-guide-admin/html/concepts-things-to-consider.html> )____
____
HTH____
____
Robert____
____
On Wed, Sep 28, 2016 at 1:17 PM, Denis Dolinský <denis.dolinsky@gmail.com <mailto:denis.dolinsky@gmail.com>> wrote:____
Hi guys,____
I have following setup in place:____
remote server - 192.168.1.10____
pseudomizer - syslog-ng PE in client mode - 192.168.2.10____
SIEM - 192.168.3.10____
So I am sending syslog logs from remote server to pseudomizer:____
source src { internal()};____
destination dst { udp ("192.168.2.10) port (514);};____
log { source(src); destination (dst);____
this is very old config from syslog v4____
Then on pseudomizer - syslog-ng LTS 6.0.1 PE, I am collecting the logs, processing them - removing private data, putting pseudonyms instead and forwarding them to SIEM.
source s_net_udp514 { syslog( ip(192.168.2.10) ip-protocol(4) transport("udp") so_rcvbuf(2097152) ); };
source src { internal(); unix-dgram("/dev/log"); system (); };
destination d_net_udp514 { syslog ( "192.168.3.10" port(514) transport(udp) spoof_source(yes) mark_mode(periodical));}; rewrite r_rewrite { subst("admin", "pseudonym000001", value("MESSAGE"), flags("global"));
log { source(s_net_udp514); source (src); rewrite(r_rewrite); # do the pseudomizing destination(d_net_udp514); };____
On SIEM device, I can see only pseudomizer internal logs (src), not processed logs from remote server.____
Any advice ?____
Many thanks.____
Denis____
____________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng <https://lists.balabit.hu/mailman/listinfo/syslog-ng> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng <http://www.balabit.com/support/documentation/?product=syslog-ng> FAQ: http://www.balabit.com/wiki/syslog-ng-faq <http://www.balabit.com/wiki/syslog-ng-faq>____
____
__ __
------------------------------------------------------------
------------
NOTICE: Morgan Stanley is not acting as a municipal advisor and the opinions or views contained herein are not intended to be, and do not constitute, advice within the meaning of Section 975 of the Dodd-Frank Wall Street Reform and Consumer Protection Act. If you have received this communication in error, please destroy all electronic and paper copies and notify the sender immediately. Mistransmission is not intended to waive confidentiality or privilege. Morgan Stanley reserves the right, to the extent permitted under applicable law, to monitor electronic communications. This message is subject to terms available at the following link: http://www.morganstanley.com/disclaimers <http://www.morganstanley.com/disclaimers> If you cannot access these links, please notify us by reply message and we will send the contents to you. By communicating with Morgan Stanley you consent to the foregoing and to the voice recording of conversations with personnel of Morgan Stanley.____
____________________________________________________________
__________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng <https://lists.balabit.hu/mailman/listinfo/syslog-ng> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng <http://www.balabit.com/support/documentation/?product=syslog-ng> FAQ: http://www.balabit.com/wiki/syslog-ng-faq <http://www.balabit.com/wiki/syslog-ng-faq>
____
-- ____
Ing. Denis Dolinský denis.dolinsky@gmail.com <mailto:denis.dolinsky@gmail.com> private cell: _+421 907 530711 <tel:%2B421%20907%20530711>_____
------------------------------------------------------------
------------
NOTICE: Morgan Stanley is not acting as a municipal advisor and the opinions or views contained herein are not intended to be, and do not constitute, advice within the meaning of Section 975 of the Dodd-Frank Wall Street Reform and Consumer Protection Act. If you have received this communication in error, please destroy all electronic and paper copies and notify the sender immediately. Mistransmission is not intended to waive confidentiality or privilege. Morgan Stanley reserves the right, to the extent permitted under applicable law, to monitor electronic communications. This message is subject to terms available at the following link: http://www.morganstanley.com/disclaimers <http://www.morganstanley.com/disclaimers> If you cannot access these links, please notify us by reply message and we will send the contents to you. By communicating with Morgan Stanley you consent to the foregoing and to the voice recording of conversations with personnel of Morgan Stanley.
____________________________________________________________
__________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng <https://lists.balabit.hu/mailman/listinfo/syslog-ng> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng <http://www.balabit.com/support/documentation/?product=syslog-ng> FAQ: http://www.balabit.com/wiki/syslog-ng-faq <http://www.balabit.com/wiki/syslog-ng-faq>
-- Ing. Denis Dolinský denis.dolinsky@gmail.com <mailto:denis.dolinsky@gmail.com> private cell: _+421 907 530711_
____________________________________________________________
__________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/? product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
____________________________________________________________ __________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/? product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
-- Ing. Denis Dolinský denis.dolinsky@gmail.com private cell: *+421 907 530711*
Source spoofing is the remnant of a world where devices didn't have tcp based logging, and were broken enough not to recognise the source hostname out of a syslog message. The world us supposed to have changed and I would drop spoof source any time if it weren't for backward compatibility. Using spoof source and hence UDP is not a good way to deploy a serious log management system in 2016. On Sep 29, 2016 09:09, "Denis Dolinský" <denis.dolinsky@gmail.com> wrote:
hi Sandor,
these are my global options:
# # Global options. # options { chain_hostnames(yes); keep_hostname(yes); keep_timestamp(yes); flush_lines(0); perm(0640); stats_freq(3600); };
so spoof source is not necessary here ?
Thanks.
Denis
2016-09-29 9:49 GMT+02:00 Sandor Geller <sandor.geller@ericsson.com>:
Hi,
Source spoofing fakes the source IP address of the outgoing packets, as this can't work with connection-oriented protocols it is usable only with UDP datagrams.
Actually source spoofing isn't needed in most cases and won't even work when spoofing protection is enabled in firewalls / routers and not all hosts are on the same subnet.
Take a look at keep-hostname() and chain-hostnames()
Regards,
Sandor
On 09/29/2016 09:29 AM, Denis Dolinský wrote:
Hi,
yes, I need spoof_source to be enabled for source identification ...
Denis
2016-09-28 16:44 GMT+02:00 Szalai, Attila <Attila.Szalai@morganstanley.com <mailto:Attila.Szalai@morganst anley.com>>:
Just a quick note.____
The warning message about the binding issue caused by the spoof_source option. Is that option necessary?____
__ __
*From:*syslog-ng-bounces@lists.balabit.hu <mailto:syslog-ng-bounces@lists.balabit.hu> [mailto:syslog-ng-bounces@lists.balabit.hu <mailto:syslog-ng-bounces@lists.balabit.hu>] *On Behalf Of *Denis Dolinský *Sent:* Wednesday, September 28, 2016 3:47 PM
*To:* Syslog-ng users' and developers' mailing list *Subject:* Re: [syslog-ng] syslog-ng forwarding and processing issue____
__ __
hi guys,____
__ __
this is stats:____
__ __
destination;d_net_udp514;;a;processed;13 source;s_net_udp514;;a;processed;3 dst.syslog;d_net_udp514#0;udp,192.168.3.1:514;a;dropped;0 dst.syslog;d_net_udp514#0;udp,192.168.3.1:514;a;processed;10 dst.syslog;d_net_udp514#0;udp,192.168.3.1:514;a;stored;0____
__ __
from debug:____
__ __
Incoming log entry; source='s_net_udp514#0', line='<78> remote_server /usr/sbin/cron[24934]: Can't bind hostname for the IP address, therefore using IP address as hostname; IP address='192.168.2.1'____
__ __
Do you see anything what I do not do ?____
__ __
Thanks.____
__ __
Denis____
__ __
2016-09-28 14:02 GMT+02:00 Szalai, Attila <Attila.Szalai@morganstanley.com <mailto:Attila.Szalai@morganstanley.com>>:____
Hi,____
____
In case of udp, the syslog source should handle receiving logs with old and the new version too. (But that is more an exception than the rule, so matching the receiver and the sender is a good idea generaly.)____
____
Before anything else I would check if the logs arrive to the anonymizer host or not. The statistics can help on this. Also, if there are parsing issue, the syslog-ng would tell this through its log.____
____
After that starting the syslog-ng with enabled debug logs can also help on discovering what happening with the received log.____
____
*From:*syslog-ng-bounces@lists.balabit.hu <mailto:syslog-ng-bounces@lists.balabit.hu> [mailto:syslog-ng-bounces@lists.balabit.hu <mailto:syslog-ng-bounces@lists.balabit.hu>] *On Behalf Of *Fekete, Róbert *Sent:* Wednesday, September 28, 2016 1:47 PM *To:* Syslog-ng users' and developers' mailing list *Subject:* Re: [syslog-ng] syslog-ng forwarding and processing issue____
____
Hi, ____
____
The destination on your remote server and the source on the pseudomizer host do not match: the first one uses the udp() driver (RFC3164 protocol), while the second uses the syslog() driver (RFC5424) protocol. ____
____
Change the destination driver to syslog() on the remote server. (For more possibilities, see https://www.balabit.com/documents/syslog-ng-pe-latest-guide s/en/syslog-ng-pe-guide-admin/html/concepts-things-to-consider.html <https://www.balabit.com/documents/syslog-ng-pe-latest-guid es/en/syslog-ng-pe-guide-admin/html/concepts-things-to-consider.html> )____
____
HTH____
____
Robert____
____
On Wed, Sep 28, 2016 at 1:17 PM, Denis Dolinský <denis.dolinsky@gmail.com <mailto:denis.dolinsky@gmail.com>> wrote:____
Hi guys,____
I have following setup in place:____
remote server - 192.168.1.10____
pseudomizer - syslog-ng PE in client mode - 192.168.2.10____
SIEM - 192.168.3.10____
So I am sending syslog logs from remote server to pseudomizer:____
source src { internal()};____
destination dst { udp ("192.168.2.10) port (514);};____
log { source(src); destination (dst);____
this is very old config from syslog v4____
Then on pseudomizer - syslog-ng LTS 6.0.1 PE, I am collecting the logs, processing them - removing private data, putting pseudonyms instead and forwarding them to SIEM.
source s_net_udp514 { syslog( ip(192.168.2.10) ip-protocol(4) transport("udp") so_rcvbuf(2097152) ); };
source src { internal(); unix-dgram("/dev/log"); system (); };
destination d_net_udp514 { syslog ( "192.168.3.10" port(514) transport(udp) spoof_source(yes) mark_mode(periodical));}; rewrite r_rewrite { subst("admin", "pseudonym000001", value("MESSAGE"), flags("global"));
log { source(s_net_udp514); source (src); rewrite(r_rewrite); # do the pseudomizing destination(d_net_udp514); };____
On SIEM device, I can see only pseudomizer internal logs (src), not processed logs from remote server.____
Any advice ?____
Many thanks.____
Denis____
___________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng <https://lists.balabit.hu/mailman/listinfo/syslog-ng> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng <http://www.balabit.com/support/documentation/?product=syslog-ng> FAQ: http://www.balabit.com/wiki/syslog-ng-faq <http://www.balabit.com/wiki/syslog-ng-faq>____
____
__ __
-----------------------------------------------------------
-------------
NOTICE: Morgan Stanley is not acting as a municipal advisor and the opinions or views contained herein are not intended to be, and do not constitute, advice within the meaning of Section 975 of the Dodd-Frank Wall Street Reform and Consumer Protection Act. If you have received this communication in error, please destroy all electronic and paper copies and notify the sender immediately. Mistransmission is not intended to waive confidentiality or privilege. Morgan Stanley reserves the right, to the extent permitted under applicable law, to monitor electronic communications. This message is subject to terms available at the following link: http://www.morganstanley.com/disclaimers <http://www.morganstanley.com/disclaimers> If you cannot access these links, please notify us by reply message and we will send the contents to you. By communicating with Morgan Stanley you consent to the foregoing and to the voice recording of conversations with personnel of Morgan Stanley.____
___________________________________________________________
___________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng <https://lists.balabit.hu/mailman/listinfo/syslog-ng> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng <http://www.balabit.com/support/documentation/?product=syslog-ng> FAQ: http://www.balabit.com/wiki/syslog-ng-faq <http://www.balabit.com/wiki/syslog-ng-faq>
____
-- ____
Ing. Denis Dolinský denis.dolinsky@gmail.com <mailto:denis.dolinsky@gmail.com> private cell: _+421 907 530711 <tel:%2B421%20907%20530711>_____
-----------------------------------------------------------
-------------
NOTICE: Morgan Stanley is not acting as a municipal advisor and the opinions or views contained herein are not intended to be, and do not constitute, advice within the meaning of Section 975 of the Dodd-Frank Wall Street Reform and Consumer Protection Act. If you have received this communication in error, please destroy all electronic and paper copies and notify the sender immediately. Mistransmission is not intended to waive confidentiality or privilege. Morgan Stanley reserves the right, to the extent permitted under applicable law, to monitor electronic communications. This message is subject to terms available at the following link: http://www.morganstanley.com/disclaimers <http://www.morganstanley.com/disclaimers> If you cannot access these links, please notify us by reply message and we will send the contents to you. By communicating with Morgan Stanley you consent to the foregoing and to the voice recording of conversations with personnel of Morgan Stanley.
___________________________________________________________
___________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng <https://lists.balabit.hu/mailman/listinfo/syslog-ng> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng <http://www.balabit.com/support/documentation/?product=syslog-ng> FAQ: http://www.balabit.com/wiki/syslog-ng-faq <http://www.balabit.com/wiki/syslog-ng-faq>
-- Ing. Denis Dolinský denis.dolinsky@gmail.com <mailto:denis.dolinsky@gmail.com> private cell: _+421 907 530711_
____________________________________________________________
__________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product= syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
____________________________________________________________ __________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product= syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
-- Ing. Denis Dolinský denis.dolinsky@gmail.com private cell: *+421 907 530711 <%2B421%20907%20530711>*
____________________________________________________________ __________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/? product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
Hi Balazs, I totally agree with you. New log management concept I am going to produce shortly, I just wanted this od config to do the job till I will not migrate it to new solution. So, in new solution, what kind of transport should I consider - TCP ? From source to log processor and then to SIEM ? I don't want to use DNS, I want to keep hostnames, encryption is not required - maybe I would consider it in log processor to SIEM traffic. Thanks. Denis 2016-09-29 17:38 GMT+02:00 Scheidler, Balázs <balazs.scheidler@balabit.com>:
Source spoofing is the remnant of a world where devices didn't have tcp based logging, and were broken enough not to recognise the source hostname out of a syslog message.
The world us supposed to have changed and I would drop spoof source any time if it weren't for backward compatibility.
Using spoof source and hence UDP is not a good way to deploy a serious log management system in 2016.
On Sep 29, 2016 09:09, "Denis Dolinský" <denis.dolinsky@gmail.com> wrote:
hi Sandor,
these are my global options:
# # Global options. # options { chain_hostnames(yes); keep_hostname(yes); keep_timestamp(yes); flush_lines(0); perm(0640); stats_freq(3600); };
so spoof source is not necessary here ?
Thanks.
Denis
2016-09-29 9:49 GMT+02:00 Sandor Geller <sandor.geller@ericsson.com>:
Hi,
Source spoofing fakes the source IP address of the outgoing packets, as this can't work with connection-oriented protocols it is usable only with UDP datagrams.
Actually source spoofing isn't needed in most cases and won't even work when spoofing protection is enabled in firewalls / routers and not all hosts are on the same subnet.
Take a look at keep-hostname() and chain-hostnames()
Regards,
Sandor
On 09/29/2016 09:29 AM, Denis Dolinský wrote:
Hi,
yes, I need spoof_source to be enabled for source identification ...
Denis
2016-09-28 16:44 GMT+02:00 Szalai, Attila <Attila.Szalai@morganstanley.com <mailto:Attila.Szalai@morganst anley.com>>:
Just a quick note.____
The warning message about the binding issue caused by the spoof_source option. Is that option necessary?____
__ __
*From:*syslog-ng-bounces@lists.balabit.hu <mailto:syslog-ng-bounces@lists.balabit.hu> [mailto:syslog-ng-bounces@lists.balabit.hu <mailto:syslog-ng-bounces@lists.balabit.hu>] *On Behalf Of *Denis Dolinský *Sent:* Wednesday, September 28, 2016 3:47 PM
*To:* Syslog-ng users' and developers' mailing list *Subject:* Re: [syslog-ng] syslog-ng forwarding and processing issue____
__ __
hi guys,____
__ __
this is stats:____
__ __
destination;d_net_udp514;;a;processed;13 source;s_net_udp514;;a;processed;3 dst.syslog;d_net_udp514#0;udp,192.168.3.1:514;a;dropped;0 dst.syslog;d_net_udp514#0;udp,192.168.3.1:514;a;processed;10 dst.syslog;d_net_udp514#0;udp,192.168.3.1:514;a;stored;0____
__ __
from debug:____
__ __
Incoming log entry; source='s_net_udp514#0', line='<78> remote_server /usr/sbin/cron[24934]: Can't bind hostname for the IP address, therefore using IP address as hostname; IP address='192.168.2.1'____
__ __
Do you see anything what I do not do ?____
__ __
Thanks.____
__ __
Denis____
__ __
2016-09-28 14:02 GMT+02:00 Szalai, Attila <Attila.Szalai@morganstanley.com <mailto:Attila.Szalai@morganstanley.com>>:____
Hi,____
____
In case of udp, the syslog source should handle receiving logs with old and the new version too. (But that is more an exception than the rule, so matching the receiver and the sender is a good idea generaly.)____
____
Before anything else I would check if the logs arrive to the anonymizer host or not. The statistics can help on this. Also, if there are parsing issue, the syslog-ng would tell this through its log.____
____
After that starting the syslog-ng with enabled debug logs can also help on discovering what happening with the received log.____
____
*From:*syslog-ng-bounces@lists.balabit.hu <mailto:syslog-ng-bounces@lists.balabit.hu> [mailto:syslog-ng-bounces@lists.balabit.hu <mailto:syslog-ng-bounces@lists.balabit.hu>] *On Behalf Of *Fekete, Róbert *Sent:* Wednesday, September 28, 2016 1:47 PM *To:* Syslog-ng users' and developers' mailing list *Subject:* Re: [syslog-ng] syslog-ng forwarding and processing issue____
____
Hi, ____
____
The destination on your remote server and the source on the pseudomizer host do not match: the first one uses the udp() driver (RFC3164 protocol), while the second uses the syslog() driver (RFC5424) protocol. ____
____
Change the destination driver to syslog() on the remote server. (For more possibilities, see https://www.balabit.com/documents/syslog-ng-pe-latest-guide s/en/syslog-ng-pe-guide-admin/html/concepts-things-to-consider.html <https://www.balabit.com/documents/syslog-ng-pe-latest-guid es/en/syslog-ng-pe-guide-admin/html/concepts-things-to-consider.html> )____
____
HTH____
____
Robert____
____
On Wed, Sep 28, 2016 at 1:17 PM, Denis Dolinský <denis.dolinsky@gmail.com <mailto:denis.dolinsky@gmail.com>> wrote:____
Hi guys,____
I have following setup in place:____
remote server - 192.168.1.10____
pseudomizer - syslog-ng PE in client mode - 192.168.2.10____
SIEM - 192.168.3.10____
So I am sending syslog logs from remote server to pseudomizer:____
source src { internal()};____
destination dst { udp ("192.168.2.10) port (514);};____
log { source(src); destination (dst);____
this is very old config from syslog v4____
Then on pseudomizer - syslog-ng LTS 6.0.1 PE, I am collecting the logs, processing them - removing private data, putting pseudonyms instead and forwarding them to SIEM.
source s_net_udp514 { syslog( ip(192.168.2.10) ip-protocol(4) transport("udp") so_rcvbuf(2097152) ); };
source src { internal(); unix-dgram("/dev/log"); system (); };
destination d_net_udp514 { syslog ( "192.168.3.10" port(514) transport(udp) spoof_source(yes) mark_mode(periodical));}; rewrite r_rewrite { subst("admin", "pseudonym000001", value("MESSAGE"), flags("global"));
log { source(s_net_udp514); source (src); rewrite(r_rewrite); # do the pseudomizing destination(d_net_udp514); };____
On SIEM device, I can see only pseudomizer internal logs (src), not processed logs from remote server.____
Any advice ?____
Many thanks.____
Denis____
___________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng <https://lists.balabit.hu/mailman/listinfo/syslog-ng> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng <http://www.balabit.com/support/documentation/?product=syslog-ng> FAQ: http://www.balabit.com/wiki/syslog-ng-faq <http://www.balabit.com/wiki/syslog-ng-faq>____
____
__ __
-----------------------------------------------------------
-------------
NOTICE: Morgan Stanley is not acting as a municipal advisor and the opinions or views contained herein are not intended to be, and do not constitute, advice within the meaning of Section 975 of the Dodd-Frank Wall Street Reform and Consumer Protection Act. If you have received this communication in error, please destroy all electronic and paper copies and notify the sender immediately. Mistransmission is not intended to waive confidentiality or privilege. Morgan Stanley reserves the right, to the extent permitted under applicable law, to monitor electronic communications. This message is subject to terms available at the following link: http://www.morganstanley.com/disclaimers <http://www.morganstanley.com/disclaimers> If you cannot access these links, please notify us by reply message and we will send the contents to you. By communicating with Morgan Stanley you consent
to
the foregoing and to the voice recording of conversations with personnel of Morgan Stanley.____
___________________________________________________________
___________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng <https://lists.balabit.hu/mailman/listinfo/syslog-ng> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng <http://www.balabit.com/support/documentation/?product=syslog-ng> FAQ: http://www.balabit.com/wiki/syslog-ng-faq <http://www.balabit.com/wiki/syslog-ng-faq>
____
-- ____
Ing. Denis Dolinský denis.dolinsky@gmail.com <mailto:denis.dolinsky@gmail.com> private cell: _+421 907 530711 <tel:%2B421%20907%20530711>_____
-----------------------------------------------------------
-------------
NOTICE: Morgan Stanley is not acting as a municipal advisor and the opinions or views contained herein are not intended to be, and do not constitute, advice within the meaning of Section 975 of the Dodd-Frank Wall Street Reform and Consumer Protection Act. If you have received this communication in error, please destroy all electronic and paper copies and notify the sender immediately. Mistransmission is not intended to waive confidentiality or privilege. Morgan Stanley reserves the right, to the extent permitted under applicable law, to monitor electronic communications. This message is subject to terms available at the following link: http://www.morganstanley.com/disclaimers <http://www.morganstanley.com/disclaimers> If you cannot access these links, please notify us by reply message and we will send the contents to you. By communicating with Morgan Stanley you consent
to
the foregoing and to the voice recording of conversations with personnel of Morgan Stanley.
___________________________________________________________
___________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng <https://lists.balabit.hu/mailman/listinfo/syslog-ng> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng <http://www.balabit.com/support/documentation/?product=syslog-ng> FAQ: http://www.balabit.com/wiki/syslog-ng-faq <http://www.balabit.com/wiki/syslog-ng-faq>
-- Ing. Denis Dolinský denis.dolinsky@gmail.com <mailto:denis.dolinsky@gmail.com> private cell: _+421 907 530711_
____________________________________________________________
__________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support /documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
____________________________________________________________ __________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support /documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
-- Ing. Denis Dolinský denis.dolinsky@gmail.com private cell: *+421 907 530711 <%2B421%20907%20530711>*
____________________________________________________________ __________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product= syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
____________________________________________________________ __________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/? product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
-- Ing. Denis Dolinský denis.dolinsky@gmail.com private cell: *+421 907 530711*
Hi, Yup, either rfc5424 based format or simple plain old syslog, depending on what the SIEM can accept. syslog(transport(tcp)) is good as it supports both rfc5424 and traditional and has framing, thus can support multiline logs. -- Bazsi On Fri, Sep 30, 2016 at 11:48 AM, Denis Dolinský <denis.dolinsky@gmail.com> wrote:
Hi Balazs,
I totally agree with you. New log management concept I am going to produce shortly, I just wanted this od config to do the job till I will not migrate it to new solution.
So, in new solution, what kind of transport should I consider - TCP ? From source to log processor and then to SIEM ? I don't want to use DNS, I want to keep hostnames, encryption is not required - maybe I would consider it in log processor to SIEM traffic.
Thanks.
Denis
2016-09-29 17:38 GMT+02:00 Scheidler, Balázs <balazs.scheidler@balabit.com
:
Source spoofing is the remnant of a world where devices didn't have tcp based logging, and were broken enough not to recognise the source hostname out of a syslog message.
The world us supposed to have changed and I would drop spoof source any time if it weren't for backward compatibility.
Using spoof source and hence UDP is not a good way to deploy a serious log management system in 2016.
On Sep 29, 2016 09:09, "Denis Dolinský" <denis.dolinsky@gmail.com> wrote:
hi Sandor,
these are my global options:
# # Global options. # options { chain_hostnames(yes); keep_hostname(yes); keep_timestamp(yes); flush_lines(0); perm(0640); stats_freq(3600); };
so spoof source is not necessary here ?
Thanks.
Denis
2016-09-29 9:49 GMT+02:00 Sandor Geller <sandor.geller@ericsson.com>:
Hi,
Source spoofing fakes the source IP address of the outgoing packets, as this can't work with connection-oriented protocols it is usable only with UDP datagrams.
Actually source spoofing isn't needed in most cases and won't even work when spoofing protection is enabled in firewalls / routers and not all hosts are on the same subnet.
Take a look at keep-hostname() and chain-hostnames()
Regards,
Sandor
On 09/29/2016 09:29 AM, Denis Dolinský wrote:
Hi,
yes, I need spoof_source to be enabled for source identification ...
Denis
2016-09-28 16:44 GMT+02:00 Szalai, Attila <Attila.Szalai@morganstanley.com <mailto:Attila.Szalai@morganst anley.com>>:
Just a quick note.____
The warning message about the binding issue caused by the spoof_source option. Is that option necessary?____
__ __
*From:*syslog-ng-bounces@lists.balabit.hu <mailto:syslog-ng-bounces@lists.balabit.hu> [mailto:syslog-ng-bounces@lists.balabit.hu <mailto:syslog-ng-bounces@lists.balabit.hu>] *On Behalf Of *Denis Dolinský *Sent:* Wednesday, September 28, 2016 3:47 PM
*To:* Syslog-ng users' and developers' mailing list *Subject:* Re: [syslog-ng] syslog-ng forwarding and processing issue____
__ __
hi guys,____
__ __
this is stats:____
__ __
destination;d_net_udp514;;a;processed;13 source;s_net_udp514;;a;processed;3 dst.syslog;d_net_udp514#0;udp,192.168.3.1:514;a;dropped;0 dst.syslog;d_net_udp514#0;udp,192.168.3.1:514;a;processed;10 dst.syslog;d_net_udp514#0;udp,192.168.3.1:514;a;stored;0____
__ __
from debug:____
__ __
Incoming log entry; source='s_net_udp514#0', line='<78> remote_server /usr/sbin/cron[24934]: Can't bind hostname for the IP address, therefore using IP address as hostname; IP address='192.168.2.1'____
__ __
Do you see anything what I do not do ?____
__ __
Thanks.____
__ __
Denis____
__ __
2016-09-28 14:02 GMT+02:00 Szalai, Attila <Attila.Szalai@morganstanley.com <mailto:Attila.Szalai@morganstanley.com>>:____
Hi,____
____
In case of udp, the syslog source should handle receiving logs with old and the new version too. (But that is more an exception than the rule, so matching the receiver and the sender is a good idea generaly.)____
____
Before anything else I would check if the logs arrive to the anonymizer host or not. The statistics can help on this. Also, if there are parsing issue, the syslog-ng would tell this through its log.____
____
After that starting the syslog-ng with enabled debug logs can also help on discovering what happening with the received log.____
____
*From:*syslog-ng-bounces@lists.balabit.hu <mailto:syslog-ng-bounces@lists.balabit.hu> [mailto:syslog-ng-bounces@lists.balabit.hu <mailto:syslog-ng-bounces@lists.balabit.hu>] *On Behalf Of *Fekete, Róbert *Sent:* Wednesday, September 28, 2016 1:47 PM *To:* Syslog-ng users' and developers' mailing list *Subject:* Re: [syslog-ng] syslog-ng forwarding and processing issue____
____
Hi, ____
____
The destination on your remote server and the source on the pseudomizer host do not match: the first one uses the udp() driver (RFC3164 protocol), while the second uses the syslog() driver (RFC5424) protocol. ____
____
Change the destination driver to syslog() on the remote server. (For more possibilities, see https://www.balabit.com/documents/syslog-ng-pe-latest-guide s/en/syslog-ng-pe-guide-admin/html/concepts-things-to-consider.html <https://www.balabit.com/documents/syslog-ng-pe-latest-guid es/en/syslog-ng-pe-guide-admin/html/concepts-things-to-consider.html> )____
____
HTH____
____
Robert____
____
On Wed, Sep 28, 2016 at 1:17 PM, Denis Dolinský <denis.dolinsky@gmail.com <mailto:denis.dolinsky@gmail.com>> wrote:____
Hi guys,____
I have following setup in place:____
remote server - 192.168.1.10____
pseudomizer - syslog-ng PE in client mode - 192.168.2.10____
SIEM - 192.168.3.10____
So I am sending syslog logs from remote server to pseudomizer:____
source src { internal()};____
destination dst { udp ("192.168.2.10) port (514);};____
log { source(src); destination (dst);____
this is very old config from syslog v4____
Then on pseudomizer - syslog-ng LTS 6.0.1 PE, I am collecting the logs, processing them - removing private data, putting pseudonyms instead and forwarding them to SIEM.
source s_net_udp514 { syslog( ip(192.168.2.10) ip-protocol(4) transport("udp") so_rcvbuf(2097152) ); };
source src { internal(); unix-dgram("/dev/log"); system (); };
destination d_net_udp514 { syslog ( "192.168.3.10" port(514) transport(udp) spoof_source(yes) mark_mode(periodical));}; rewrite r_rewrite { subst("admin", "pseudonym000001", value("MESSAGE"), flags("global"));
log { source(s_net_udp514); source (src); rewrite(r_rewrite); # do the pseudomizing destination(d_net_udp514); };____
On SIEM device, I can see only pseudomizer internal logs (src), not processed logs from remote server.____
Any advice ?____
Many thanks.____
Denis____
___________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng <https://lists.balabit.hu/mailman/listinfo/syslog-ng> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng <http://www.balabit.com/support/documentation/?product=syslog-ng> FAQ: http://www.balabit.com/wiki/syslog-ng-faq <http://www.balabit.com/wiki/syslog-ng-faq>____
____
__ __
-----------------------------------------------------------
-------------
NOTICE: Morgan Stanley is not acting as a municipal advisor and
the
opinions or views contained herein are not intended to be, and do not constitute, advice within the meaning of Section 975 of the Dodd-Frank Wall Street Reform and Consumer Protection Act. If you have received this communication in error, please destroy all electronic and paper copies and notify the sender immediately. Mistransmission is not intended to waive confidentiality or privilege. Morgan Stanley reserves the right, to the extent permitted under applicable law, to monitor electronic communications. This message is subject to terms available at the following link: http://www.morganstanley.com/disclaimers <http://www.morganstanley.com/disclaimers> If you cannot access these links, please notify us by reply message and we will send
the
contents to you. By communicating with Morgan Stanley you consent
to
the foregoing and to the voice recording of conversations with personnel of Morgan Stanley.____
___________________________________________________________
___________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng <https://lists.balabit.hu/mailman/listinfo/syslog-ng> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng <http://www.balabit.com/support/documentation/?product=syslog-ng> FAQ: http://www.balabit.com/wiki/syslog-ng-faq <http://www.balabit.com/wiki/syslog-ng-faq>
____
-- ____
Ing. Denis Dolinský denis.dolinsky@gmail.com <mailto:denis.dolinsky@gmail.com> private cell: _+421 907 530711 <tel:%2B421%20907%20530711>_____
-----------------------------------------------------------
-------------
NOTICE: Morgan Stanley is not acting as a municipal advisor and
the
opinions or views contained herein are not intended to be, and do not constitute, advice within the meaning of Section 975 of the Dodd-Frank Wall Street Reform and Consumer Protection Act. If you have received this communication in error, please destroy all electronic and paper copies and notify the sender immediately. Mistransmission is not intended to waive confidentiality or privilege. Morgan Stanley reserves the right, to the extent permitted under applicable law, to monitor electronic communications. This message is subject to terms available at the following link: http://www.morganstanley.com/disclaimers <http://www.morganstanley.com/disclaimers> If you cannot access these links, please notify us by reply message and we will send
the
contents to you. By communicating with Morgan Stanley you consent
to
the foregoing and to the voice recording of conversations with personnel of Morgan Stanley.
___________________________________________________________
___________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng <https://lists.balabit.hu/mailman/listinfo/syslog-ng> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng <http://www.balabit.com/support/documentation/?product=syslog-ng> FAQ: http://www.balabit.com/wiki/syslog-ng-faq <http://www.balabit.com/wiki/syslog-ng-faq>
-- Ing. Denis Dolinský denis.dolinsky@gmail.com <mailto:denis.dolinsky@gmail.com> private cell: _+421 907 530711_
____________________________________________________________
__________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support /documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
____________________________________________________________ __________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support /documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
-- Ing. Denis Dolinský denis.dolinsky@gmail.com private cell: *+421 907 530711 <%2B421%20907%20530711>*
____________________________________________________________ __________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support /documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
____________________________________________________________ __________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product= syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
-- Ing. Denis Dolinský denis.dolinsky@gmail.com private cell: *+421 907 530711 <%2B421%20907%20530711>*
____________________________________________________________ __________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/? product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
participants (5)
-
Denis Dolinský
-
Fekete, Róbert
-
Sandor Geller
-
Scheidler, Balázs
-
Szalai, Attila