[syslog-ng] syslog-ng forwarding and processing issue
Denis Dolinský
denis.dolinsky at gmail.com
Fri Sep 30 11:48:07 CEST 2016
Hi Balazs,
I totally agree with you. New log management concept I am going to produce
shortly, I just wanted this od config to do the job till I will not migrate
it to new solution.
So, in new solution, what kind of transport should I consider - TCP ? From
source to log processor and then to SIEM ?
I don't want to use DNS, I want to keep hostnames, encryption is not
required - maybe I would consider it in log processor to SIEM traffic.
Thanks.
Denis
2016-09-29 17:38 GMT+02:00 Scheidler, Balázs <balazs.scheidler at balabit.com>:
> Source spoofing is the remnant of a world where devices didn't have tcp
> based logging, and were broken enough not to recognise the source hostname
> out of a syslog message.
>
> The world us supposed to have changed and I would drop spoof source any
> time if it weren't for backward compatibility.
>
> Using spoof source and hence UDP is not a good way to deploy a serious log
> management system in 2016.
>
> On Sep 29, 2016 09:09, "Denis Dolinský" <denis.dolinsky at gmail.com> wrote:
>
>> hi Sandor,
>>
>> these are my global options:
>>
>> #
>> # Global options.
>> #
>> options { chain_hostnames(yes); keep_hostname(yes); keep_timestamp(yes);
>> flush_lines(0); perm(0640); stats_freq(3600); };
>>
>> so spoof source is not necessary here ?
>>
>> Thanks.
>>
>> Denis
>>
>> 2016-09-29 9:49 GMT+02:00 Sandor Geller <sandor.geller at ericsson.com>:
>>
>>> Hi,
>>>
>>> Source spoofing fakes the source IP address of the outgoing packets, as
>>> this can't work with connection-oriented protocols it is usable only
>>> with UDP datagrams.
>>>
>>> Actually source spoofing isn't needed in most cases and won't even work
>>> when spoofing protection is enabled in firewalls / routers and not all
>>> hosts are on the same subnet.
>>>
>>> Take a look at keep-hostname() and chain-hostnames()
>>>
>>> Regards,
>>>
>>> Sandor
>>>
>>> On 09/29/2016 09:29 AM, Denis Dolinský wrote:
>>> > Hi,
>>> >
>>> > yes, I need spoof_source to be enabled for source identification ...
>>> >
>>> > Denis
>>> >
>>> > 2016-09-28 16:44 GMT+02:00 Szalai, Attila
>>> > <Attila.Szalai at morganstanley.com <mailto:Attila.Szalai at morganst
>>> anley.com>>:
>>> >
>>> > Just a quick note.____
>>> >
>>> > The warning message about the binding issue caused by the
>>> > spoof_source option. Is that option necessary?____
>>> >
>>> > __ __
>>> >
>>> > *From:*syslog-ng-bounces at lists.balabit.hu
>>> > <mailto:syslog-ng-bounces at lists.balabit.hu>
>>> > [mailto:syslog-ng-bounces at lists.balabit.hu
>>> > <mailto:syslog-ng-bounces at lists.balabit.hu>] *On Behalf Of *Denis
>>> > Dolinský
>>> > *Sent:* Wednesday, September 28, 2016 3:47 PM
>>> >
>>> >
>>> > *To:* Syslog-ng users' and developers' mailing list
>>> > *Subject:* Re: [syslog-ng] syslog-ng forwarding and processing
>>> issue____
>>> >
>>> > __ __
>>> >
>>> > hi guys,____
>>> >
>>> > __ __
>>> >
>>> > this is stats:____
>>> >
>>> > __ __
>>> >
>>> > destination;d_net_udp514;;a;processed;13
>>> > source;s_net_udp514;;a;processed;3
>>> > dst.syslog;d_net_udp514#0;udp,192.168.3.1:514;a;dropped;0
>>> > dst.syslog;d_net_udp514#0;udp,192.168.3.1:514;a;processed;10
>>> > dst.syslog;d_net_udp514#0;udp,192.168.3.1:514;a;stored;0____
>>> >
>>> > __ __
>>> >
>>> > from debug:____
>>> >
>>> > __ __
>>> >
>>> > Incoming log entry; source='s_net_udp514#0',
>>> > line='<78> remote_server /usr/sbin/cron[24934]:
>>> > Can't bind hostname for the IP address, therefore using IP address
>>> > as hostname; IP address='192.168.2.1'____
>>> >
>>> > __ __
>>> >
>>> > Do you see anything what I do not do ?____
>>> >
>>> > __ __
>>> >
>>> > Thanks.____
>>> >
>>> > __ __
>>> >
>>> > Denis____
>>> >
>>> > __ __
>>> >
>>> > 2016-09-28 14:02 GMT+02:00 Szalai, Attila
>>> > <Attila.Szalai at morganstanley.com
>>> > <mailto:Attila.Szalai at morganstanley.com>>:____
>>> >
>>> > Hi,____
>>> >
>>> > ____
>>> >
>>> > In case of udp, the syslog source should handle receiving logs with
>>> > old and the new version too. (But that is more an exception than
>>> the
>>> > rule, so matching the receiver and the sender is a good idea
>>> > generaly.)____
>>> >
>>> > ____
>>> >
>>> > Before anything else I would check if the logs arrive to the
>>> > anonymizer host or not. The statistics can help on this. Also, if
>>> > there are parsing issue, the syslog-ng would tell this through its
>>> > log.____
>>> >
>>> > ____
>>> >
>>> > After that starting the syslog-ng with enabled debug logs can also
>>> > help on discovering what happening with the received log.____
>>> >
>>> > ____
>>> >
>>> > *From:*syslog-ng-bounces at lists.balabit.hu
>>> > <mailto:syslog-ng-bounces at lists.balabit.hu>
>>> > [mailto:syslog-ng-bounces at lists.balabit.hu
>>> > <mailto:syslog-ng-bounces at lists.balabit.hu>] *On Behalf Of
>>> *Fekete,
>>> > Róbert
>>> > *Sent:* Wednesday, September 28, 2016 1:47 PM
>>> > *To:* Syslog-ng users' and developers' mailing list
>>> > *Subject:* Re: [syslog-ng] syslog-ng forwarding and processing
>>> issue____
>>> >
>>> > ____
>>> >
>>> > Hi, ____
>>> >
>>> > ____
>>> >
>>> > The destination on your remote server and the source on the
>>> > pseudomizer host do not match: the first one uses the udp() driver
>>> > (RFC3164 protocol), while the second uses the syslog() driver
>>> > (RFC5424) protocol. ____
>>> >
>>> > ____
>>> >
>>> > Change the destination driver to syslog() on the remote server.
>>> (For
>>> > more possibilities, see
>>> > https://www.balabit.com/documents/syslog-ng-pe-latest-guide
>>> s/en/syslog-ng-pe-guide-admin/html/concepts-things-to-consider.html
>>> > <https://www.balabit.com/documents/syslog-ng-pe-latest-guid
>>> es/en/syslog-ng-pe-guide-admin/html/concepts-things-to-consider.html>
>>> )____
>>> >
>>> > ____
>>> >
>>> > HTH____
>>> >
>>> > ____
>>> >
>>> > Robert____
>>> >
>>> > ____
>>> >
>>> > On Wed, Sep 28, 2016 at 1:17 PM, Denis Dolinský
>>> > <denis.dolinsky at gmail.com <mailto:denis.dolinsky at gmail.com>>
>>> wrote:____
>>> >
>>> > Hi guys,____
>>> >
>>> > I have following setup in place:____
>>> >
>>> > remote server - 192.168.1.10____
>>> >
>>> > pseudomizer - syslog-ng PE in client mode - 192.168.2.10____
>>> >
>>> > SIEM - 192.168.3.10____
>>> >
>>> > So I am sending syslog logs from remote server to pseudomizer:____
>>> >
>>> > source src { internal()};____
>>> >
>>> > destination dst { udp ("192.168.2.10) port (514);};____
>>> >
>>> > log { source(src); destination (dst);____
>>> >
>>> > this is very old config from syslog v4____
>>> >
>>> > Then on pseudomizer - syslog-ng LTS 6.0.1 PE, I am collecting the
>>> > logs, processing them - removing private data, putting pseudonyms
>>> > instead and forwarding them to SIEM.
>>> >
>>> > source s_net_udp514 {
>>> > syslog(
>>> > ip(192.168.2.10)
>>> > ip-protocol(4)
>>> > transport("udp")
>>> > so_rcvbuf(2097152)
>>> > );
>>> > };
>>> >
>>> > source src {
>>> > internal();
>>> > unix-dgram("/dev/log");
>>> > system ();
>>> > };
>>> >
>>> > destination d_net_udp514 {
>>> > syslog (
>>> > "192.168.3.10"
>>> > port(514)
>>> > transport(udp)
>>> > spoof_source(yes)
>>> > mark_mode(periodical));};
>>> > rewrite r_rewrite {
>>> > subst("admin", "pseudonym000001", value("MESSAGE"),
>>> flags("global"));
>>> >
>>> > log {
>>> > source(s_net_udp514); source (src);
>>> > rewrite(r_rewrite); # do the pseudomizing
>>> > destination(d_net_udp514);
>>> > };____
>>> >
>>> > On SIEM device, I can see only pseudomizer internal logs (src), not
>>> > processed logs from remote server.____
>>> >
>>> > Any advice ?____
>>> >
>>> > Many thanks.____
>>> >
>>> > Denis____
>>> >
>>> >
>>> > ___________________________________________________________
>>> ___________________
>>> > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>> > <https://lists.balabit.hu/mailman/listinfo/syslog-ng>
>>> > Documentation:
>>> > http://www.balabit.com/support/documentation/?product=syslog-ng
>>> > <http://www.balabit.com/support/documentation/?product=syslog-ng>
>>> > FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>> > <http://www.balabit.com/wiki/syslog-ng-faq>____
>>> >
>>> > ____
>>> >
>>> > __ __
>>> >
>>> > -----------------------------------------------------------
>>> -------------
>>> >
>>> >
>>> > NOTICE: Morgan Stanley is not acting as a municipal advisor and the
>>> > opinions or views contained herein are not intended to be, and do
>>> > not constitute, advice within the meaning of Section 975 of the
>>> > Dodd-Frank Wall Street Reform and Consumer Protection Act. If you
>>> > have received this communication in error, please destroy all
>>> > electronic and paper copies and notify the sender immediately.
>>> > Mistransmission is not intended to waive confidentiality or
>>> > privilege. Morgan Stanley reserves the right, to the extent
>>> > permitted under applicable law, to monitor electronic
>>> > communications. This message is subject to terms available at the
>>> > following link: http://www.morganstanley.com/disclaimers
>>> > <http://www.morganstanley.com/disclaimers> If you cannot access
>>> > these links, please notify us by reply message and we will send the
>>> > contents to you. By communicating with Morgan Stanley you consent
>>> to
>>> > the foregoing and to the voice recording of conversations with
>>> > personnel of Morgan Stanley.____
>>> >
>>> >
>>> > ___________________________________________________________
>>> ___________________
>>> > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>> > <https://lists.balabit.hu/mailman/listinfo/syslog-ng>
>>> > Documentation:
>>> > http://www.balabit.com/support/documentation/?product=syslog-ng
>>> > <http://www.balabit.com/support/documentation/?product=syslog-ng>
>>> > FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>> > <http://www.balabit.com/wiki/syslog-ng-faq>
>>> >
>>> > ____
>>> >
>>> >
>>> >
>>> >
>>> > -- ____
>>> >
>>> > Ing. Denis Dolinský
>>> > denis.dolinsky at gmail.com <mailto:denis.dolinsky at gmail.com>
>>> > private cell: _+421 907 530711 <tel:%2B421%20907%20530711>_____
>>> >
>>> >
>>> >
>>> > -----------------------------------------------------------
>>> -------------
>>> >
>>> > NOTICE: Morgan Stanley is not acting as a municipal advisor and the
>>> > opinions or views contained herein are not intended to be, and do
>>> > not constitute, advice within the meaning of Section 975 of the
>>> > Dodd-Frank Wall Street Reform and Consumer Protection Act. If you
>>> > have received this communication in error, please destroy all
>>> > electronic and paper copies and notify the sender immediately.
>>> > Mistransmission is not intended to waive confidentiality or
>>> > privilege. Morgan Stanley reserves the right, to the extent
>>> > permitted under applicable law, to monitor electronic
>>> > communications. This message is subject to terms available at the
>>> > following link: http://www.morganstanley.com/disclaimers
>>> > <http://www.morganstanley.com/disclaimers> If you cannot access
>>> > these links, please notify us by reply message and we will send the
>>> > contents to you. By communicating with Morgan Stanley you consent
>>> to
>>> > the foregoing and to the voice recording of conversations with
>>> > personnel of Morgan Stanley.
>>> >
>>> >
>>> > ___________________________________________________________
>>> ___________________
>>> > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>> > <https://lists.balabit.hu/mailman/listinfo/syslog-ng>
>>> > Documentation:
>>> > http://www.balabit.com/support/documentation/?product=syslog-ng
>>> > <http://www.balabit.com/support/documentation/?product=syslog-ng>
>>> > FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>> > <http://www.balabit.com/wiki/syslog-ng-faq>
>>> >
>>> >
>>> >
>>> >
>>> >
>>> > --
>>> > Ing. Denis Dolinský
>>> > denis.dolinsky at gmail.com <mailto:denis.dolinsky at gmail.com>
>>> > private cell: _+421 907 530711_
>>> >
>>> >
>>> > ____________________________________________________________
>>> __________________
>>> > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>> > Documentation: http://www.balabit.com/support
>>> /documentation/?product=syslog-ng
>>> > FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>> >
>>>
>>>
>>> ____________________________________________________________
>>> __________________
>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>> Documentation: http://www.balabit.com/support
>>> /documentation/?product=syslog-ng
>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>>
>>>
>>
>>
>> --
>> Ing. Denis Dolinský
>> denis.dolinsky at gmail.com
>> private cell: *+421 907 530711 <%2B421%20907%20530711>*
>>
>> ____________________________________________________________
>> __________________
>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>> Documentation: http://www.balabit.com/support/documentation/?product=
>> syslog-ng
>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>
>>
>>
> ____________________________________________________________
> __________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?
> product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>
>
--
Ing. Denis Dolinský
denis.dolinsky at gmail.com
private cell: *+421 907 530711*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.balabit.hu/pipermail/syslog-ng/attachments/20160930/8280687f/attachment-0001.htm
More information about the syslog-ng
mailing list