[syslog-ng] spoof-source performance issues

Tue Jun 29 18:44:44 CEST 2010

Hello,

2010-06-29 16:52 keltezéssel, Martin Holste írta:
> I agree that is a better solution.  So I should have no problems
> compiling against 1.1.4?  I'm on SuSE 10.2 
That is EoL for more than a year...

> which has 1.1.2.1
> (apparently unpatched), so I guess some vendors are behind.  I looked
> at the src rpm for SuSE 11
Which SuSE version exactly? 11.0 (also EoL soon), 11.1, 11.2 or 11.3,
which is still in development. If this last one, we could still get it
fixed (OK, complete freeze of factory was just announced today, but even
if it is too late there, it would be a good candidate for on-line update)

>  and it is also missing the correct checksum
> code, so as far as I can tell, spoof_source will never work correctly
> on SuSE without manual patching.
>   
This is the content of the latest source rpm:

-rw-r--r--  1 czanik users    913 2007 jan 16 libnet-1.1.2.1-arrray-fix.diff
-rw-r--r--  1 czanik users    437 2007 jan 16 libnet-1.1.2.1-makefile.diff
-rw-r--r--  1 czanik users   1700 2007 jan 16
libnet-1.1.2.1-strict-aliasing-fix.diff
-rw-r--r--  1 czanik users    505 2007 jan 16
libnet-1.1.2.1-uninitialized-fix.diff
-rw-r--r--  1 czanik users   1351 2007 jan 16 libnet-endianess-fix.diff
-rw-r--r--  1 czanik users   1203 2009 okt  4 libnet-shared.diff
-rw-r--r--  1 czanik users   4663 jún 10 02.53 libnet.spec
-rw-r--r--  1 czanik users 767201 2007 jan 16 libnet.tar.bz2

So, it still seems to be missing in factory (patches were not touched
since previous release).

Could somebody point me to the missing patch with a description what it
fixes exactly? I'll pass it on to SuSE release manager and libnet
maintainers as soon as I receive it.
Thanks, bye,
CzP

> On Tue, Jun 29, 2010 at 4:30 AM, Sandor Geller
> <Sandor.Geller at morganstanley.com> wrote:
>   
>> Hi,
>>
>> Disabling checksums would be a very bad workaround. If you're using a
>> buggy libnet version then it is up to you to fix it - or build
>> syslog-ng against a fixed version as libnet is linked statically.
>> Linux distributors either ship a patched libnet 1.1.2.1 version or
>> ship 1.1.4 instead which doesn't have the checksum bug.
>>
>> Regards,
>>
>> Sandor
>>
>> On Tue, Jun 29, 2010 at 4:31 AM, Martin Holste <mcholste at gmail.com> wrote:
>>     
>>> Actually, I did more research on this and found that two separate
>>> people back in 2007 had this same problem on the mailing list.  See
>>> threads "Lost packets; UDP Checksum (chksum) errors; forwarding -
>>> source spoofing; libnet bug" as well as "Forwarding + Spoofing =
>>> Errors & Dropped Packets?"  I believe I've definitively proven the
>>> problem to be invalid UDP checksums sent by libnet 1.1.2.1 as
>>> indicated in the first thread by Marvin Nipper.  Further research
>>> shows that there is a Linux kernel-level setting that can act as a
>>> workaround by setting the socket option SO_NO_CHECK, which disables
>>> checksum verifications.  So, either Syslog-NG needs to incorporate a
>>> newer, fixed libnet version (it was indicated that it did not compile
>>> using 1.1.3 Beta), or a socket option for receiving needs to be set or
>>> made as an available option to set like the receive buffer.
>>>
>>> On Mon, Jun 28, 2010 at 1:40 PM, Zoltán Pallagi <pzolee at balabit.hu> wrote:
>>>       
>>>> Hi,
>>>>
>>>> I think it will be an udp kernel buffer problem (and not syslog problem),
>>>> see the earlier thread of "[syslog-ng] Tests using loggen - not receiving
>>>> all the packets" in this mail list.
>>>>
>>>>
>>>>
>>>>
>>>> 2010.06.28. 20:21 keltezéssel, Martin Holste írta:
>>>>         
>>>>> I'm finding that with a destination of udp("10.x.x.x", port(514)
>>>>> spoof_source(yes)) about half of messages get lost when going from one
>>>>> syslog-ng host to another at a high message rate (>  3k/sec).  This is
>>>>> on 3.1 OSE and the hosts are on the same subnet and switch, so there
>>>>> shouldn't be any network devices interfering.  Has anyone else had
>>>>> this same issue?  My hunch is that it's either a performance issue
>>>>> with the way the libnet (I'm using 1.1.2.1 on SuSE 10.2) API is
>>>>> implemented or it's an issue within the libnet API.  Has anyone else
>>>>> noticed performance problems when using spoof_source?
>>>>>
>>>>> ______________________________________________________________________________
>>>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>>>> Documentation:
>>>>> http://www.balabit.com/support/documentation/?product=syslog-ng
>>>>> FAQ: http://www.campin.net/syslog-ng/faq.html
>>>>>
>>>>>
>>>>>
>>>>>           
>>>>
>>>> --
>>>> pzolee
>>>>
>>>>         
>>> ______________________________________________________________________________
>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
>>> FAQ: http://www.campin.net/syslog-ng/faq.html
>>>
>>>
>>>       
>> ______________________________________________________________________________
>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
>> FAQ: http://www.campin.net/syslog-ng/faq.html
>>
>>
>>     
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.campin.net/syslog-ng/faq.html
>
>
>   