[syslog-ng] Multi-core use in threaded mode

[Sandor Geller]

Hi,

See the 'Losing to much remote sent logs' thread

There are issues with UDP in threaded mode so the workaround is that
instead of adding threaded(yes) to the global options it should set at
the individual non-UDP sources / destinations.

hth,

Sandor

On Wed, Apr 18, 2012 at 8:55 AM, Fekete Robert <frobert at balabit.hu> wrote:
> The problem with UDP is that it is a connectionless protocol which doesn't scale
> to multiple threads in syslog-ng. If you must stick to UDP, ypu can try to
> separate the messages to different source or destination drivers, so syslog-ng
> will start separate threads for them.
>
> I don't know if Bazsi is planning to add multiple threads to UDP handling.
>
> Robert
>
> On 04/17/2012 09:12 PM, Martin Holste wrote:
>
>> Yes, threaded(yes) is enabled as confirmed when using top -H to see
>> individual threads.  I have one inbound UDP source which gets a
>> thread, then one outbound UDP source and a program source.  Each looks
>> to get about 20-30% of a CPU, but the sum total of all three can never
>> go above 100%.  So, is it individual log {} stanzas that are tied to a
>> single CPU?
>>
>> 2012/4/17 Fekete Róbert<frobert at balabit.hu>:
>>> Hi,
>>>
>>> is threading enabled? It is off by default.
>>> Try:
>>> options {threaded(yes) ; };
>>>
>>> Also, scaling depends on the sources/destinations you use. For example, a tcp source can scale only if you have multiple clients that send messages to the same source.
>>>
>>> Regards,
>>>
>>> Robert
>>>
>>>
>>> On Tuesday, April 17, 2012 20:34 CEST, Martin Holste<mcholste at gmail.com>  wrote:
>>>
>>>> I'm seeing that on a very busy syslog-ng 3.3.4 box, CPU never rises
>>>> above 103% and log loss occurs.  Does threaded mode not allow separate
>>>> threads to utilize multiple cores?  I need to scale above a single
>>>> CPU.
>>>> ______________________________________________________________________________
>>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>>> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
>>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>>>
>>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> ______________________________________________________________________________
>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>>
>> ______________________________________________________________________________
>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>
>>
>
>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>