[syslog-ng] syslog-ng blocking application
Ulrich David
david.ulrich at netplus.pro
Fri Mar 16 15:54:09 CET 2012
i don't have the possibility to test it with threads on, on gentoo it seems to be only on 3.2.5 activated.
But I have done a test on a 3.2.4 and having two difference log statements works (same syslog-ng.conf than 3.0.4). More surprising in one statement it works too.
# syslog-ng --version
syslog-ng 3.2.4
Installer-Version: 3.2.4
Revision: ssh+git://bazsi@git.balabit//var/scm/git/syslog-ng/syslog-ng-ose--mainline--3.2#master#ef7b91e4a1b1f9628c66138b4ae83de7e4c697c6
Compile-Date: Jul 13 2011 15:17:37
Enable-Threads: off
Enable-Debug: off
Enable-GProf: off
Enable-Memtrace: off
Enable-Sun-STREAMS: off
Enable-IPv6: on
Enable-Spoof-Source: off
Enable-TCP-Wrapper: on
Enable-SSL: on
Enable-SQL: off
Enable-Linux-Caps: off
Enable-Pcre: on
Enable-Pacct: off
Is it possible there is just a small bug on 3.0.4 ?
David Ulrich
---
Head of Systems, Quality and Security
e-mail: david.ulrich at netplus.pro
Direct: +41275657531
Groupe: +41275657530
Réception: +41275657511
Fax: +41275657512
netplus.ch SA
Techno-pôle 3
CH-3960 Sierre
Le 16 mars 2012 à 15:05, Martin Holste a écrit :
> What happens if you recompile with threads enabled and then add
> threads(yes); to your global config? You should get multiple threads
> running for each log statement which should theoretically alleviate
> the blocking problem.
>
> On Fri, Mar 16, 2012 at 8:51 AM, Ulrich David <david.ulrich at netplus.pro> wrote:
>> Hi,
>>
>> Thanks for your answer.
>>
>> I tried to separate the destination in two log statements :
>> log {
>> source(dhcp_src);
>> filter(f_chroot_l2);
>> destination(dhcp_log);
>> };
>> log {
>> source(dhcp_src);
>> filter(f_chroot_l2);
>> destination(dhcp_sql);
>> flags(final);
>> };
>>
>> But i got the same result, blocking dhcp_sql destination blocks dhcp_log destination.
>> I have found a workaround, i throw my log in a unix socket and the file dhcpd.log together, then i re-take the socket to put in SQL database. (not so pretty)
>>
>> New versions of syslog-ng have a different behavior which is better to not block inputs ?
>>
>> My version of syslog-ng is :
>>
>> dhcp-SEIC-3 syslog-ng # syslog-ng --version
>> syslog-ng 3.0.4
>> Revision: ssh+git://bazsi@git.balabit//var/scm/git/syslog-ng/syslog-ng-ose--mainline--3.0#master#1b5d618e301ad94aa20e692ffba16469dece8d10
>> Compile-Date: Dec 3 2009 16:17:01
>> Enable-Threads: off
>> Enable-Debug: off
>> Enable-GProf: off
>> Enable-Memtrace: off
>> Enable-Sun-STREAMS: off
>> Enable-Sun-Door: off
>> Enable-IPv6: on
>> Enable-Spoof-Source: off
>> Enable-TCP-Wrapper: on
>> Enable-SSL: on
>> Enable-SQL: off
>> Enable-Linux-Caps: off
>> Enable-Pcre: on
>>
>>
>> David Ulrich
>> ---
>> Head of Systems, Quality and Security
>>
>> e-mail: david.ulrich at netplus.pro
>> Direct: +41275657531
>> Groupe: +41275657530
>> Réception: +41275657511
>> Fax: +41275657512
>>
>> netplus.ch SA
>> Techno-pôle 3
>> CH-3960 Sierre
>>
>> Le 16 mars 2012 à 13:24, Balazs Scheidler a écrit :
>>
>>> On Tue, 2012-02-21 at 14:10 +0100, Ulrich David wrote:
>>>> Hi,
>>>>
>>>> I have an ISC-DHCP who is logging on a local facility on a chrooted system :
>>>>
>>>> #options {
>>>> # chain_hostnames(no);
>>>> # use_dns(no);
>>>> # flush_lines(1);
>>>> # stats_freq(3600);
>>>> #};
>>>> #source dhcp_src {
>>>> # unix-stream("/chroot/dhcp/dev/log" max-connections(256));
>>>> #};
>>>>
>>>> My syslog-ng take logs from this source, filter on program and then put the logs on two destinations :
>>>>
>>>> #filter f_chroot_l2 {
>>>> # program(dhcpd);
>>>> #};
>>>> #
>>>> #destination dhcp_log { file("/var/log/dhcpd.log" perm(0644)); };
>>>> #destination dhcp_sql { program("/usr/local/sbin/dhcp_log_to_mysql"); };
>>>> #
>>>> #log {
>>>> # source(dhcp_src);
>>>> # filter(f_chroot_l2);
>>>> # destination(dhcp_log);
>>>> # destination(dhcp_sql);
>>>> # flags(final );
>>>> #};
>>>>
>>>> Everything goes well… until I block my program "dhcp_log_to_mysql" (I do it with a lock on the mysql table it uses or with a sleep in the script - for test).
>>>>
>>>> When "dhcp_log_to_mysql" is blocked, logs continues to be written on dhcp_log for some minutes/seconds then it stops. My chrooted dhcpd hangs on that too.
>>>>
>>>> I don't know why syslog-ng is not just dropping logs (continuing eating them on the source). I don't have flow-control enabled… and I find nothing which can explain the problem.
>>>>
>>>> I try to resize the buffer output with very big number (log_fifo_size(45000000)) but it changes nothing. Logs and dhcpd hangs at the same moment.
>>>
>>> Well, I probably know the reason for this, however I agree that it is
>>> confusing.
>>>
>>> The history for the behaviour is this:
>>> - earlier syslog-ng prioritized all output operations over input ones
>>> - this meant that if any output driver had things to do, it inhibited
>>> all input drivers to accept further messages.
>>> - however with the threaded behaviour the same prioritization is very
>>> difficult to implement (and would probably cause a lot of performance
>>> penalties), so file destinations enable flow-control implicitly
>>> - this flow-control is not completely the same as using
>>> "flags(flow-control)" as when the file destination is suspended because
>>> of an error, it is not in effect. This is called "soft-flow-control" in
>>> the codebase.
>>> - flow-control is inherited to all parallel branches of a log
>>> statement.
>>>
>>> This means that if you used separate log statements, the program
>>> destination wouldn't be flow controlled.
>>>
>>> I remember having to readd this "inheritance" of flow-control (as it was
>>> removed first while implementing the threaded architecture), because
>>> without it flow-control wasn't in effect when filtering was used in one
>>> of the branches.
>>>
>>> Anyway, I'd have to give it some thought, since this case is clearly
>>> difficult to diagnose and understand.
>>>
>>> For now, use a separate log statement, that should settle it.
>>>
>>> --
>>> Bazsi
>>>
>>>
>>> ______________________________________________________________________________
>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>>
>>
>> ______________________________________________________________________________
>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
More information about the syslog-ng
mailing list