Hi, Thanks for your answer. I tried to separate the destination in two log statements : log { source(dhcp_src); filter(f_chroot_l2); destination(dhcp_log); }; log { source(dhcp_src); filter(f_chroot_l2); destination(dhcp_sql); flags(final); }; But i got the same result, blocking dhcp_sql destination blocks dhcp_log destination. I have found a workaround, i throw my log in a unix socket and the file dhcpd.log together, then i re-take the socket to put in SQL database. (not so pretty) New versions of syslog-ng have a different behavior which is better to not block inputs ? My version of syslog-ng is : dhcp-SEIC-3 syslog-ng # syslog-ng --version syslog-ng 3.0.4 Revision: ssh+git://bazsi@git.balabit//var/scm/git/syslog-ng/syslog-ng-ose--mainline--3.0#master#1b5d618e301ad94aa20e692ffba16469dece8d10 Compile-Date: Dec 3 2009 16:17:01 Enable-Threads: off Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-Sun-STREAMS: off Enable-Sun-Door: off Enable-IPv6: on Enable-Spoof-Source: off Enable-TCP-Wrapper: on Enable-SSL: on Enable-SQL: off Enable-Linux-Caps: off Enable-Pcre: on David Ulrich --- Head of Systems, Quality and Security e-mail: david.ulrich@netplus.pro Direct: +41275657531 Groupe: +41275657530 Réception: +41275657511 Fax: +41275657512 netplus.ch SA Techno-pôle 3 CH-3960 Sierre Le 16 mars 2012 à 13:24, Balazs Scheidler a écrit :
On Tue, 2012-02-21 at 14:10 +0100, Ulrich David wrote:
Hi,
I have an ISC-DHCP who is logging on a local facility on a chrooted system :
#options { # chain_hostnames(no); # use_dns(no); # flush_lines(1); # stats_freq(3600); #}; #source dhcp_src { # unix-stream("/chroot/dhcp/dev/log" max-connections(256)); #};
My syslog-ng take logs from this source, filter on program and then put the logs on two destinations :
#filter f_chroot_l2 { # program(dhcpd); #}; # #destination dhcp_log { file("/var/log/dhcpd.log" perm(0644)); }; #destination dhcp_sql { program("/usr/local/sbin/dhcp_log_to_mysql"); }; # #log { # source(dhcp_src); # filter(f_chroot_l2); # destination(dhcp_log); # destination(dhcp_sql); # flags(final ); #};
Everything goes well… until I block my program "dhcp_log_to_mysql" (I do it with a lock on the mysql table it uses or with a sleep in the script - for test).
When "dhcp_log_to_mysql" is blocked, logs continues to be written on dhcp_log for some minutes/seconds then it stops. My chrooted dhcpd hangs on that too.
I don't know why syslog-ng is not just dropping logs (continuing eating them on the source). I don't have flow-control enabled… and I find nothing which can explain the problem.
I try to resize the buffer output with very big number (log_fifo_size(45000000)) but it changes nothing. Logs and dhcpd hangs at the same moment.
Well, I probably know the reason for this, however I agree that it is confusing.
The history for the behaviour is this: - earlier syslog-ng prioritized all output operations over input ones - this meant that if any output driver had things to do, it inhibited all input drivers to accept further messages. - however with the threaded behaviour the same prioritization is very difficult to implement (and would probably cause a lot of performance penalties), so file destinations enable flow-control implicitly - this flow-control is not completely the same as using "flags(flow-control)" as when the file destination is suspended because of an error, it is not in effect. This is called "soft-flow-control" in the codebase. - flow-control is inherited to all parallel branches of a log statement.
This means that if you used separate log statements, the program destination wouldn't be flow controlled.
I remember having to readd this "inheritance" of flow-control (as it was removed first while implementing the threaded architecture), because without it flow-control wasn't in effect when filtering was used in one of the branches.
Anyway, I'd have to give it some thought, since this case is clearly difficult to diagnose and understand.
For now, use a separate log statement, that should settle it.
-- Bazsi
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq