https://bugzilla.balabit.com/show_bug.cgi?id=217 --- Comment #2 from Evan Rempel <erempel@uvic.ca> 2013-01-22 07:00:04 --- The best I can tell is that there is something wrong with the file destination in 3.4 Using a file destination seems to block everything. I can't get stats via /usr/local/sbin/syslog-ng-ctl stats --control=/var/local/syslog-ng.ctl because it just hangs. It seems like it does not honour the flush_lines option, or if it does, it fsyncs every line or something. The wait I/O goes to 40% on what is otherwise an idle box. Even though I use a large (2,000,000) log_fifo_size destination d_archive { file("/var/syslog/syslog.test.log" owner("root") group("syslogs") perm(0640) log_fifo_size(2000000) template(t_standardfile)); }; And have source unix_network_tcp { tcp(localip(catamount.comp.uvic.ca) port(514) max_connections(10) log_fetch_limit(500) log_iw_size(1000000) ); }; The destination will not queue up messages more than about 1000. After 3 minutes of elapsed time, the first stats came back with dst.file;d_archive#0;/var/syslog/syslog.test.log;a;dropped;0 dst.file;d_archive#0;/var/syslog/syslog.test.log;a;processed;6000 dst.file;d_archive#0;/var/syslog/syslog.test.log;a;stored;500 If I use a destination of destination d_cat { program("/bin/cat >> /var/syslog/syslog.test.log" log_fifo_size(2000000) template(t_standardfile)); }; Then the messages are processed quickly and the CPU hits 100% for the syslog-ng process. I also see the queued count reach 5,000 at times with this destination. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

https://bugzilla.balabit.com/show_bug.cgi?id=217 Gergely Nagy <algernon@balabit.hu> changed: What |Removed |Added ---------------------------------------------------------------------------- Target Milestone|--- |3.4.1 CC| |algernon@balabit.hu AssignedTo|bazsi@balabit.hu |algernon@balabit.hu -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

https://bugzilla.balabit.com/show_bug.cgi?id=217 --- Comment #4 from Gergely Nagy <algernon@balabit.hu> 2013-01-22 13:12:00 --- Reproduced, both the insanely slow file destination, and that it starves pretty much everything else. I'm running it under cachegrind, and even ^C is awkwardly slow, let alone reacting to syslog-ng-ctl. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

https://bugzilla.balabit.com/show_bug.cgi?id=217 Gergely Nagy <algernon@balabit.hu> changed: What |Removed |Added ---------------------------------------------------------------------------- Resolution| |FIXED Status|ASSIGNED |RESOLVED --- Comment #6 from Gergely Nagy <algernon@balabit.hu> 2013-01-22 13:45:26 --- Right, this is terribly embarassing. Back in May, when restoring fsync(yes) functionality, the order of arguments to one function was mixed up, so that flush-lines was set as if it was fsync, and the value of fsync was set as flush-lines. This resulted in fsync() being permanently enabled, and flush-lines() always being 0 or 1, and this led to horrible performance. Back then, this passed my tests, as I only tested that fsync() gets called, and it did. Lesson learned, I need to do negative tests too. Nevertheless, I corrected this mistake on 3.4's master: https://github.com/balabit/syslog-ng-3.4/commit/c5775b689720c811905ebf55ee07... -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.