I think I'm getting this behavior as well. I use a master syslog-ng process to send filtered packets to another syslog-ng process on the same box over tcp, and its going horridly slow. I tried replacing the syslog-ng child process with netcat, and it the master process is sending really fast. But whenever I start up the child syslog-ng, it starts backing up and the master process starts buffering all the data it cant send.
This behavior seems to have started after upgrading to 3.1.0. I'll try downgrading back to 3.0.5 and see if it continues, as well as try to figure out why its performing so bad.
I've done an strace, and its definitely having something fishy happen with the socket.

(syslog-ng.516 is the process name of the child process)
# strace -p `pidof syslog-ng.516` -i -r -T
...
     0.000029 [      370640d510] write(47, "15364 [3845467040] Apr  1 15:21:"..., 125) = 125 <0.000009>
     0.000046 [      370640dd40] lseek(47, 0, SEEK_END) = 3546874498 <0.000006>
     0.000029 [      370640d510] write(47, "15364 [3845467040] Apr  1 15:21:"..., 186) = 186 <0.000009>
     0.000067 [      3705ccae0f] poll([{fd=3, events=POLLIN}, {fd=5, events=POLLIN}, {fd=7, events=POLLIN}, {fd=9, events=0}], 4, 0) = 0 (Timeout) <0.000008>
     0.000307 [      3705ccae0f] poll([{fd=3, events=POLLIN}, {fd=5, events=POLLIN}, {fd=7, events=POLLIN}, {fd=9, events=0}], 4, 0) = 0 (Timeout) <0.000010>
     0.000310 [      3705ccae0f] poll([{fd=3, events=POLLIN}, {fd=5, events=POLLIN}, {fd=7, events=POLLIN}, {fd=9, events=0}], 4, 0) = 0 (Timeout) <0.000008>
     0.000262 [      3705ccae0f] poll([{fd=3, events=POLLIN}, {fd=5, events=POLLIN}, {fd=7, events=POLLIN}, {fd=9, events=0}], 4, 4648) = 0 (Timeout) <4.648231>
     4.648391 [      370640dd40] lseek(15, 0, SEEK_END) = 413347569 <0.000009>
     0.000041 [      370640d510] write(15, "Apr  1 15:21:49 gwsinam03 postfi"..., 230) = 230 <0.000015>
     0.000074 [      370640dd40] lseek(15, 0, SEEK_END) = 413347799 <0.000006>
     0.000033 [      370640d510] write(15, "Apr  1 15:21:49 gwsinam03 postfi"..., 68) = 68 <0.000023>
As you can see, its sitting on that poll for 4 seconds before continuing.
>From lsof, the FDs its polling:
syslog-ng 29232 root    3r  FIFO                0,6             214383200 pipe
syslog-ng 29232 root    5u  unix 0xffff81022f7bf680             214383203 /var/lib/syslog-ng/syslog-ng.ctl
syslog-ng 29232 root    7u  IPv4          214383201                   TCP 127.0.0.1:516 (LISTEN)
syslog-ng 29232 root    9u  IPv4          214383285                   TCP 127.0.0.1->127.0.0.1:28061 (ESTABLISHED)



Sent: Thursday, April 01, 2010 7:33:12 AM
From: Martin Holste <mcholste@gmail.com>
To: cdukes@cdukes.com, Syslog-ng users' and developers' mailing list <syslog-ng@lists.balabit.hu>
Subject: Re: [syslog-ng] Tests using loggen - not receiving all the packets
What do you get if you send the loggen data to a simple netcat session with its output redirected to a flat file?  Do you see all 55k messages using wc -l?

On Thu, Apr 1, 2010 at 6:51 AM, Clayton Dukes <cdukes@gmail.com> wrote:
I should have mentioned that this is logging directly to a file.

destination df_logzilla {
   file("/var/log/logzilla/syslog.log"
   template("$HOST\t$FACILITY\t$LEVEL\t$TAG\t$YEAR-$MONTH-$DAY\t$HOUR:$MIN:$SEC\t$PROGRAM\t$MSG\n")
   );
};



On Wed, Mar 31, 2010 at 11:47 PM, Clayton Dukes <cdukes@gmail.com> wrote:
Hi Folks,
I'm trying to run a test to check insert rates.
If I run this command:

./loggen -r 5000 -D -I 10 127.0.0.1 514

The output shows:
average rate = 5441.60 msg/sec, count=54420, time=10.007, msg size=256, bandwidth=1360.40 kB/sec

But, my stats don't show that many messages received:

syslog-ng[6660]: Log statistics; dropped=\'pipe(/dev/xconsole)=0\', processed=\'center(queued)=24232\', processed=\'center(received)=8077, processed=\'destination(df_logzilla)=8077\'

As you can see, it sent 55k messages, but I only received 8k.
Am I doing something wrong?

Here are my options in the syslog-ng config:
options {
      long_hostnames(off);
      log_msg_size(8192);
      flush_lines(1);
      log_fifo_size(16384);
      time_reopen(10);
      use_dns(yes);
      dns_cache(yes);
      use_fqdn(yes);
      keep_hostname(yes);
      chain_hostnames(no);
      perm(0644);
     stats_freq(60);

};


--
______________________________________________________________

Clayton Dukes
______________________________________________________________



--
______________________________________________________________

Clayton Dukes
______________________________________________________________

______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.campin.net/syslog-ng/faq.html




______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html