file. What network adapter and Ethernet driver does the machine use?
This is a Dell R610, which reports: Ethernet controller: Broadcom Corporation NetXtreme II BCM5709 Gigabit Ethernet The kernel is using the "bnx2" driver. There doesn't appear to be a problem getting packets from the network to a user space program like awk or netcat.
Which operating system?
Linux (CentOS 5.5).
Did you try tweaking:
log_fetch_limit() so_rcvbuf() so_sndbuf()
A little. It wasn't clear from the documentation how so_rcvbuf() interacts with the kernel settings (net.core.rmem_default and net.core.rmem_max). My naive assumption is that setting so_rcvbuf() is largely the same as setting net.core.rmem_default (but with a more local impact). Trying various values for log_fetch_limit() didn't appear to have much impact on things, but absent some guidance on what would constitute sensible numbers I was sort of flailing around.
Do you know for sure that when 'loggen' says it's creating X msgs/sec, that this reading is actually accurate?
Yes.
Can you profile syslog-ng using oprofile or the various built in profiling and debugging options in its configure script? Can you check if the CFLAGS look good?
I'm using the binary packages from balabit.com. If these aren't optimally built, I'm looking at the wrong product.