Kevin, The internal buffering saved the day, as fyi we are running at about 1100 packets/s in udp of 1 pix(fluctuates heavily during peak hours to about 1500-1600), and we are going to be adding two more pix's soon that this syslog engine will have to handle aswell so this really did the trick. Its helping my analyzer out a bit aswell. Thanks alot for your time to take a look at my config. William -----Original Message----- From: syslog-ng-bounces@lists.balabit.hu on behalf of Kevin Sent: Tue 6/28/2005 10:26 PM To: syslog-ng@lists.balabit.hu Subject: Re: [syslog-ng] multiple source/destination setup On 6/28/05, William Bell <williamb@cwie.net> wrote:
For a while now I have been using a configuration that includes 3 sources (remote udp, remote tcp, internal) and 3 destinations ( remote archive, local, loopback for an analyzer). Today I tried to move this setup that works on all levels to another machine. This new machine has an EXTREMELY large udp traffic base. About 7.5gb of syslog per day. The setup of the new machine is exactly the same the only variable is this large traffic volume.
Is the hardware the same? Can you provide details on the average and peak packets per second (PPS) for the new host? Perhaps you might monitor the switch port, perhaps by using something like MRTG?
Now, to the problem. I am receiving all the syslog on the remote destination archive across normal syslog export mechanisms. However the two log destinations under it are not receiving the "copies" of the logs.
So you are saying that the archive is locally logging the messages correctly, but is failing to forward copies of the messages out?
I was wondering if anyone else uses syslog-ng to log large pix firewall syslog traffic, and have seen anything similar. My theory is that syslog-ng is just not keeping up with the speed that its receiving data.
This is possible. A syslog server is limited by the log software, by the OS, and by the drives and drive controller. The limits are usually more about the number of messages (packets) per second than bytes per second. Packets can be dropped in syslog-ng itself (logged via the internal source, usually as "stats"), by the TCP/IP stack (logged in OS-specific kernel data structures usually viewable via "netstat -p udp") or because the OS or NIC cannot handle the data coming in over the LAN (often logged nowhere).
Here is a copy of the conf file with only the private info adjusted. . . . log_fifo_size(0); . . . sync(1); . . .
Ouch. With these settings, syslog-ng isn't doing any internal buffering, In my experience, a high-volume logger needs these two values set high, I use the following: sync(256); log_fifo_size(4095); With these settings, we are currently processing UDP syslog data totalling just over 11GB/day (65 million messages/day) without significant packet loss, there is some loss (perhaps one in fifty thousand packets) during the peak hour of the day -- at the peak hour the logger processes 6 million messages totalling just over 1GB, at that volume a few packets dropped by the OS is acceptable :) Kevin Kadow (P.S. Our record for log events successfully received and written in a one second interval is 2,100; this is based on the message timestamp, I haven't yet tried the recently posted patch to count messages/second.) _______________________________________________ syslog-ng maillist - syslog-ng@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html