this seems to be a completely unrelated issue. are you sure syslog isn't dropped by packet filtering, firewalls etc? ----- Original message -----
Hello all,
I hope what I'm asking hasn't been covered previously, I tried some searches with no luck. If I'm duplicating something else, I apologize.
My problem is, I have 6 DHCP servers with identical syslog-ng.conf and syslog.conf files on them. The set up is as so:
dhcp-a-01 and dhcp-b-01 are a DHCP failover pair dhcp-a-02 and dhcp-b-02 are a DHCP failover pair dhcp-a-03 and dhcp-b-03 are a DHCP failover pair
The 'dhcp-a' servers are in the A data center. 'dhcp-b' servers are in the B data center.
Again, the syslog-ng.conf files on all of them are identical, checked with sha1sum. It is confirmed that all of them are using syslog-ng for logging.
I have them all set to log to the same remote logging server. Logs from dhcp-[a,b]-01 and dhcp-[a,b]-03 are making it to the remote server with no issues. I can see it on the remote server and I can see it when doing a 'tcpdump port 514' on the servers themselves.
For some reason, I'm not seeing any logs from dhcp-[a,b]-02 on the remote server and when I do 'tcpdump port 514' for a length of time, I get:
dhcp-b-02:~# tcpdump port 514 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes ^C 0 packets captured 0 packets received by filter 0 packets dropped by kernel
when the other servers, done at the same time, show packets captured.
I just did a "tail -f /var/log/syslog > /tmp/test" all of the servers between 11:43:26 and 11:45:38 (2m12s). In that time:
dhcp-[a,b]-01 had roughly 2700 lines dhcp-[a-b]-02 had roughly 11000 lines dhcp-[a-b]-03 had roughly 1100 lines
So to me it seems like there's some sort of throttling on the data that's able to be sent. There's ~5x more traffic on pair 2 than 1 (which will be rebalanced, just trying to get this working first) so that would make sense. The only thing that I could find that looks like it would help is the log_fifo_size option, but that doesn't seem to help -- I've made several adjustments to it, but it doesn't seem to make any difference.
Can someone please let me know what I'm missing? Thanks!
Brian