Balazs, Thank you for the response to my question. I thought I responded to both you and the list and didn't, so I wanted to make sure it made it to the list. It turns out that that's exactly what it was -- there were some changes in the original configuration that caused the 2nd pair to have iptables in place blocking the logging. I'll get it fixed and be good to go. Thanks for catching that and apologies to the list for not thinking of it before I posted. Brian On Tue, Sep 11, 2012 at 2:14 PM, Balazs Scheidler <bazsi77@gmail.com> wrote:
**
this seems to be a completely unrelated issue. are you sure syslog isn't dropped by packet filtering, firewalls etc?
----- Original message -----
Hello all,
I hope what I'm asking hasn't been covered previously, I tried some searches with no luck. If I'm duplicating something else, I apologize.
My problem is, I have 6 DHCP servers with identical syslog-ng.conf and syslog.conf files on them. The set up is as so:
dhcp-a-01 and dhcp-b-01 are a DHCP failover pair dhcp-a-02 and dhcp-b-02 are a DHCP failover pair dhcp-a-03 and dhcp-b-03 are a DHCP failover pair
The 'dhcp-a' servers are in the A data center. 'dhcp-b' servers are in the B data center.
Again, the syslog-ng.conf files on all of them are identical, checked with sha1sum. It is confirmed that all of them are using syslog-ng for logging.
I have them all set to log to the same remote logging server. Logs from dhcp-[a,b]-01 and dhcp-[a,b]-03 are making it to the remote server with no issues. I can see it on the remote server and I can see it when doing a 'tcpdump port 514' on the servers themselves.
For some reason, I'm not seeing any logs from dhcp-[a,b]-02 on the remote server and when I do 'tcpdump port 514' for a length of time, I get:
dhcp-b-02:~# tcpdump port 514 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes ^C 0 packets captured 0 packets received by filter 0 packets dropped by kernel
when the other servers, done at the same time, show packets captured.
I just did a "tail -f /var/log/syslog > /tmp/test" all of the servers between 11:43:26 and 11:45:38 (2m12s). In that time:
dhcp-[a,b]-01 had roughly 2700 lines dhcp-[a-b]-02 had roughly 11000 lines dhcp-[a-b]-03 had roughly 1100 lines
So to me it seems like there's some sort of throttling on the data that's able to be sent. There's ~5x more traffic on pair 2 than 1 (which will be rebalanced, just trying to get this working first) so that would make sense. The only thing that I could find that looks like it would help is the log_fifo_size option, but that doesn't seem to help -- I've made several adjustments to it, but it doesn't seem to make any difference.
Can someone please let me know what I'm missing? Thanks!
Brian