<div>You make a good point. I initially thought the same thing and did some checking on the bandwidth usage and we aren't saturating any of the links or even getting close. I also didn't see any errors or drops on the interfaces. The big question for me is how does this all play out in the virtualized environment could I be running into a limitation there, rhetorical question. All of these hosts live physically on the same piece of hardware and on the same vlan. I'll keep poking around in that arena to see if anything turns up. Maybe play with tcp to the archive host, I just worry about performance implications.</div>
<div> </div>
<div>Do you see anything else in my options config that looks amiss?</div>
<div> </div>
<div>Thanks for the suggestion Joe.<br></div>
<div>Hardware stats:</div>
<div>relays:</div>
<div>2 3gig procs</div>
<div>4 gig mem</div>
<div>1 TB disk</div>
<div> </div>
<div>archive</div>
<div>4 3 gig procs</div>
<div>6 gig mem</div>
<div>5.5 TB disk</div>
<div> </div>
<div>Network bandwidth stats:</div>
<div>relay 01: in-850KBps out-300KBps (I'm assuming the descrepancy here is due to the fifo and flush settings.)</div>
<div>relay 02: in-60KBps out-55KBps</div>
<div>relay 03: in-nill out-nill</div>
<div> </div>
<div>Archive:</div>
<div>network utilization: 600KBps<br></div>
<div class="gmail_quote">On Wed, Jun 17, 2009 at 8:58 AM, Fegan, Joe <span dir="ltr"><<a href="mailto:Joe.Fegan@hp.com">Joe.Fegan@hp.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="PADDING-LEFT: 1ex; MARGIN: 0px 0px 0px 0.8ex; BORDER-LEFT: #ccc 1px solid">
<div>
<div dir="ltr" align="left"><span><font face="Arial" color="#0000ff" size="2">Knee jerk reaction: are you using udp? You probably know that udp is a connection-less, fire-and-forget protocol so if the packet gets lost neither the sender nor the intended recipent will know (or care).</font></span></div>
<br>
<div lang="en-us" dir="ltr" align="left">
<hr>
<font face="Tahoma" size="2"><b>From:</b> <a href="mailto:syslog-ng-bounces@lists.balabit.hu" target="_blank">syslog-ng-bounces@lists.balabit.hu</a> [mailto:<a href="mailto:syslog-ng-bounces@lists.balabit.hu" target="_blank">syslog-ng-bounces@lists.balabit.hu</a>] <b>On Behalf Of </b>Aaron Robel<br>
<b>Sent:</b> 17 June 2009 16:20<br><b>To:</b> <a href="mailto:syslog-ng@lists.balabit.hu" target="_blank">syslog-ng@lists.balabit.hu</a><br><b>Subject:</b> [syslog-ng] Syslog-ng 3.0.2 statistics<br></font><br></div>
<div>
<div></div>
<div class="h5">
<div></div>
<div>Hello,</div>
<div> </div>
<div>My apologies in advance, this is my first posting and I'm quite the rook' when it comes to Linux and Syslog-ng. I keep wondering why this is my project.</div>
<div> </div>
<div>I have a 4 server syslog deployment with 3 front end "relay" boxes and 1 backend archive box all within a virtualized SLES environment.</div>
<div> </div>
<div>Recently I noticed that the relay's together are averaging about 2500 messages per second (mps). The majority of the messages are coming from a single relay, about 2000 mps. Yet the archive box is only averaging about 400 mps.</div>
<div> </div>
<div>Since we are running 3.0.2 I decided to turn up the stats_level to (1). I don't see any drops to the about 150 file destinations that I've built.</div>
<div> </div>
<div>What does stamp, processed, stored, etc.. mean? I couldn't find any detailed documentation about the different statistics. </div>
<div> </div>
<div>Why am I getting such a large discrepency between "stamp" and "processed" in the log stats?</div>
<div> </div>
<div>Finally, since I'm sending the email does anyone see an issue with the way I've got the flow control set up in the global options?</div>
<div> </div>
<div>Here are my stats in question off my archive box:</div>
<div>processed='src.udp(s_network#0)=22020892', <br>stamp='src.udp(s_network#0)=1245249328'</div>
<div> </div>
<div>Here's the global's off the archive box:</div>
<div>options {<br> time_sleep(10);<br> log_fetch_limit(250);<br> log_fifo_size(2000);<br> use_dns(no);<br> keep_timestamp(yes);<br> dns_cache(no);<br> long_hostnames(off);<br>
flush_lines(2000);<br> flush_timeout(200);<br> perm(0644);<br> stats_freq(1800);<br> stats_level(1);<br> time_reopen(10);<br> create_dirs(yes);<br> dir_perm(755);<br>
};<br clear="all"></div>
<div></div>
<div>Thanks!</div>
<div> </div></div></div></div><br>______________________________________________________________________________<br>Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" target="_blank">https://lists.balabit.hu/mailman/listinfo/syslog-ng</a><br>
Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" target="_blank">http://www.balabit.com/support/documentation/?product=syslog-ng</a><br>FAQ: <a href="http://www.campin.net/syslog-ng/faq.html" target="_blank">http://www.campin.net/syslog-ng/faq.html</a><br>
<br><br></blockquote></div><br><br clear="all">
<div></div><br>-- <br>Aaron Robel<br>