Thank you for the quick reply. Sorry for not mention, since are starting this now, we used the latest version, so syslog-ng 2.0.6... We looked at the stats (set to 10s), and saw the following: $ ./syslog-ng -v -e -F -f /tmp/syslog-ng.conf.solaris syslog-ng starting up; version='2.0.6' Log statistics; processed='center(queued)=0', processed='center(received)=0', processed='destination(all)=0', processed='source(local)=0' Log statistics; processed='center(queued)=0', processed='center(received)=1', processed='destination(all)=0', processed='source(local)=1' Log statistics; processed='center(queued)=0', processed='center(received)=1', processed='destination(all)=0', processed='source(local)=1' Initializing destination file writer; template='/tmp/messages-ng', filename='/tmp/messages-ng' Log statistics; processed='center(queued)=46862', processed='center(received)=46863', processed='destination(all)=46862', processed='source(local)=46863' Log statistics; processed='center(queued)=136634', processed='center(received)=136635', processed='destination(all)=136634', processed='source(local)=136635' <...> Log statistics; processed='center(queued)=578629', processed='center(received)=578630', processed='destination(all)=578629', processed='source(local)=578630' Log statistics; processed='center(queued)=578629', processed='center(received)=578630', processed='destination(all)=578629', processed='source(local)=578630' Log statistics; processed='center(queued)=578629', processed='center(received)=578630', processed='destination(all)=578629', processed='source(local)=578630' Our test program (same host, a Sun Fire v440 running Solaris 10) had tried to log 1000000 messages. I guess this means the loss is before it reached Syslog-ng? Do you have any idea how we can get around this? The more logging-intensive applications, have their sources available to us, a few however don't, so maybe combining some other source with /dev/log could help? Any and all ideas are welcome! Best regards, Andrew On Dec 21, 2007 10:09 AM, Balazs Scheidler <bazsi@balabit.hu> wrote:
On Fri, 2007-12-21 at 09:14 +0100, Andrew Séguin wrote:
Hello,
I'm sorry if I'm missing something obvious, admittedly I've only had the chance to quickly search through the manuals and mailing list and not read them as deep as I'd like to, yet.
I'm working in a project where we were considering replacing syslog to take advantage of reliable transmission over tcp. Performance is a factor for us, so we wrote a short program that simply loops and logs a counter and time stamp (to avoid "last message repeated x messages"). We ran it with syslog (saving locally to a file, and then sending remotely to another station which is logging to file) and got some reference numbers for the hardware/OS (Solaris 10). We then started the same test with syslog-ng. Performance didn't get to be an issue: under the pressure of the performance test, only some 5-600,000 lines are logged although 1,000,000 were sent!
We tried a few tweaks to the configuration file for buffering (see below), but it hasn't helped unfortunately. Is there a way to avoid this problem? Will we encounter this same problem on the remote host (considering the remote/logging host is planned to be accepting messages from two servers with a lot of traffic)
Thanks for any tips/info! Andrew Seguin
ps: here is the configuration file we have used...
# # Syslog-ng example configuration file for Solaris # # Copyright (c) 1999 Balazs Scheidler # $Id: syslog-ng.conf.solaris,v 1.2 1999/11/15 12:30:41 bazsi Exp $ # # Solaris 2.5.1 and below uses the STREAMS driver, above extends it # with doors. For 2.5.1 remove the door() option from the source declaration. #
options { sync (0); log_fifo_size (65535); gc_idle_threshold(30); gc_busy_threshold(3000); };
source local { sun-streams("/dev/log" door("/etc/.syslog_door")); internal(); };
destination all { file("/tmp/messages-ng" log_fifo_size(60000)); };
filter filter_local6 { facility(local6); }; log { source(local); filter(filter_local6); destination(all); };
The syslog-ng version would be a useful information.
What you need to find out where the lossage occurs, it might happen on the /dev/log device, or inside syslog-ng.
To find out whether it's the latter case, please check the "Log statistics" message (or STATS in syslog-ng 1.6.x). If the drop counters are zero, then it is the streams device which is dropping messages.
I don't remember all the STREAMS details whether it can lose messages, but before digging any further it'd be useful to know where the messages get actually lost.
-- Bazsi
_______________________________________________ syslog-ng maillist - syslog-ng@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html