Thank you for the quick reply.
Sorry for not mention, since are starting this now, we used the latest version, so syslog-ng 2.0.6...
We looked at the stats (set to 10s), and saw the following:
$ ./syslog-ng -v -e -F -f /tmp/syslog-
ng.conf.solaris
syslog-ng starting up; version='2.0.6'
Log statistics; processed='center(queued)=0', processed='center(received)=0', processed='destination(all)=0', processed='source(local)=0'
Log statistics; processed='center(queued)=0', processed='center(received)=1', processed='destination(all)=0', processed='source(local)=1'
Log statistics; processed='center(queued)=0', processed='center(received)=1', processed='destination(all)=0', processed='source(local)=1'
Initializing destination file writer; template='/tmp/messages-ng', filename='/tmp/messages-ng'
Log statistics; processed='center(queued)=46862', processed='center(received)=46863', processed='destination(all)=46862', processed='source(local)=46863'
Log statistics; processed='center(queued)=136634', processed='center(received)=136635', processed='destination(all)=136634', processed='source(local)=136635'
<...>
Log statistics; processed='center(queued)=578629', processed='center(received)=578630', processed='destination(all)=578629', processed='source(local)=578630'
Log statistics; processed='center(queued)=578629', processed='center(received)=578630', processed='destination(all)=578629', processed='source(local)=578630'
Log statistics; processed='center(queued)=578629', processed='center(received)=578630', processed='destination(all)=578629', processed='source(local)=578630'
Our test program (same host, a Sun Fire v440 running Solaris 10) had tried to log 1000000 messages. I guess this means the loss is before it reached Syslog-ng?
Do you have any idea how we can get around this? The more logging-intensive applications, have their sources available to us, a few however don't, so maybe combining some other source with /dev/log could help?
Any and all ideas are welcome!
Best regards,
Andrew
The syslog-ng version would be a useful information.
On Fri, 2007-12-21 at 09:14 +0100, Andrew Séguin wrote:
> Hello,
>
> I'm sorry if I'm missing something obvious, admittedly I've only had
> the chance to quickly search through the manuals and mailing list and
> not read them as deep as I'd like to, yet.
>
> I'm working in a project where we were considering replacing syslog to
> take advantage of reliable transmission over tcp. Performance is a
> factor for us, so we wrote a short program that simply loops and logs
> a counter and time stamp (to avoid "last message repeated x
> messages"). We ran it with syslog (saving locally to a file, and then
> sending remotely to another station which is logging to file) and got
> some reference numbers for the hardware/OS (Solaris 10). We then
> started the same test with syslog-ng. Performance didn't get to be an
> issue: under the pressure of the performance test, only some 5-600,000
> lines are logged although 1,000,000 were sent!
>
> We tried a few tweaks to the configuration file for buffering (see
> below), but it hasn't helped unfortunately. Is there a way to avoid
> this problem? Will we encounter this same problem on the remote host
> (considering the remote/logging host is planned to be accepting
> messages from two servers with a lot of traffic)
>
> Thanks for any tips/info!
> Andrew Seguin
>
>
> ps: here is the configuration file we have used...
>
> #
> # Syslog-ng example configuration file for Solaris
> #
> # Copyright (c) 1999 Balazs Scheidler
> # $Id: syslog-ng.conf.solaris,v 1.2 1999/11/15 12:30:41 bazsi Exp $
> #
> # Solaris 2.5.1 and below uses the STREAMS driver, above extends it
> # with doors. For 2.5.1 remove the door() option from the source
> declaration.
> #
>
> options {
> sync (0);
> log_fifo_size (65535);
> gc_idle_threshold(30); gc_busy_threshold(3000);
> };
>
> source local { sun-streams("/dev/log" door("/etc/.syslog_door"));
> internal(); };
>
> destination all { file("/tmp/messages-ng" log_fifo_size(60000)); };
>
> filter filter_local6 { facility(local6); };
> log { source(local); filter(filter_local6); destination(all); };
What you need to find out where the lossage occurs, it might happen on
the /dev/log device, or inside syslog-ng.
To find out whether it's the latter case, please check the "Log
statistics" message (or STATS in syslog-ng 1.6.x). If the drop counters
are zero, then it is the streams device which is dropping messages.
I don't remember all the STREAMS details whether it can lose messages,
but before digging any further it'd be useful to know where the messages
get actually lost.
--
Bazsi
_______________________________________________
syslog-ng maillist - syslog-ng@lists.balabit.hu
https://lists.balabit.hu/mailman/listinfo/syslog-ng
Frequently asked questions at http://www.campin.net/syslog-ng/faq.html