[syslog-ng]newbie Solaris question

[Hamilton, Andrew Mr RAYTHEON 5 SIG CMD]

That helps some.  What is your log_fifo_size() set to?  I assume you are
looking at doing this over the network, yes?  In the original email you said
you were looking at low-volume situations so the suggestions we made were
based on that.  For higher volumes you may need to do other things and that
depends on your set-up.  Of course I/O system calls are very expensive
performance wise and a heavy amount of disk activity is going to slow you
down.  You will have to raise your sync some to buffer your messages and
slow down the writes.  The sync  number is the number of lines buffered
before write.  If you are sending 1000 lines per second try raising your
sync to 1000 and writing every second or so.  You'll have to play with the
numbers. Some of your performance issues are going to be hardware related,
slow disks, etc.   If you are logging to the machine that you are running
the program on log_fifo_size isn't going to help.  That is for slow
networks, not slow disks.  If you are writing your messages over the network
and you want to slow down your network writes you raise your log_fifo_size
on the client.  There is some memory usage cost associated with that but you
can save a considerable amount of slowdown on your log system.

Regards,
Drew

-----Original Message-----
From: T. Esting [mailto:T_Esting@excite.com]
Sent: Saturday, June 23, 2001 5:38 PM
To: HamiltonA@hq.5sigcmd.army.mil
Cc: syslog-ng@lists.balabit.hu
Subject: RE: [syslog-ng]newbie Solaris question 



  Drew, et al - 

  Thanks for writing back and for the sync() parameter suggestion.  I tried
setting sync(0) and that does indeed work for low-volume situations. 
However, the penalty for doing it this way seems to be exacted during
high-volume situations, e.g. several hundred messages per second.  I have a
rather simple test program that I run while looking at my output file:

#include <syslog.h>
#include <unistd.h>

main( int argc, char **argv) {

   if (argc == 2) {
        int i, loop = 0;
        loop = atoi(argv[1]);
        if (loop != 0) {
          openlog( "EER-TEST: ", LOG_PID, LOG_LOCAL5);
          for (i  = 1; i <= loop; i++) {
            syslog( LOG_INFO, "testing, 1, 2, 3 ... %d", i);
            }
        }
   }
}

This attempts to simulate an nmap scan or DoS-type attack against my
firewall; I am hoping to lose as few log messages as possible in these
scenarios.  With the stock Solaris syslog (brain-dead as it may be), I am
able to run this program with arguments of 1000 or 2000 and not lose any
messages.  After setting sync(0) so that I wouldn't lose any messages in
low-volume situations, I ran the same tests and, although syslog-ng kept up
with a burst of 100 or 200 messages, 1000 turned out to be way too much;
over 300 messages were lost.  I'm hoping to find the sweet spot for tuning
syslog-ng, since it obviously has many superior features to the standard
syslog daemon that ships with Solaris, but I'm running into some roadblocks.

TIA.

--------------------

What do you mean you can't get it to output every message it receives?  I 
have used syslog-ng on Solaris for 2 years with very few problems and I log 
over 1000 devices.  Do you know if it is receiving messages?  You're not 
running syslogd as well are you?  In low volumes the defaults for garbage 
collection should be sufficient.  How many hosts are you logging?  Try 
setting sync(0). 

Regards, 
Drew 

-----Original Message----- 
From: T. Esting [mailto:T_Esting@excite.com] 
Tuesday, June 12, 2001 6:16 PM 
To: syslog-ng@lists.balabit.hu 
Subject: [syslog-ng]newbie Solaris question 



I've been trying to get both 1.4.11 and 1.5.7 to run on my Solaris system, 
and have met with limited success.  Compiling and installing were fairly 
easy, as was creating a configuration file.  I understand from the docs that

the mark() feature still doesn't work, and from the mailing list that 
sync_freq() should actually be sync().  I am having trouble, however, with 
flushing output in low-volume scenarios.  Here is the configuration file I 
am using: 

                     options { 
                            use_dns(no); 
                            log_fifo_size(XXX);       
                            gc_idle_threshold(YYY);   
                            gc_busy_threshold(ZZZ); 
                            use_time_recvd(yes); 
                            }; 

                     source sun { 
                            sun-streams ( "/dev/log"
door("/etc/.syslog_door")); 
                            internal(); 
                     }; 

                     destination testout { 
                            file ( "/var/adm/testout.txt" sync(WWW)); 
                     } ; 


                     log {   source (sun); 
                            destination (testout); 
                     }; 

I've tried various values for WWW, XXX, YYY, and ZZZ, even setting WWW to 1.

I have not been able to influence syslog-ng to write out every message it 
receives.  Does anyone have any pointers on what I might want to try next to

get syslog-ng to more reliably receive and log messages on Solaris? 

Thanks. 





_______________________________________________________
Send a cool gift with your E-Card
http://www.bluemountain.com/giftcenter/