- syslog-ng - lists.balabit.hu

Logging network messages in Solaris
by Ripley, Scott 27 Jun '01

27 Jun '01

I have syslog-ng running successfully on several of our Solaris servers, but I can't get it to work with our logging server. I have the following config: source local { sun-streams("/dev/log" door("/etc/.syslog_door")); internal(); }; source s_tcp { tcp(ip(0.0.0.0)); }; source s_udp { udp(); }; <...filters removed...> log { source(local); filter (DEFAULT); destination(all); }; log { source(s_udp); filter (DEFAULT); destination(all); }; log { source(s_tcp); filter (DEFAULT); destination(all); }; I've simplified the config just to see if I can get messages to show up anywhere. It's not logging any messages from network devices, although it logs local messages just fine. I know the messages are getting there b/c syslogd is logging them and I've sniffed the network to verify UDP and port 514. I've also tried declaring the tcp and udp sources in the local definition. I know this can work. What am I missing? Scott Ripley DNS Administrator, House Information Resources 202.226.2833 - mailto:scott.ripley@mail.house.gov <mailto:scott.ripley@mail.house.gov>

2 1

Re: [syslog-ng]syslog entries without hostnames
by Balazs Scheidler 26 Jun '01

26 Jun '01

> I watched with interest the small discussion that I started on the mailing > list, but it never really reached a good conclusion. I would really > appreciate it if you could clarify a couple things.... > > When a syslog message arrives via UDP, can't you just record the IP address > of the host that sent it? Or is this somehow the job of the device that is > sending logs, to send it's hostname along? I am still trying to figure out > why when our Riverstone routers send logs, there is no device name or IP > address recorded. Our NOC engineers can't tell what device sent the > message... there are the following options to control hostname generation. keep_hostname() chain_hostname() keep_hostname tells syslog-ng not to modify received hostname field. if keep_hostname is off, it always overwrites received hostname, if chain_hostnames() if on, it appends the hostname the message was received from to the hostname, otherwise it replaces the received value. -- Bazsi PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1

1 0

logrotate
by colette tostivint 25 Jun '01

25 Jun '01

Hi everybody, I setup a syslog server (Linux Mandrake 7.0) in order to log my firewalls and routeurs in my network. I configure all systems and the file /etc/logrotate.d/syslog. Rotaion is ok but I must reboat my process syslog because logs are in file.1 and not in file Thanks Colette

1 0

RE: [syslog-ng]newbie Solaris question
by Hamilton, Andrew Mr RAYTHEON 5 SIG CMD 25 Jun '01

25 Jun '01

That helps some. What is your log_fifo_size() set to? I assume you are looking at doing this over the network, yes? In the original email you said you were looking at low-volume situations so the suggestions we made were based on that. For higher volumes you may need to do other things and that depends on your set-up. Of course I/O system calls are very expensive performance wise and a heavy amount of disk activity is going to slow you down. You will have to raise your sync some to buffer your messages and slow down the writes. The sync number is the number of lines buffered before write. If you are sending 1000 lines per second try raising your sync to 1000 and writing every second or so. You'll have to play with the numbers. Some of your performance issues are going to be hardware related, slow disks, etc. If you are logging to the machine that you are running the program on log_fifo_size isn't going to help. That is for slow networks, not slow disks. If you are writing your messages over the network and you want to slow down your network writes you raise your log_fifo_size on the client. There is some memory usage cost associated with that but you can save a considerable amount of slowdown on your log system. Regards, Drew -----Original Message----- From: T. Esting [mailto:T_Esting@excite.com] Sent: Saturday, June 23, 2001 5:38 PM To: HamiltonA(a)hq.5sigcmd.army.mil Cc: syslog-ng(a)lists.balabit.hu Subject: RE: [syslog-ng]newbie Solaris question Drew, et al - Thanks for writing back and for the sync() parameter suggestion. I tried setting sync(0) and that does indeed work for low-volume situations. However, the penalty for doing it this way seems to be exacted during high-volume situations, e.g. several hundred messages per second. I have a rather simple test program that I run while looking at my output file: #include <syslog.h> #include <unistd.h> main( int argc, char **argv) { if (argc == 2) { int i, loop = 0; loop = atoi(argv[1]); if (loop != 0) { openlog( "EER-TEST: ", LOG_PID, LOG_LOCAL5); for (i = 1; i <= loop; i++) { syslog( LOG_INFO, "testing, 1, 2, 3 ... %d", i); } } } } This attempts to simulate an nmap scan or DoS-type attack against my firewall; I am hoping to lose as few log messages as possible in these scenarios. With the stock Solaris syslog (brain-dead as it may be), I am able to run this program with arguments of 1000 or 2000 and not lose any messages. After setting sync(0) so that I wouldn't lose any messages in low-volume situations, I ran the same tests and, although syslog-ng kept up with a burst of 100 or 200 messages, 1000 turned out to be way too much; over 300 messages were lost. I'm hoping to find the sweet spot for tuning syslog-ng, since it obviously has many superior features to the standard syslog daemon that ships with Solaris, but I'm running into some roadblocks. TIA. -------------------- What do you mean you can't get it to output every message it receives? I have used syslog-ng on Solaris for 2 years with very few problems and I log over 1000 devices. Do you know if it is receiving messages? You're not running syslogd as well are you? In low volumes the defaults for garbage collection should be sufficient. How many hosts are you logging? Try setting sync(0). Regards, Drew -----Original Message----- From: T. Esting [mailto:T_Esting@excite.com] Tuesday, June 12, 2001 6:16 PM To: syslog-ng(a)lists.balabit.hu Subject: [syslog-ng]newbie Solaris question I've been trying to get both 1.4.11 and 1.5.7 to run on my Solaris system, and have met with limited success. Compiling and installing were fairly easy, as was creating a configuration file. I understand from the docs that the mark() feature still doesn't work, and from the mailing list that sync_freq() should actually be sync(). I am having trouble, however, with flushing output in low-volume scenarios. Here is the configuration file I am using: options { use_dns(no); log_fifo_size(XXX); gc_idle_threshold(YYY); gc_busy_threshold(ZZZ); use_time_recvd(yes); }; source sun { sun-streams ( "/dev/log" door("/etc/.syslog_door")); internal(); }; destination testout { file ( "/var/adm/testout.txt" sync(WWW)); } ; log { source (sun); destination (testout); }; I've tried various values for WWW, XXX, YYY, and ZZZ, even setting WWW to 1. I have not been able to influence syslog-ng to write out every message it receives. Does anyone have any pointers on what I might want to try next to get syslog-ng to more reliably receive and log messages on Solaris? Thanks. _______________________________________________________ Send a cool gift with your E-Card http://www.bluemountain.com/giftcenter/

1 0

Mail message not delivered
by MAILER-DAEMON＠colltech.com 25 Jun '01

25 Jun '01

Your message, titled: Re: [syslog-ng]newbie Solaris question that you sent to rdees(a)colltech.com has not been delivered. The user at that address is no longer available. The following alias addresses are also no longer available. dees_randy(a)colltech.com randy_dees(a)colltech.com Please update your address book Thank You

1 0

Mail message not delivered
by MAILER-DAEMON＠colltech.com 25 Jun '01

25 Jun '01

Your message, titled: [syslog-ng]Mail message not delivered that you sent to rdees(a)colltech.com has not been delivered. The user at that address is no longer available. The following alias addresses are also no longer available. dees_randy(a)colltech.com randy_dees(a)colltech.com Please update your address book Thank You

4 3

RE: [syslog-ng]newbie Solaris question
by T. Esting 25 Jun '01

25 Jun '01

Drew, et al - Thanks for writing back and for the sync() parameter suggestion. I tried setting sync(0) and that does indeed work for low-volume situations. However, the penalty for doing it this way seems to be exacted during high-volume situations, e.g. several hundred messages per second. I have a rather simple test program that I run while looking at my output file: #include <syslog.h> #include <unistd.h> main( int argc, char **argv) { if (argc == 2) { int i, loop = 0; loop = atoi(argv[1]); if (loop != 0) { openlog( "EER-TEST: ", LOG_PID, LOG_LOCAL5); for (i = 1; i <= loop; i++) { syslog( LOG_INFO, "testing, 1, 2, 3 ... %d", i); } } } } This attempts to simulate an nmap scan or DoS-type attack against my firewall; I am hoping to lose as few log messages as possible in these scenarios. With the stock Solaris syslog (brain-dead as it may be), I am able to run this program with arguments of 1000 or 2000 and not lose any messages. After setting sync(0) so that I wouldn't lose any messages in low-volume situations, I ran the same tests and, although syslog-ng kept up with a burst of 100 or 200 messages, 1000 turned out to be way too much; over 300 messages were lost. I'm hoping to find the sweet spot for tuning syslog-ng, since it obviously has many superior features to the standard syslog daemon that ships with Solaris, but I'm running into some roadblocks. TIA. -------------------- What do you mean you can't get it to output every message it receives? I have used syslog-ng on Solaris for 2 years with very few problems and I log over 1000 devices. Do you know if it is receiving messages? You're not running syslogd as well are you? In low volumes the defaults for garbage collection should be sufficient. How many hosts are you logging? Try setting sync(0). Regards, Drew -----Original Message----- From: T. Esting [mailto:T_Esting@excite.com] Tuesday, June 12, 2001 6:16 PM To: syslog-ng(a)lists.balabit.hu Subject: [syslog-ng]newbie Solaris question I've been trying to get both 1.4.11 and 1.5.7 to run on my Solaris system, and have met with limited success. Compiling and installing were fairly easy, as was creating a configuration file. I understand from the docs that the mark() feature still doesn't work, and from the mailing list that sync_freq() should actually be sync(). I am having trouble, however, with flushing output in low-volume scenarios. Here is the configuration file I am using: options { use_dns(no); log_fifo_size(XXX); gc_idle_threshold(YYY); gc_busy_threshold(ZZZ); use_time_recvd(yes); }; source sun { sun-streams ( "/dev/log" door("/etc/.syslog_door")); internal(); }; destination testout { file ( "/var/adm/testout.txt" sync(WWW)); } ; log { source (sun); destination (testout); }; I've tried various values for WWW, XXX, YYY, and ZZZ, even setting WWW to 1. I have not been able to influence syslog-ng to write out every message it receives. Does anyone have any pointers on what I might want to try next to get syslog-ng to more reliably receive and log messages on Solaris? Thanks. _______________________________________________________ Send a cool gift with your E-Card http://www.bluemountain.com/giftcenter/

2 1

RE: [syslog-ng]newbie Solaris question
by Hamilton, Andrew Mr RAYTHEON 5 SIG CMD 25 Jun '01

25 Jun '01

That helps some. What is your log_fifo_size() set to? I assume you are looking at doing this over the network, yes? In the original email you said you were looking at low-volume situations so the suggestions we made were based on that. For higher volumes you may need to do other things and that depends on your set-up. Of course I/O system calls are very expensive performance wise and a heavy amount of disk activity is going to slow you down. You will have to raise your sync some to buffer your messages and slow down the writes. The sync number is the number of lines buffered before write. If you are sending 1000 lines per second try raising your sync to 1000 and writing every second or so. You'll have to play with the numbers. Some of your performance issues are going to be hardware related, slow disks, etc. If you are logging to the machine that you are running the program on log_fifo_size isn't going to help. That is for slow networks, not slow disks. If you are writing your messages over the network and you want to slow down your network writes you raise your log_fifo_size on the client. There is some memory usage cost associated with that but you can save a considerable amount of slowdown on your log system. Regards, Drew -----Original Message----- From: T. Esting [mailto:T_Esting@excite.com] Sent: Saturday, June 23, 2001 5:38 PM To: HamiltonA(a)hq.5sigcmd.army.mil Cc: syslog-ng(a)lists.balabit.hu Subject: RE: [syslog-ng]newbie Solaris question Drew, et al - Thanks for writing back and for the sync() parameter suggestion. I tried setting sync(0) and that does indeed work for low-volume situations. However, the penalty for doing it this way seems to be exacted during high-volume situations, e.g. several hundred messages per second. I have a rather simple test program that I run while looking at my output file: #include <syslog.h> #include <unistd.h> main( int argc, char **argv) { if (argc == 2) { int i, loop = 0; loop = atoi(argv[1]); if (loop != 0) { openlog( "EER-TEST: ", LOG_PID, LOG_LOCAL5); for (i = 1; i <= loop; i++) { syslog( LOG_INFO, "testing, 1, 2, 3 ... %d", i); } } } } This attempts to simulate an nmap scan or DoS-type attack against my firewall; I am hoping to lose as few log messages as possible in these scenarios. With the stock Solaris syslog (brain-dead as it may be), I am able to run this program with arguments of 1000 or 2000 and not lose any messages. After setting sync(0) so that I wouldn't lose any messages in low-volume situations, I ran the same tests and, although syslog-ng kept up with a burst of 100 or 200 messages, 1000 turned out to be way too much; over 300 messages were lost. I'm hoping to find the sweet spot for tuning syslog-ng, since it obviously has many superior features to the standard syslog daemon that ships with Solaris, but I'm running into some roadblocks. TIA. -------------------- What do you mean you can't get it to output every message it receives? I have used syslog-ng on Solaris for 2 years with very few problems and I log over 1000 devices. Do you know if it is receiving messages? You're not running syslogd as well are you? In low volumes the defaults for garbage collection should be sufficient. How many hosts are you logging? Try setting sync(0). Regards, Drew -----Original Message----- From: T. Esting [mailto:T_Esting@excite.com] Tuesday, June 12, 2001 6:16 PM To: syslog-ng(a)lists.balabit.hu Subject: [syslog-ng]newbie Solaris question I've been trying to get both 1.4.11 and 1.5.7 to run on my Solaris system, and have met with limited success. Compiling and installing were fairly easy, as was creating a configuration file. I understand from the docs that the mark() feature still doesn't work, and from the mailing list that sync_freq() should actually be sync(). I am having trouble, however, with flushing output in low-volume scenarios. Here is the configuration file I am using: options { use_dns(no); log_fifo_size(XXX); gc_idle_threshold(YYY); gc_busy_threshold(ZZZ); use_time_recvd(yes); }; source sun { sun-streams ( "/dev/log" door("/etc/.syslog_door")); internal(); }; destination testout { file ( "/var/adm/testout.txt" sync(WWW)); } ; log { source (sun); destination (testout); }; I've tried various values for WWW, XXX, YYY, and ZZZ, even setting WWW to 1. I have not been able to influence syslog-ng to write out every message it receives. Does anyone have any pointers on what I might want to try next to get syslog-ng to more reliably receive and log messages on Solaris? Thanks. _______________________________________________________ Send a cool gift with your E-Card http://www.bluemountain.com/giftcenter/

1 0

Mail message not delivered
by MAILER-DAEMON＠colltech.com 25 Jun '01

25 Jun '01

Your message, titled: RE: [syslog-ng]newbie Solaris question that you sent to rdees(a)colltech.com has not been delivered. The user at that address is no longer available. The following alias addresses are also no longer available. dees_randy(a)colltech.com randy_dees(a)colltech.com Please update your address book Thank You

1 0

Mail message not delivered
by MAILER-DAEMON＠colltech.com 25 Jun '01

25 Jun '01

Your message, titled: [syslog-ng]Mail message not delivered that you sent to rdees(a)colltech.com has not been delivered. The user at that address is no longer available. The following alias addresses are also no longer available. dees_randy(a)colltech.com randy_dees(a)colltech.com Please update your address book Thank You

1 0

Mail message not delivered
by MAILER-DAEMON＠colltech.com 25 Jun '01

25 Jun '01

Your message, titled: [syslog-ng]Mail message not delivered that you sent to rdees(a)colltech.com has not been delivered. The user at that address is no longer available. The following alias addresses are also no longer available. dees_randy(a)colltech.com randy_dees(a)colltech.com Please update your address book Thank You

1 0

Mail message not delivered
by MAILER-DAEMON＠colltech.com 25 Jun '01

25 Jun '01

Your message, titled: [syslog-ng]Mail message not delivered that you sent to rdees(a)colltech.com has not been delivered. The user at that address is no longer available. The following alias addresses are also no longer available. dees_randy(a)colltech.com randy_dees(a)colltech.com Please update your address book Thank You

1 0

Mail message not delivered
by MAILER-DAEMON＠colltech.com 25 Jun '01

25 Jun '01

Your message, titled: [syslog-ng]Mail message not delivered that you sent to rdees(a)colltech.com has not been delivered. The user at that address is no longer available. The following alias addresses are also no longer available. dees_randy(a)colltech.com randy_dees(a)colltech.com Please update your address book Thank You

1 0

Mail message not delivered
by MAILER-DAEMON＠colltech.com 25 Jun '01

25 Jun '01

Your message, titled: [syslog-ng]Mail message not delivered that you sent to rdees(a)colltech.com has not been delivered. The user at that address is no longer available. The following alias addresses are also no longer available. dees_randy(a)colltech.com randy_dees(a)colltech.com Please update your address book Thank You

1 0

Mail message not delivered
by MAILER-DAEMON＠colltech.com 25 Jun '01

25 Jun '01

Your message, titled: [syslog-ng]Mail message not delivered that you sent to rdees(a)colltech.com has not been delivered. The user at that address is no longer available. The following alias addresses are also no longer available. dees_randy(a)colltech.com randy_dees(a)colltech.com Please update your address book Thank You

1 0

Mail message not delivered
by MAILER-DAEMON＠colltech.com 25 Jun '01

25 Jun '01

Your message, titled: [syslog-ng]Mail message not delivered that you sent to rdees(a)colltech.com has not been delivered. The user at that address is no longer available. The following alias addresses are also no longer available. dees_randy(a)colltech.com randy_dees(a)colltech.com Please update your address book Thank You

1 0

Mail message not delivered
by MAILER-DAEMON＠colltech.com 25 Jun '01

25 Jun '01

Your message, titled: [syslog-ng]Mail message not delivered that you sent to rdees(a)colltech.com has not been delivered. The user at that address is no longer available. The following alias addresses are also no longer available. dees_randy(a)colltech.com randy_dees(a)colltech.com Please update your address book Thank You

1 0

Mail message not delivered
by MAILER-DAEMON＠colltech.com 25 Jun '01

25 Jun '01

Your message, titled: [syslog-ng]Mail message not delivered that you sent to rdees(a)colltech.com has not been delivered. The user at that address is no longer available. The following alias addresses are also no longer available. dees_randy(a)colltech.com randy_dees(a)colltech.com Please update your address book Thank You

1 0

Mail message not delivered
by MAILER-DAEMON＠colltech.com 25 Jun '01

25 Jun '01

Your message, titled: [syslog-ng]Mail message not delivered that you sent to rdees(a)colltech.com has not been delivered. The user at that address is no longer available. The following alias addresses are also no longer available. dees_randy(a)colltech.com randy_dees(a)colltech.com Please update your address book Thank You

1 0

Mail message not delivered
by MAILER-DAEMON＠colltech.com 25 Jun '01

25 Jun '01

Your message, titled: [syslog-ng]Mail message not delivered that you sent to rdees(a)colltech.com has not been delivered. The user at that address is no longer available. The following alias addresses are also no longer available. dees_randy(a)colltech.com randy_dees(a)colltech.com Please update your address book Thank You

1 0

Mail message not delivered
by MAILER-DAEMON＠colltech.com 25 Jun '01

25 Jun '01

Your message, titled: [syslog-ng]Mail message not delivered that you sent to rdees(a)colltech.com has not been delivered. The user at that address is no longer available. The following alias addresses are also no longer available. dees_randy(a)colltech.com randy_dees(a)colltech.com Please update your address book Thank You

1 0

Mail message not delivered
by MAILER-DAEMON＠colltech.com 25 Jun '01

25 Jun '01

Your message, titled: [syslog-ng]Mail message not delivered that you sent to rdees(a)colltech.com has not been delivered. The user at that address is no longer available. The following alias addresses are also no longer available. dees_randy(a)colltech.com randy_dees(a)colltech.com Please update your address book Thank You

1 0

Mail message not delivered
by MAILER-DAEMON＠colltech.com 25 Jun '01

25 Jun '01

Your message, titled: [syslog-ng]Mail message not delivered that you sent to rdees(a)colltech.com has not been delivered. The user at that address is no longer available. The following alias addresses are also no longer available. dees_randy(a)colltech.com randy_dees(a)colltech.com Please update your address book Thank You

1 0

Mail message not delivered
by MAILER-DAEMON＠colltech.com 25 Jun '01

25 Jun '01

Your message, titled: [syslog-ng]Mail message not delivered that you sent to rdees(a)colltech.com has not been delivered. The user at that address is no longer available. The following alias addresses are also no longer available. dees_randy(a)colltech.com randy_dees(a)colltech.com Please update your address book Thank You

1 0

Mail message not delivered
by MAILER-DAEMON＠colltech.com 25 Jun '01

25 Jun '01

Your message, titled: [syslog-ng]Mail message not delivered that you sent to rdees(a)colltech.com has not been delivered. The user at that address is no longer available. The following alias addresses are also no longer available. dees_randy(a)colltech.com randy_dees(a)colltech.com Please update your address book Thank You

1 0