June 2003 - syslog-ng - lists.balabit.hu

RE: [syslog-ng]What means "dropped" ?
by Loubet Jean-Michel 10 Jun '03

10 Jun '03

> the relevant part is what kind of destinations you have. (e.g. the rest of > the config) destination std { file("/var/log/HOSTS/$HOST/$YEAR/$MONTH/$DAY/syslog" owner(admsys) group(system) perm(0600) dir_perm(0700) create_dirs(yes) template ("$DATE $FULLHOST $FACILITY.$PRIORITY $MSG\n") ); }; log { source(src); destination(std); }; destination std_syslog_collector { file("/var/log/syslog_collector" owner(admsys) group(system) perm(0600) dir_perm(0700) create_dirs(yes) template ("LOG $FULLHOST $FACILITY.$PRIORITY $DATE $MSG\n") ); }; log { source(src); destination(std_syslog_collector); }; JM Loubet

2 1

Re:[syslog-ng]How do I adjust logging for "foreign" facilites?
by Wayne Sweatt 10 Jun '03

10 Jun '03

Balasz, I'm not sure how much of your reply is use of a built-in methods or just examples. Is the "process" keyword an undocumented method, or just very new? I've solved the problem with the CRON logs, by using the match() method. Every log I received consistently had one of three keywords. The only remaining facilities I need to deal with are the Darwin - "netinfo", and Linux, etc - "authpriv". I am not able to filter by OS Type, since the host names are irrelevant to OS. From the logs that I've collected, I can see that I can safely translate all incoming hex "c" facility values to "netinfo" and could also safely replace all hex "a" facilities to "authpriv". No Solaris systems/apps are using those values anywhere on our network. I am able to use the pipe() method, but that means running a dedicated, non-syslog-ng process in the background to write to the logs. I'm not crazy about that convaluted scheme. I haven't tried the program() method yet. Would that be the ticket for me if I want to just replace strings? Could I write a simple Perl loop that replaces the priority string on <STDIN> and then writes out to the desired log file? You use a "facility_rewrite_hack" in your example. How/where would that be defined? Thanks, -Wayne On Mon, Jun 02, 2003 at 05:03:45PM -0600, Wayne Sweatt wrote: > I am running syslog-ng-1.6.0rc2+20030423 on Solaris 8. > I see a problem with the udp logging of facilities from non-Solaris > clients. > For example, syslog-ng is storing the authpriv facility from Linux > clients as "a" as the $FACILITY value, and cron as "9". > It also sets the value of "a" for MacOS X client's netinfo facility. > I can understand strange values for the "non_Solaris" facilites - > authpriv and netinfo, but . the cron value is set correctly from Solaris > clients while not for Linux clients. I thought these numeric Facility > codes were "standard' between OSes. > Any explanations? It should be, however it is not the case. Syslog-ng uses the values whatever platform it was compiled for. The table on Linux is: #define LOG_KERN (0<<3) /* kernel messages */ #define LOG_USER (1<<3) /* random user-level messages */ #define LOG_MAIL (2<<3) /* mail system */ #define LOG_DAEMON (3<<3) /* system daemons */ #define LOG_AUTH (4<<3) /* security/authorization messages */ #define LOG_SYSLOG (5<<3) /* messages generated internally by syslogd */ #define LOG_LPR (6<<3) /* line printer subsystem */ #define LOG_NEWS (7<<3) /* network news subsystem */ #define LOG_UUCP (8<<3) /* UUCP subsystem */ #define LOG_CRON (9<<3) /* clock daemon */ #define LOG_AUTHPRIV (10<<3) /* security/authorization messages (private) */ #define LOG_FTP (11<<3) /* ftp daemon */ /* other codes through 15 reserved for system use */ #define LOG_LOCAL0 (16<<3) /* reserved for local use */ #define LOG_LOCAL1 (17<<3) /* reserved for local use */ #define LOG_LOCAL2 (18<<3) /* reserved for local use */ #define LOG_LOCAL3 (19<<3) /* reserved for local use */ #define LOG_LOCAL4 (20<<3) /* reserved for local use */ #define LOG_LOCAL5 (21<<3) /* reserved for local use */ #define LOG_LOCAL6 (22<<3) /* reserved for local use */ #define LOG_LOCAL7 (23<<3) /* reserved for local use */ > > Is there a suggested way to convert these values at runtime using > filters(?), or maybe there an easy way to re-compile this functionality > in for Solaris? > The logs are arriving and being stored just fine, it's just the > $FACILITY value that I would like to translate back to it's native > "name" for clean parsing reasons. It is not currently possible. It is not difficult to translate facility values, the difficult thing is to select which translation table to apply (as each message might origin from a different platform) Using a hack like: filter f_solaris { host("^sol$") and facility_rewrite_hack(0x3, 0x9); }; this would result in facility_rewrite_hack() called in case the first part of the filter matches. A more elegant solution would be to introduce the process keyword: filter f_solaris { host("^sol$"); }; process p_remap_sol_fac { facility_map(0x3, 0x9); }; log { source(s_net); filter(f_solaris); process(p_remap_sol_fac); destination(d_files); }; Wayne Sweatt Sr. UNIX System Administrator Comforce Technical Services LANL SCC Team

2 1

Log Rotation
by Jim Gifford 10 Jun '03

10 Jun '03

I tried searching google for this information. How can I setup syslog-ng to rotate logs by date??

2 1

sending e-mail
by netsec novice 10 Jun '03

10 Jun '03

I am using postfix and did have it running when I originally set up my Linux box almost a year ago but I am now attempting to send e-mails as a destination with no luck. My brain has obviously archived the details of my original struggle. I don't need a full SMTP server - just something that will relay to our internal SMTP host. Anyone have suggestions for resources on how to setup and test postfix. Thanks in advance, Nicole _________________________________________________________________ MSN 8 helps eliminate e-mail viruses. Get 2 months FREE*. http://join.msn.com/?page=features/virus

2 1

time() system calls slow down syslog-ng
by Loubet Jean-Michel 10 Jun '03

10 Jun '03

Hi, truss -c on syslog-ng 1.6.0rc3 indicates there are about 5 "time()" calls for 1 received message. I've tried to change "use_time_received" but there was so many "time()" calls. The problem is that this call was responsible for the high cpu load on my host (and for my messages losing ...) After replacing time() calls in libol and syslog-ng with a constant, my syslog-ng accepted much more messages with lower cpu usage. Is there any way to reduce time system calls ? Here are my options, if you need them : keep_hostname(yes); long_hostnames(no); log_fifo_size(8192); log_msg_size (2048); sync(1024); use_time_recvd(no); use_dns(no); use_fqdn(no); dns_cache(no); stats (0); (I also set udp recv_hiwat to 65536). Thanks for your answers, JM Loubet

1 0

Re: [syslog-ng]Log Rotation
by Dan Edwards 10 Jun '03

10 Jun '03

Add a destination using macros. Here is an example destination. destination d_example { file("/var/logs/$HOST/$HOST.$YEAR-$MONTH-$DAY.txt"); }; That would put the data in a file based on the name of the device sending it and the current date. Example ("/var/logs/device1/device1.2003-06-07.txt"). It doesn't per say rotate the log but it will give you new logs based off day/month etc.. ---------- Original Message ---------------------------------- From: "Jim Gifford" <maillist(a)jg555.com> Reply-To: syslog-ng(a)lists.balabit.hu Date: Mon, 9 Jun 2003 19:22:58 -0700 >I tried searching google for this information. > >How can I setup syslog-ng to rotate logs by date?? >

1 0

RE: [syslog-ng]Log monitoring
by Marc Mamane 09 Jun '03

09 Jun '03

Try this... http://www.estpak.ee/~risto/sec/ Marc Mamane GuardedNet, Inc. -----Original Message----- From: netsec novice [mailto:netsec9@hotmail.com] Sent: Monday, June 09, 2003 1:42 PM To: syslog-ng(a)lists.balabit.hu Subject: [syslog-ng]Log monitoring I am looking for a tool that would allow me to perform an action(send e-mail) when a particular event meets a threshhold. I have my IDS tuned to the point where I have a good sense of how many alerts I receive in an hour. I know I can send an alert based on matching a particular alert but what I would really like to do is send notification based on whether I receive more than 10 alerts in less than an hour. I hope my intention is clear here... I know there are products out there such as Swatch or logwatch but I haven't seen anything that alerts on thresholds rather than pattern matching only. My idea here is to set up something that watches my logs continuously and if I get more than 10 alerts within an hour or less during any part of the day - I would be paged. I am not a Perl guru so any help I can get in getting started is appreciated. My guess is that someone has already invented the wheel - I just don't know where it is. Thanks for any guidance... Nicole _________________________________________________________________ Add photos to your e-mail with MSN 8. Get 2 months FREE*. http://join.msn.com/?page=features/featuredemail _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html

2 1

log file permission
by Moktar KONE 09 Jun '03

09 Jun '03

I am using syslog-ng-1.6.0rc3 I want to create my log file with 777 permission i allready checked out the perm line and I had this line: perm(777); in the global options. but the file is creating with an other permission!!

1 0

Re: [syslog-ng]Log monitoring
by netsec novice 09 Jun '03

09 Jun '03

Thanks so much for sharing. That should give me a good start. Cheers, Nicole >From: Nicholas Bernstein <nick(a)docmagic.com> >Reply-To: syslog-ng(a)lists.balabit.hu >To: syslog-ng(a)lists.balabit.hu >Subject: Re: [syslog-ng]Log monitoring >Date: 09 Jun 2003 11:54:53 -0700 > >there should probably be "" around the $(date ...) in the grep now that >I think about it. > >-Nick > >On Mon, 2003-06-09 at 11:51, Nicholas Bernstein wrote: > > Ok.... > > > > something like : > > grep "Jun 9 11" messages| grep ftpd > > will get you all of the messages containing ftpd on June 9th, during the > > 11th hour. if you wanted to be inexact you can get the date by doing: > > date | awk -F : '{ print $1 }' > > so to put it all together you could do : > > grep $(date | awk -F : '{ print $1 }') <file> | <grep whatever your > > looking for> | wc -l > > > > or somesuch... > > > > If I was going to really do it, I'd probably use perl. To be accurate, > > you would want to convert the date into minutes, and then do a grep > > based only look at the entries within "Now - (Minutes)" otherwise, it > > effectivly resets the counter every hour. > > > > We aren't doing the exact same thing, otherwise I would try to be of > > more help, but I'm guessing that should be enough to get you started. > > > > > > man sed > > man awk > > man wc > > man bash (of course) > > http://www.perldoc.com > > > > > > > > On Mon, 2003-06-09 at 11:01, netsec novice wrote: > > > It sounds like what I'm looking for. Would you mind sending me more > > > details? I am a relative newbie to syslog and linux. I appreciate >anything > > > you can share. > > > > > > Nicole > > > > > > > > > >From: Nicholas Bernstein <nick(a)docmagic.com> > > > >Reply-To: syslog-ng(a)lists.balabit.hu > > > >To: syslog-ng(a)lists.balabit.hu > > > >Subject: Re: [syslog-ng]Log monitoring > > > >Date: 09 Jun 2003 10:56:57 -0700 > > > > > > > >We have a similar circumstance, where we basically have our logs > > > >filtered, and the events that we want put into a separate log file. >Then > > > >we grep the log based on time stamp & count the number of lines. The > > > >counter runs from cron every minute, and sends out an email if the > > > >number is higher than the threshold. > > > > > > > >Hope that helps! > > > >Nick > > > > > > > >On Mon, 2003-06-09 at 10:42, netsec novice wrote: > > > > > I am looking for a tool that would allow me to perform an >action(send > > > > > e-mail) when a particular event meets a threshhold. I have my IDS >tuned > > > >to > > > > > the point where I have a good sense of how many alerts I receive >in an > > > >hour. > > > > > I know I can send an alert based on matching a particular alert >but > > > >what I > > > > > would really like to do is send notification based on whether I >receive > > > >more > > > > > than 10 alerts in less than an hour. I hope my intention is clear > > > >here... > > > > > I know there are products out there such as Swatch or logwatch but >I > > > >haven't > > > > > seen anything that alerts on thresholds rather than pattern >matching > > > >only. > > > > > My idea here is to set up something that watches my logs >continuously > > > >and if > > > > > I get more than 10 alerts within an hour or less during any part >of the > > > >day > > > > > - I would be paged. I am not a Perl guru so any help I can get in > > > >getting > > > > > started is appreciated. My guess is that someone has already >invented > > > >the > > > > > wheel - I just don't know where it is. > > > > > > > > > > Thanks for any guidance... > > > > > Nicole > > > > > > > > > > _________________________________________________________________ > > > > > Add photos to your e-mail with MSN 8. Get 2 months FREE*. > > > > > http://join.msn.com/?page=features/featuredemail > > > > > > > > > > _______________________________________________ > > > > > syslog-ng maillist - syslog-ng(a)lists.balabit.hu > > > > > https://lists.balabit.hu/mailman/listinfo/syslog-ng > > > > > Frequently asked questions at >http://www.campin.net/syslog-ng/faq.html > > > >-- > > > >+---------------------------------------------------------------+ > > > >| Nicholas Bernstein | nick(a)docmagic.com | > > > >| UNIX Systems Administrator | http://www.docmagic.com | > > > >| Document Systems Inc. | | > > > >+---------------------------------------------------------------+ > > > > > > > >_______________________________________________ > > > >syslog-ng maillist - syslog-ng(a)lists.balabit.hu > > > >https://lists.balabit.hu/mailman/listinfo/syslog-ng > > > >Frequently asked questions at >http://www.campin.net/syslog-ng/faq.html > > > > > > > > > > _________________________________________________________________ > > > Add photos to your messages with MSN 8. Get 2 months FREE*. > > > http://join.msn.com/?page=features/featuredemail > > > > > > _______________________________________________ > > > syslog-ng maillist - syslog-ng(a)lists.balabit.hu > > > https://lists.balabit.hu/mailman/listinfo/syslog-ng > > > Frequently asked questions at http://www.campin.net/syslog-ng/faq.html >-- >+---------------------------------------------------------------+ >| Nicholas Bernstein | nick(a)docmagic.com | >| UNIX Systems Administrator | http://www.docmagic.com | >| Document Systems Inc. | | >+---------------------------------------------------------------+ > >_______________________________________________ >syslog-ng maillist - syslog-ng(a)lists.balabit.hu >https://lists.balabit.hu/mailman/listinfo/syslog-ng >Frequently asked questions at http://www.campin.net/syslog-ng/faq.html > _________________________________________________________________ Tired of spam? Get advanced junk mail protection with MSN 8. http://join.msn.com/?page=features/junkmail

1 0

Re: [syslog-ng]Log monitoring
by netsec novice 09 Jun '03

09 Jun '03

It sounds like what I'm looking for. Would you mind sending me more details? I am a relative newbie to syslog and linux. I appreciate anything you can share. Nicole >From: Nicholas Bernstein <nick(a)docmagic.com> >Reply-To: syslog-ng(a)lists.balabit.hu >To: syslog-ng(a)lists.balabit.hu >Subject: Re: [syslog-ng]Log monitoring >Date: 09 Jun 2003 10:56:57 -0700 > >We have a similar circumstance, where we basically have our logs >filtered, and the events that we want put into a separate log file. Then >we grep the log based on time stamp & count the number of lines. The >counter runs from cron every minute, and sends out an email if the >number is higher than the threshold. > >Hope that helps! >Nick > >On Mon, 2003-06-09 at 10:42, netsec novice wrote: > > I am looking for a tool that would allow me to perform an action(send > > e-mail) when a particular event meets a threshhold. I have my IDS tuned >to > > the point where I have a good sense of how many alerts I receive in an >hour. > > I know I can send an alert based on matching a particular alert but >what I > > would really like to do is send notification based on whether I receive >more > > than 10 alerts in less than an hour. I hope my intention is clear >here... > > I know there are products out there such as Swatch or logwatch but I >haven't > > seen anything that alerts on thresholds rather than pattern matching >only. > > My idea here is to set up something that watches my logs continuously >and if > > I get more than 10 alerts within an hour or less during any part of the >day > > - I would be paged. I am not a Perl guru so any help I can get in >getting > > started is appreciated. My guess is that someone has already invented >the > > wheel - I just don't know where it is. > > > > Thanks for any guidance... > > Nicole > > > > _________________________________________________________________ > > Add photos to your e-mail with MSN 8. Get 2 months FREE*. > > http://join.msn.com/?page=features/featuredemail > > > > _______________________________________________ > > syslog-ng maillist - syslog-ng(a)lists.balabit.hu > > https://lists.balabit.hu/mailman/listinfo/syslog-ng > > Frequently asked questions at http://www.campin.net/syslog-ng/faq.html >-- >+---------------------------------------------------------------+ >| Nicholas Bernstein | nick(a)docmagic.com | >| UNIX Systems Administrator | http://www.docmagic.com | >| Document Systems Inc. | | >+---------------------------------------------------------------+ > >_______________________________________________ >syslog-ng maillist - syslog-ng(a)lists.balabit.hu >https://lists.balabit.hu/mailman/listinfo/syslog-ng >Frequently asked questions at http://www.campin.net/syslog-ng/faq.html > _________________________________________________________________ Add photos to your messages with MSN 8. Get 2 months FREE*. http://join.msn.com/?page=features/featuredemail

2 2