March 2007 - syslog-ng - lists.balabit.hu

Syslog-ng log file overwrite
by Padmanabhan, Rajeesh (GE Healthcare) 09 Mar '07

09 Mar '07

Hi Bazsi, Do syslog-ng overwrites log file by default? If yes, what is the process/strategy done by syslog-ng to overwrite the file? Thanks, Rajeesh

1 0

Growing log files (i.e. access.log) and regex.
by Robert.Becker＠motoristsgroup.com 08 Mar '07

08 Mar '07

Here's the scenario. We have a JBoss server that outputs to *.log files. After doing all the research I could, I found out the only way to get these logs sent to syslog-ng was through a tail & logger command. The problem with that is logger appends the date and some other information to that logged message. When attempting to put that information in a mysql database I'm getting errors. So, what I would like to do is use some form of regex to alter the statement so it can be inserted into a database. There are basically two things that need to be changed. First, logger appends the date/time it receives the message and it also says it is coming from logger. I would like to remove that date and switch logger to jboss or something similar. Here is an example of what I am doing. This is a line from the jboss log file: 2005-10-11 13:14:15,848 DEBUG [org.jboss.resource.connectionmanager.IdleRemover] run: IdleRemover notifying pools, interval: 450000 Then I send the log through logger to syslog-ng: #!/bin/sh tail -f /opt/jboss/server/default/log/boot.log | logger -p local7.info The log gets turned into this: Oct 11 13:14:15 src@linuxp17 logger: 2005-10-11 13:14:15,848 DEBUG [org.jboss.resource.connectionmanager.IdleRemover] run: IdleRemover notifying pools, interval: 450000 What I need is: Oct 11 13:14:15 src@linuxp17 jboss: DEBUG [org.jboss.resource.connectionmanager.IdleRemover] run: IdleRemover notifying pools, interval: 450000 Now, I've thought of using awk in the tail/logger script to remove the date, but that does not take care of the logger/jboss substitution. Anyone have any ideas? Can syslog-ng do this with the match() function? -Rob Becker Systems Engineer Motorists Insurance ********************************************************************** The information contained in this message is confidential and is intended for the addressee(s) only. If you have received this message in error or there are any problems please notify the originator immediately. The unauthorized use, disclosure, copying or alteration of this message is strictly forbidden. Motorists Insurance Group will not be liable for direct, special, indirect or consequential damages arising from the alteration of the contents of this message by a third party or as a result of any virus being passed on. **********************************************************************

6 5

Unable to compile version 1.6.12 on Solaris 9
by roel＠raphor.net 08 Mar '07

08 Mar '07

I'm trying to compile syslog-ng-1.6.12 on Solaris 9. This aborts with the following error: gcc -g -O2 -Wall -I/usr/local/src/syslog-ng-1.6.12/libol-0.3.18/src -D_GNU_SOURCE -o syslog-ng main.o sources.o center.o filters.o destinations.o log.o cfgfile.o cfg-grammar.o cfg-lex.o affile.o afsocket.o afunix.o afinet.o afinter.o afuser.o afstreams.o afprogram.o afremctrl.o nscache.o utils.o syslog-names.o macros.o getopt.o getopt1.o -lpthread -lrt -lnsl -lsocket -ldoor -lresolv -Wl,-Bstatic /usr/local/src/syslog-ng-1.6.12/libol-0.3.18/src/.libs/libol.a -lsocket -lnsl -lxnet -ll -Wl,-Bdynamic ld: fatal: library -lxnet: not found ld: fatal: File processing errors. No output written to syslog-ng collect2: ld returned 1 exit status make[3]: *** [syslog-ng] Error 1 make[3]: Leaving directory `/usr/local/src/syslog-ng-1.6.12/src' make[2]: *** [all-recursive] Error 1 make[2]: Leaving directory `/usr/local/src/syslog-ng-1.6.12/src' make[1]: *** [all] Error 2 make[1]: Leaving directory `/usr/local/src/syslog-ng-1.6.12/src' make: *** [all-recursive] Error 1 Whereas syslog-ng-1.6.11 compiles without any problems. Any suggestions? Cheers, Roel

3 2

root logout is not tracked by syslog-ng
by Padmanabhan, Rajeesh (GE Healthcare) 08 Mar '07

08 Mar '07

Hi All, I have setup syslog-ng 2.0.1 on both Solaris & linux. I noticed that, syslog-ng is not tracking root log out over ssh or telnet. But other normal user log outs are tracked properly. Please help me to know if any configuration change is required to audit this. Thanks, Rajeesh

2 1

swatch in syslog-ng
by R. V. C. 07 Mar '07

07 Mar '07

List, I am trying (in vain) to get swatch to work with syslog-ng. Here is a list of my system specifics: Freebsd 6.2 syslog-ng-1.6.11 swatch-3.1.1_1 stunnel-4.20 mysql-client-5.1.14 mysql-server-5.1.14 Ok so here is the problem. I can get swatch to read a file via the tail-file option: destination swatch { program("/usr/local/bin/swatch --config-file=/root/.swatchrc --tail-file=/var/log/all.log"); }; This works fine. However when I try to use the read-pipe option, nothing ever comes out: destination swatch { program("/usr/local/bin/swatch --read-pipe=\"cat /dev/fd/0\""); }; I even tried using my mysql.pipe and creating a fifo pipe (swatch.pipe) just for swatch: destination swatch.pipe { pipe("/var/log/swatch.pipe" template("$FULLDATE $HOST $FACILITY:$PRIORITY $MSG\n")); }; destination swatch { program("/usr/local/bin/swatch --config-file=/root/.swatchrc --tail-file=/var/log/swatch.pipe"); }; No dice. My log statement is the same for all of them: log { source(logserver); filter(f_no_stats); destination(swatch); }; with logserver being: source logserver { tcp(ip(127.0.0.1) port(5141) keep-alive(yes) max_connections(100)); }; Which runs over stunnel. Everything else works fine, dumping to mysql, or to a flat file. I tried putting all the destinations on one line, and then separating them out still no dice. From what I've read on the web this is pretty easy to setup. Can anyone tell me what I am doing wrong. Thanks in advance. I am including a copy of my syslog-ng.conf file: # # FreeBSD /etc/syslog.conf file. # # # options # options { long_hostnames(off); keep_hostname(yes); bad_hostname("gconfd"); bad_hostname("^(ctld.|cmd|tmd|last)$"); log_fifo_size(4096); use_dns(yes); dns_cache(yes); time_reopen(10); stats(3600); sync(0); }; # # sources # source localhost { unix-dgram("/var/run/log"); unix-dgram("/var/run/logpriv" perm(0600)); #udp(); #tcp(ip(127.0.0.1) port(5141) keep-alive(yes) max_connections(100)); internal(); file("/dev/klog"); }; source logserver { tcp(ip(127.0.0.1) port(5141) keep-alive(yes) max_connections(100)); }; # # destinations # destination messages { file("/var/log/messages"); }; destination security { file("/var/log/security"); }; destination authlog { file("/var/log/auth.log"); }; destination maillog { file("/var/log/maillog"); }; destination lpd-errs { file("/var/log/lpd-errs"); }; destination xferlog { file("/var/log/xferlog"); }; destination cron { file("/var/log/cron"); }; destination debuglog { file("/var/log/debug.log"); }; destination consolelog { file("/var/log/console.log"); }; destination all { file("/var/log/all.log"); }; destination newscrit { file("/var/log/news/news.crit"); }; destination newserr { file("/var/log/news/news.err"); }; destination newsnotice { file("/var/log/news/news.notice"); }; destination slip { file("/var/log/slip.log"); }; destination ppp { file("/var/log/ppp.log"); }; destination console { file("/dev/console"); }; destination allusers { usertty("*"); }; # Custom destinations destination mysql { pipe("/var/log/mysql.pipe" template("INSERT INTO logs (host, facility, priority, level, tag, datetime, program, msg) VALUES ( '$HOST', '$FACILITY' , '$PRIORITY', '$LEVEL', '$TAG', '$YEAR-$MONTH-$DAY $HOUR:$MIN:$SEC', '$PROGRAM', '$MSG' );\n") template-escape(yes)); }; destination archive { file("/var/log/servers/$HOST/$YEAR/$MONTH/$DAY/$FACILITY/$HOST-$FACILITY.$ YEAR$MONTH$DAY" owner(root) group(root) perm(0600) dir_perm(0700) create_dirs(yes)); }; destination swatch { program("/usr/local/bin/swatch --read-pipe=\"cat /dev/fd/0\""); }; #destination swatch { program("/usr/local/bin/swatch --config-file=\"/root/.swatchrc\" --read-pipe=\"/bin/cat /dev/fd/0 \""); }; #destination swatch.pipe { pipe("/var/log/swatch.pipe" template("$FULLDATE $HOST $FACILITY:$PRIORITY $MSG\n")); }; #destination swatch { program("/usr/local/bin/swatch --config-file=/root/.swatchrc --tail-file=/var/log/all.log"); }; #destination sec { program("/usr/local/bin/sec -input=\"-\" -conf=/usr/local/etc/sec/general.sec"); }; # # log facility filters # filter f_auth { facility(auth); }; filter f_authpriv { facility(authpriv); }; filter f_not_authpriv { not facility(authpriv); }; filter f_console { facility(console); }; filter f_cron { facility(cron); }; filter f_daemon { facility(daemon); }; filter f_ftp { facility(ftp); }; filter f_kern { facility(kern); }; filter f_lpr { facility(lpr); }; filter f_mail { facility(mail); }; filter f_news { facility(news); }; filter f_security { facility(security); }; filter f_user { facility(user); }; filter f_uucp { facility(uucp); }; filter f_local0 { facility(local0); }; filter f_local1 { facility(local1); }; filter f_local2 { facility(local2); }; filter f_local3 { facility(local3); }; filter f_local4 { facility(local4); }; filter f_local5 { facility(local5); }; filter f_local6 { facility(local6); }; filter f_local7 { facility(local7); }; # # log level filters # filter f_emerg { level(emerg); }; filter f_alert { level(alert..emerg); }; filter f_crit { level(crit..emerg); }; filter f_err { level(err..emerg); }; filter f_warning { level(warning..emerg); }; filter f_notice { level(notice..emerg); }; filter f_info { level(info..emerg); }; filter f_debug { level(debug..emerg); }; filter f_is_debug { level(debug); }; # # program filters # filter f_ppp { program("ppp"); }; filter f_slip { program("startslip"); }; # # custom filters # filter f_no_stats { not match("STATS: dropped 0"); }; # # *.err;kern.warning;auth.notice;mail.crit /dev/console # log { source(localhost); filter(f_err); destination(console); }; log { source(localhost); filter(f_kern); filter(f_warning); destination(console); }; log { source(localhost); filter(f_auth); filter(f_notice); destination(console); }; log { source(localhost); filter(f_mail); filter(f_crit); destination(console); }; # # *.notice;authpriv.none;kern.debug;lpr.info;mail.crit;news.err /var/log/messages # log { source(localhost); filter(f_notice); filter(f_not_authpriv); destination(messages); }; log { source(localhost); filter(f_kern); filter(f_debug); destination(messages); }; log { source(localhost); filter(f_lpr); filter(f_info); destination(messages); }; log { source(localhost); filter(f_mail); filter(f_crit); destination(messages); }; log { source(localhost); filter(f_news); filter(f_err); destination(messages); }; # # security.* /var/log/security # log { source(localhost); filter(f_security); destination(security); }; # # auth.info;authpriv.info /var/log/auth.log log { source(localhost); filter(f_auth); filter(f_info); destination(authlog); }; log { source(localhost); filter(f_authpriv); filter(f_info); destination(authlog); }; # # mail.info /var/log/maillog # log { source(localhost); filter(f_mail); filter(f_info); destination(maillog); }; # # lpr.info /var/log/lpd-errs # log { source(localhost); filter(f_lpr); filter(f_info); destination(lpd-errs); }; # # ftp.info /var/log/xferlog # log { source(localhost); filter(f_ftp); filter(f_info); destination(xferlog); }; # # cron.* /var/log/cron # log { source(localhost); filter(f_cron); destination(cron); }; # # *.=debug /var/log/debug.log # log { source(localhost); filter(f_is_debug); destination(debuglog); }; # # *.emerg * # log { source(localhost); filter(f_emerg); destination(allusers); }; # # uncomment this to log all writes to /dev/console to /var/log/console.log # console.info /var/log/console.log # log { source(localhost); filter(f_console); filter(f_info); destination(consolelog); }; # # Log to mysql and the archive directory # log { source(logserver); filter(f_no_stats); destination(mysql); destination(archive); }; # # Swatch # #log { source(logserver); filter(f_no_stats); destination(all); }; log { source(logserver); filter(f_no_stats); destination(swatch); }; # # Local server # log { source(localhost); filter(f_no_stats); destination(archive); destination(mysql); }; # # DEBUG # #log { source(localhost); destination(all); }; And a copy of my .swatchrc watchfor /fail/ mail=robert.coward.ctr(a)deploymenthealth.osd.mil echo exec echo $0 | mail -s\"notice\" $0 R. V. C. R. V. C.

3 2

Looking for syslog log analyzer
by Daniel Chojecki 07 Mar '07

07 Mar '07

Hello, I use syslog-ng 2.0.2 with MySQL. I'm looking for analyzer program for my system. I've searched through syslog-ng maillist archives, google and found no app that analyze logs stored in MySQL. -- Greetz Daniel

5 5

RE: [syslog-ng] Looking for syslog log analyzer
by Ivey, Chris 07 Mar '07

07 Mar '07

What are you trying to accomplish with analysis of your logs stored in MySQL? Give us a clear definition of what you are looking to do with the logs, and someone on the list may be able to provide you with some help that is properly applicable. Thanks! Chris Ivey Affiliated Computer Services Enterprise Management Integration Services Infrastructure Management Senior Analyst chris.ivey(a)acs-inc.com "When you find yourself in a hole, the best thing to do is stop digging!" -- Nick Stokes "I reject your reality, and substitute my own!" -- Adam Savage _____ From: syslog-ng-bounces(a)lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Daniel Chojecki Sent: Tuesday, March 06, 2007 3:25 PM To: syslog-ng(a)lists.balabit.hu Subject: [syslog-ng] Looking for syslog log analyzer Hello, I use syslog-ng 2.0.2 with MySQL. I'm looking for analyzer program for my system. I've searched through syslog-ng maillist archives, google and found no app that analyze logs stored in MySQL. -- Greetz Daniel

1 0

compilation errors with --enable-spoof-source
by Ravi Papisetti -X (rpapiset - HCL＠Cisco) 06 Mar '07

06 Mar '07

Hi, I am using syslog-ng 1.6.11 and trying to compile this package with ./configure --enable-spoof-source, it throws errors as below checking whether to enable Sun STREAMS support... yes checking whether to enable Sun door support... yes checking whether to enable TCP wrapper support... no checking whether to enable spoof_source support... ./configure: test: too many arguments configure: error: libnet-config not found It compiles fine without --enable-spoof-source this option. Could you do the needful. Thanks, Ravi Kumar P.

3 11

Re: [syslog-ng] MySQL syslog-ng regex
by Fat Wallet 06 Mar '07

06 Mar '07

Thanks Bazsi, I'd appreciate if you could send me a patch. Currently, I need to parse upto 18 arguments to before inserting into MySQL database. muchas gracius. ----- Original Message ---- From: Balazs Scheidler <bazsi(a)balabit.hu> To: Syslog-ng users' and developers' mailing list <syslog-ng(a)lists.balabit.hu> Sent: Thursday, March 1, 2007 2:38:32 AM Subject: Re: [syslog-ng] MySQL syslog-ng regex On Thu, 2007-03-01 at 10:24 +0000, Hari Sekhon wrote: > one more thought, did you try ${10} or something? > > This works in shell so it's worth a try. currently syslog-ng supports max $9. It's not too difficult to add more, I'll look into it. -- Bazsi _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html ____________________________________________________________________________________ Need Mail bonding? Go to the Yahoo! Mail Q&A for great tips from Yahoo! Answers users. http://answers.yahoo.com/dir/?link=list&sid=396546091

3 5

Incorrect message: SIGTERM received
by bryanh＠giraffe-data.com 05 Mar '07

05 Mar '07

If you send a SIGINT signal to Syslog-ng, it issues the message SIGTERM received, terminating Looking at the code, it uses the same code for both SIGTERM and SIGINT. I suggest the message be reworded as SIGTERM or SIGINT received, terminating -- Bryan Henderson San Jose, California

2 2