February 2005 - syslog-ng - lists.balabit.hu

new to syslog-ng
by Mike Pepe 17 Feb '05

17 Feb '05

Hello all, I'm new to syslog-ng. I have the program ready to go, but I was wondering if there's a faq or guide somewhere so I can just replace regular syslog functionality on Fedora Core 3. Once I get that working normally, I want to then log different events to different files. Any pointers would be appreciated. thanks! -Mike

5 7

Message Templates and Syslog ID
by Rhugga 16 Feb '05

16 Feb '05

Im trying to normalize the logs from my various systems I support (Solaris, IRIX, Tru64, SLES 8 & 9, RHAS 3, Cisco, Brocade, etc...) These logs are processed and stored into oracle. On Solaris, syslogd adds a field containing the syslog message ID and the facility and priority: EX: Jan 20 22:07:26 back-0202 tldd[3184]: [ID 817171 daemon.notice] Calling the drive_mount_notify ^^^^^^^^^^^^^^^^^^^^^^^^^ This behavior can only be changed by a reboot and sun also does not recommend setting msgid=0 in /kernerl/drv/log.conf. So, using the following template I can come close to normalizing this behavior: template("$DATE $HOST $PROGRAM [ID $FACILITY.$LEVEL] $MSG\n") The information that is lost is the internal syslog ID and the pid of the sending process. Here is a log entry processed thru this template: Feb 14 07:07:52 db-0201 su [ID auth.info] su: [ID 366847 auth.info] 'su oracle' succeeded for root on /dev/??? Another problem is that field in question now appears twice. Anyone know of any possible way to configure syslog-ng to solves this roadblock? Thx, CC

2 2

FIFO / pipe destination problem
by Howell, Matt 15 Feb '05

15 Feb '05

This may end up being something really easy that I am missing, but I have been struggling with it for a couple days. Googling the problem has yet to review any fruitful answers. I am trying to pass messages to MySQL via the pipe() function. I am working with syslog-ng 1.6.5-6 compiled from source for Fedora Core 3. I have setup a FIFO in the filesystem as follows: [root@furw tmp]# mkfifo mysql.pipe [root@furw tmp]# chmod 0600 mysql.pipe [root@furw tmp]# ls -al ... drwxrwxrwt 24 root root 4096 Feb 14 17:38 . drwxr-xr-x 23 root root 4096 Feb 11 16:00 .. prw------- 1 root root 0 Feb 14 17:38 mysql.pipe ... I then created my syslog-ng.conf file: ... options { sync (0); time_reopen (10); log_fifo_size (1000); long_hostnames (off); use_dns (no); use_fqdn (no); create_dirs (no); keep_hostname (yes); }; source src { unix-stream ("/dev/log"); internal(); udp(ip(0.0.0.0) port(514)); }; destination d_mysql { pipe("/tmp/mysql.pipe" template("INSERT INTO logs (host, facility, priority, level, tag, date, time, program, msg) VALUES ( '$HOST', '$FACILITY', '$PRIORITY', '$LEVEL', '$TAG', '$YEAR-$MONTH-$DAY', '$HOUR:$MIN:$SEC', '$PROGRAM', '$MSG' );\n") template-escape(yes)); }; destination d_cons { file("/dev/console"); }; destination d_mesg { file("/var/log/messages"); }; destination d_auth { file("/var/log/secure"); }; destination d_mail { file("/var/log/maillog" sync(10)); }; destination d_spol { file("/var/log/spooler"); }; destination d_boot { file("/var/log/boot.log"); }; destination d_cron { file("/var/log/cron"); }; destination d_mlal { usertty("*"); }; filter f_filter1 { facility(kern); }; filter f_filter2 { level(info..emerg) and not facility(mail,authpriv,cron); }; filter f_filter3 { facility(authpriv); }; filter f_filter4 { facility(mail); }; filter f_filter5 { level(emerg); }; filter f_filter6 { facility(uucp) or (facility(news) and level(crit..emerg)); }; filter f_filter7 { facility(local7); }; filter f_filter8 { facility(cron); }; #log { source(src); filter(f_filter1); destination(d_cons); }; log { source(src); filter(f_filter2); destination(d_mysql); }; log { source(src); filter(f_filter2); destination(d_mesg); }; log { source(src); filter(f_filter3); destination(d_auth); }; log { source(src); filter(f_filter4); destination(d_mail); }; log { source(src); filter(f_filter5); destination(d_mlal); }; log { source(src); filter(f_filter6); destination(d_spol); }; log { source(src); filter(f_filter7); destination(d_boot); }; log { source(src); filter(f_filter8); destination(d_cron); }; ... Then I started syslog-ng and received the following errors in /var/log/messages: ... Feb 14 16:21:59 furw syslog-ng[21895]: Cannot open file /var/run/mysql.pipe for writing (Permission denied) ... When I run an strace on the file, I get the following: ... uname({sys="Linux", node="furw", ...}) = 0 stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=130, ...}) = 0 open("/tmp/mysql.pipe", O_RDWR|O_NONBLOCK|O_NOCTTY|O_LARGEFILE) = -1 EACCES (Permission denied) chown32("/tmp/mysql.pipe", 0, 0) = -1 EACCES (Permission denied) chmod("/tmp/mysql.pipe", 0600) = -1 EACCES (Permission denied) time(NULL) = 1108422581 stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=130, ...}) = 0 ... Does anyone have an idea of why syslog-ng cannot read my pipe? I am open to ideas... -Matt

2 1

Update for syslog-ng FAQ
by Kevin 14 Feb '05

14 Feb '05

The FAQ (http://www.campin.net/syslog-ng/faq.html) talks about syslog volume and mentions "sites that exceed five gigabytes per day of log data". Our log volume has been growing slowly over time, I recently checked my primary logger and noticed that the raw log volume for this past Monday (the 24-hour period from midnight to midnight) was 10,982,118,488 bytes, or 10.22GiB. Peak hourly volume was 913MiB. Peak one-second volume in that hour was 2,626 messages totaling 440KiB, no duplicates. System specs: OpenBSD on a Dell 2650 (single 2.8Ghz P4) running syslog-ng 1.6.X. Logs are written to a Dell PERC 3/Di SCSI RAID-0, as a 2-drive stripe. Kevin Kadow

2 4

syslog-ng.conf.5 patch
by Jose Pedro Oliveira 14 Feb '05

14 Feb '05

Balasz, Two more patches: 1) doc/syslog-ng.conf.5 The syslog-ng.conf.5 manpage patch *only* updates the information regarding the Global Options. Problems: The global option "long_hostnames" is only documented in the contrib/syslog-ng.conf.doc file (not in the SGML file). The global option "check_hostname" is only documented in the SGML file (not in the syslog-ng.conf.doc). 2) contrib/syslog-ng.conf.doc The syslog-ng.conf.doc patch corrects the source type of /proc/kmsg (pipe -> file). Shouldn't this file be better located in the doc directory? (if this file is relocated, the contrib/README file will also need to be updated). Regards, jpo -- José Pedro Oliveira mailto: jpo(a)di.uminho.pt * http://gsd.di.uminho.pt/~jpo diff -ruN syslog-ng-1.6.6+20050208-orig/contrib/syslog-ng.conf.doc syslog-ng-1.6.6+20050208/contrib/syslog-ng.conf.doc --- syslog-ng-1.6.6+20050208-orig/contrib/syslog-ng.conf.doc 2003-04-10 11:51:00.000000000 +0100 +++ syslog-ng-1.6.6+20050208/contrib/syslog-ng.conf.doc 2005-02-09 03:33:53.370583904 +0000 @@ -283,7 +283,7 @@ # source src # { # internal(); -# pipe("/proc/kmsg" log_prefix("kernel: ")); +# file("/proc/kmsg" log_prefix("kernel: ")); # tcp(ip(127.0.0.1) port(4800) keep-alive(yes)); # udp(); # unix-stream("/dev/log"); @@ -304,7 +304,7 @@ { internal(); }; source s_kernel - { pipe("/proc/kmsg" log_prefix("kernel: ")); }; + { file("/proc/kmsg" log_prefix("kernel: ")); }; source s_tcp { tcp(port(4800) keep-alive(yes) max_connections(100)); }; diff -ruN syslog-ng-1.6.6+20050208-orig/doc/syslog-ng.conf.5 syslog-ng-1.6.6+20050208/doc/syslog-ng.conf.5 --- syslog-ng-1.6.6+20050208-orig/doc/syslog-ng.conf.5 1999-07-10 16:58:32.000000000 +0100 +++ syslog-ng-1.6.6+20050208/doc/syslog-ng.conf.5 2005-02-09 03:23:45.813946000 +0000 @@ -89,11 +89,130 @@ Where optn can be any of the following: -* mark(n) set mark frequency to n seconds - -* sync(n) set sync frequence to n lines (can be overridden locally) - -* long_hostnames(yes|no) set long hostname generation on or off +.TP +.B bad_hostname(regex) +A regexp which matches hostnames which should not be taken as such. + +.TP +.B chain_hostnames(yes|no) +Enable or disable the chained hostname format. + +.TP +.B check_hostname(yes|no) +Enable or disable whether the hostname contains valid characters. + +.TP +.B create_dirs(yes|no) +Enable or disable directory creation for destination files. + +.TP +.B dir_owner(uid) +User id. + +.TP +.B dir_group(gid) +Group id. + +.TP +.B dir_perm(perm) +Permission value (octal mask). + +.TP +.B dns_cache(yes|no) +Enable or disable DNS cache usage. + +.TP +.B dns_cache_expire(n) +Number of seconds while a successful lookup is cached. + +.TP +.B dns_cache_expire_failed(n) +Number of seconds while a failed lookup is cached. + +.TP +.B dns_cache_size(n) +Number of hostnames in the DNS cache. + +.TP +.B gc_busy_threshold(n) +Sets the threshold value for the garbage collector, when syslog-ng is +busy. GC phase starts when the number of allocated objects reach this +number. Default: 3000. + +.TP +.B gc_idle_threshold(n) +Sets the threshold value for the garbage collector, when syslog-ng is +idle. GC phase starts when the number of allocated objects reach this +number. Default: 100. + +.TP +.B group(gid) +Group id. + +.TP +.B keep_hostname(yes|no) +Enable or disable hostname rewriting. + +.TP +.B log_fifo_size(n) +The number of lines fitting to the output queue. + +.TP +.B log_msg_size(n) +Maximum length of message in bytes. + +.TP +.B long_hostnames(yes|no) +Set long hostname generation on or off. This options appears to only +really have an affect on the local system, which removes the source of +the log. As an example, normally the local logs will state +src@hostname, but with this feature off, the source is not reported. + +.TP +.B mark(n) +The number of seconds between two MARK lines. +NOTE: not implemented yet. + +.TP +.B owner(uid) +User id. + +.TP +.B perm(perm) +Permission value (octal mask). + +.TP +.B stats(n) +The number of seconds between two STATS. + +.TP +.B sync(n) +The number of lines buffered before written to file +(can be overridden locally). + +.TP +.B time_reap(n) +The time to wait before an idle destination file is closed. + +.TP +.B time_reopen(n) +The time to wait before a died connection is reestablished. + +.TP +.B use_dns(yes|no) +Enable or disable DNS usage. syslog-ng blocks on DNS queries, so +enabling DNS may lead to a Denial of Service attack. To prevent DoS, +protect your syslog-ng network endpoint with firewall rules, and make +sure that all hosts, which may get to syslog-ng is resolvable. + +.TP +.B use_fqdn(yes|no) +Add Fully Qualified Domain Name instead of short hostname. + +.TP +.B use_time_recvd(yes|no) +Use the time a message is received instead of the one specified in the +message. .SH FILES /etc/syslog-ng/syslog-ng.conf

2 4

Compiling syslog-ng-1.9.3 on OpenBSD 3.6?
by Kevin 12 Feb '05

12 Feb '05

I encounter a strange error related to the variable 'timezone' when attempting to compile syslog-ng-1.9.3 on OpenBSD 3.6. Suggestions appreciated. First problem I found was easily addressed: main.c main.c:39: wait.h: No such file or directory Replacing line 39 (#include <wait.h>) with these three lines solved that issue: #include <sys/time.h> #include <sys/resource.h> #include <sys/wait.h> After resolving that issue, I encounter the following: echo './'`logmsg.c logmsg.c: In function `log_stamp_format': logmsg.c:68: warning: assignment makes integer from pointer without a cast logmsg.c: In function `log_msg_parse': logmsg.c:234: warning: assignment makes integer from pointer without a cast logmsg.c:237: invalid operands to binary - logmsg.c:265: warning: assignment makes integer from pointer without a cast logmsg.c: In function `log_msg_init': logmsg.c:470: warning: assignment makes integer from pointer without a cast *** Error code 1 These lines are as follows: 68: target_zone_offset = timezone; 234: self->stamp.zone_offset = timezone; /* assume local timezone */ 237: self->stamp.time.tv_sec = mktime(&tm) - timezone + self->stamp.zone_offset; 265: self->stamp.zone_offset = timezone; /* assume local timezone */ 470: self->recvd.zone_offset = timezone; In OpenBSD, timezone is a function, char *timezone(int, int), not a variable. it appears that older versions of syslog-ng included options in configure for "checking for global timezone variable", but not in 1.9.3? Thanks, Kevin Kadow

3 2

Error compiling syslog-ng1.6.6 on HP-UX 11.23 IPF
by Jenny-Lou Sequeira 11 Feb '05

11 Feb '05

Hi, I get the following error when running "make" for syslog-ng1.6.6 on HP-UX 11.23 IPF. I do not get the error for 1.6.5. gcc -g -O2 -Wall -I/usr/local/include/libol -D_GNU_SOURCE -o syslog-ng main.o sources.o center.o filters.o destinations.o log.o cfgfile.o cfg-grammar.o cfg-lex.o affile.o afsocket.o afunix.o afinet.o afinter.o afuser.o afstreams.o afprogram.o afremctrl.o nscache.o utils.o syslog-names.o macros.o getopt.o getopt1.o -lnsl /usr/local/lib/libol.a -lnsl -lxnet -Wl,-Bstatic -lfl -lwrap -Wl,-Bdynamic ld: Bad keyword argument: static Fatal error. collect2: ld returned 1 exit status *** Error exit code 1 Stop. Thanks, Jenny-Lou

2 1

Am I an idiot? (tcp destinations)
by Jay 'Whip' Grizzard 10 Feb '05

10 Feb '05

Okay, perhaps I'm an idiot, but with a freshly downloaded and compiled syslog-ng 1.6.6, I get the following: syslogng.conf (one line long): destination d_tcp { tcp("10.1.2.3") port(1999); localport(999)); }; (which is, you'll notice, copied directly out of the docs) But I get: : [elfchief@bluetail /tmp]% /sbin/syslog-ng --cfgfile=/tmp/syslogng.conf --syntax-only : syntax error at 1 : Parse error reading configuration file, exiting. (line 1) ... and no matter what I put in there (I tried a lot of permutations, and even cut&pasted bits of other people's configs I saw in the mailing list archives), I can't seem to get the option to parse. Am I missing something fundamental here? Given that this is the one feature I'd really like to use syslog-ng for, it'd be nice if I could get it to work! I appreciate any thoughts anyone cares to share... -jay

4 6

Syslog -> Oracle Database
by Shane Presley 08 Feb '05

08 Feb '05

So I am considering moving my syslog-ng entries into Oracle. I've found the scripts to do that, but still working with our DBAs on it. One question I had, do we need an Oracle client on the syslog-ng server? With MySQL you can connect to a remote SQL server using the script directly. The Oracle scripts also seem to reference an Oracle Server and port. But my Oracle DBA's seem to think they have to install the Oracle client. Anyone done this before to a remote Oracle server? Thanks Shane

2 3

new Fedora/Redhat init script
by Philip J. Hollenback 08 Feb '05

08 Feb '05

Attached is a new init script I wrote. It's based on the existing init.d.Redhat and init.d.Redhat-7.3 scripts. The main deficiency I wanted to correct was that the existing scripts do not properly implement 'reload' by sending a HUP to the running syslog-ng. Other than that I just tried to clean the init script up and make it consistent with other Redhat-style init scripts. I'm calling this script init.d.Fedora since there were already those existing scripts. However, I believe my script should work on any RedHat or Fedora system, so I would ask that you remove the existing scripts. Unless, of course, there's some functionality I'm missing in my script. Thanks, P. -- Philip J. Hollenback Telemetry Investments phollenback(a)telemetry-investments.com

7 11