- syslog-ng - lists.balabit.hu

Logrotate ?
by Brian Ipsen 18 Feb '05

18 Feb '05

Hi, The re-occurring question... How com that logrotate isn't working on my logs ?? I have a file in /etc/logrotate.d like this: /var/log/snmp/*.log { weekly notifempty compress sharedscripts missingok prerotate /bin/kill -HUP `cat /var/run/syslog-ng.pid 2> /dev/null` 2> /dev/null || true /etc/init.d/syslog-ng restart endscript postrotate /bin/kill -HUP `cat /var/run/syslog-ng.pid 2> /dev/null` 2> /dev/null || true /etc/init.d/syslog-ng restart touch /var/log/snmp/network.log /var/log/snmp/snmptrapd.log /var/log/snmp/ups.log /var/log/snmp/vpn.log /var/log/snmp/witraz.log endscript } I did the kill -HUP both as pre- and post-rotate because there seems to a problem, I've even tried to restart the daemon - nothing helps... Checking my /var/log/snmp dir gives me: -rw-r--r-- 1 root root 2935158 Feb 18 20:12 network.log -rw------- 1 root root 1106 Nov 18 23:44 network.log.1.gz -rw------- 1 root root 7981 Aug 24 21:19 network.log.2.gz -rw------- 1 root root 1664 Jul 31 2004 network.log.3.gz -rw------- 1 root root 7012 Jul 23 2004 network.log.4.gz -rw-r--r-- 1 root root 70908362 Feb 18 20:14 snmptrapd.log -rw------- 1 root root 28929 Nov 19 04:00 snmptrapd.log.1.gz -rw------- 1 root root 4591 Aug 25 02:58 snmptrapd.log.2.gz -rw------- 1 root root 3366 Aug 1 2004 snmptrapd.log.3.gz -rw------- 1 root root 4729 Jul 25 2004 snmptrapd.log.4.gz -rw-r--r-- 1 root root 767 Feb 3 09:43 ups.log -rw------- 1 root root 467 Aug 24 14:36 ups.log.1.gz -rw------- 1 root root 220 Jul 15 2004 ups.log.2.gz -rw-r--r-- 1 root root 36442477 Feb 18 20:10 vpn.log -rw-r--r-- 1 root root 1045305 Dec 30 10:35 vpn.log.1.gz -rw------- 1 root root 36723 Nov 21 03:56 vpn.log.2.gz -rw-r--r-- 1 root root 104 Nov 4 09:40 vpn-trap.log.1.gz -rw-r--r-- 1 root root 1533667 Feb 17 17:17 witraz.log -rw------- 1 root root 553 Nov 19 17:49 witraz.log.1.gz -rw------- 1 root root 187 Aug 24 15:56 witraz.log.2.gz -rw------- 1 root root 620 Jul 31 2004 witraz.log.3.gz -rw------- 1 root root 703 Jul 24 2004 witraz.log.4.gz I've rotated some of the logfiles manually - but check e.g. network.log - it's been 3 months since last rotate ?? I'm running on a RHEL 3.0 Any idea why this logrotate isn't working ?? /Brian

1 0

FAQ entries wanted for performance tuning
by Nate Campi 18 Feb '05

18 Feb '05

Everyone please send me your syslog-ng performance tuning tips from the real world. I added something to the end of the FAQ, but I know there's much much more we can add from our collective experience. Email directly to me is appropriate for this. I'm happy to include OS-level stuff, since the focus here is to build a fast loghost. This includes everything along the path from the NIC through the filesystem and onto some kind of final storage. Of course the focus is on syslog-ng, though. :) -- Nate "It is good sportsmanship to not pick up lost golf balls while they are still rolling." - Samuel Clemens

1 0

new to syslog-ng
by Mike Pepe 17 Feb '05

17 Feb '05

Hello all, I'm new to syslog-ng. I have the program ready to go, but I was wondering if there's a faq or guide somewhere so I can just replace regular syslog functionality on Fedora Core 3. Once I get that working normally, I want to then log different events to different files. Any pointers would be appreciated. thanks! -Mike

5 7

Message Templates and Syslog ID
by Rhugga 16 Feb '05

16 Feb '05

Im trying to normalize the logs from my various systems I support (Solaris, IRIX, Tru64, SLES 8 & 9, RHAS 3, Cisco, Brocade, etc...) These logs are processed and stored into oracle. On Solaris, syslogd adds a field containing the syslog message ID and the facility and priority: EX: Jan 20 22:07:26 back-0202 tldd[3184]: [ID 817171 daemon.notice] Calling the drive_mount_notify ^^^^^^^^^^^^^^^^^^^^^^^^^ This behavior can only be changed by a reboot and sun also does not recommend setting msgid=0 in /kernerl/drv/log.conf. So, using the following template I can come close to normalizing this behavior: template("$DATE $HOST $PROGRAM [ID $FACILITY.$LEVEL] $MSG\n") The information that is lost is the internal syslog ID and the pid of the sending process. Here is a log entry processed thru this template: Feb 14 07:07:52 db-0201 su [ID auth.info] su: [ID 366847 auth.info] 'su oracle' succeeded for root on /dev/??? Another problem is that field in question now appears twice. Anyone know of any possible way to configure syslog-ng to solves this roadblock? Thx, CC

2 2

FIFO / pipe destination problem
by Howell, Matt 15 Feb '05

15 Feb '05

This may end up being something really easy that I am missing, but I have been struggling with it for a couple days. Googling the problem has yet to review any fruitful answers. I am trying to pass messages to MySQL via the pipe() function. I am working with syslog-ng 1.6.5-6 compiled from source for Fedora Core 3. I have setup a FIFO in the filesystem as follows: [root@furw tmp]# mkfifo mysql.pipe [root@furw tmp]# chmod 0600 mysql.pipe [root@furw tmp]# ls -al ... drwxrwxrwt 24 root root 4096 Feb 14 17:38 . drwxr-xr-x 23 root root 4096 Feb 11 16:00 .. prw------- 1 root root 0 Feb 14 17:38 mysql.pipe ... I then created my syslog-ng.conf file: ... options { sync (0); time_reopen (10); log_fifo_size (1000); long_hostnames (off); use_dns (no); use_fqdn (no); create_dirs (no); keep_hostname (yes); }; source src { unix-stream ("/dev/log"); internal(); udp(ip(0.0.0.0) port(514)); }; destination d_mysql { pipe("/tmp/mysql.pipe" template("INSERT INTO logs (host, facility, priority, level, tag, date, time, program, msg) VALUES ( '$HOST', '$FACILITY', '$PRIORITY', '$LEVEL', '$TAG', '$YEAR-$MONTH-$DAY', '$HOUR:$MIN:$SEC', '$PROGRAM', '$MSG' );\n") template-escape(yes)); }; destination d_cons { file("/dev/console"); }; destination d_mesg { file("/var/log/messages"); }; destination d_auth { file("/var/log/secure"); }; destination d_mail { file("/var/log/maillog" sync(10)); }; destination d_spol { file("/var/log/spooler"); }; destination d_boot { file("/var/log/boot.log"); }; destination d_cron { file("/var/log/cron"); }; destination d_mlal { usertty("*"); }; filter f_filter1 { facility(kern); }; filter f_filter2 { level(info..emerg) and not facility(mail,authpriv,cron); }; filter f_filter3 { facility(authpriv); }; filter f_filter4 { facility(mail); }; filter f_filter5 { level(emerg); }; filter f_filter6 { facility(uucp) or (facility(news) and level(crit..emerg)); }; filter f_filter7 { facility(local7); }; filter f_filter8 { facility(cron); }; #log { source(src); filter(f_filter1); destination(d_cons); }; log { source(src); filter(f_filter2); destination(d_mysql); }; log { source(src); filter(f_filter2); destination(d_mesg); }; log { source(src); filter(f_filter3); destination(d_auth); }; log { source(src); filter(f_filter4); destination(d_mail); }; log { source(src); filter(f_filter5); destination(d_mlal); }; log { source(src); filter(f_filter6); destination(d_spol); }; log { source(src); filter(f_filter7); destination(d_boot); }; log { source(src); filter(f_filter8); destination(d_cron); }; ... Then I started syslog-ng and received the following errors in /var/log/messages: ... Feb 14 16:21:59 furw syslog-ng[21895]: Cannot open file /var/run/mysql.pipe for writing (Permission denied) ... When I run an strace on the file, I get the following: ... uname({sys="Linux", node="furw", ...}) = 0 stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=130, ...}) = 0 open("/tmp/mysql.pipe", O_RDWR|O_NONBLOCK|O_NOCTTY|O_LARGEFILE) = -1 EACCES (Permission denied) chown32("/tmp/mysql.pipe", 0, 0) = -1 EACCES (Permission denied) chmod("/tmp/mysql.pipe", 0600) = -1 EACCES (Permission denied) time(NULL) = 1108422581 stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=130, ...}) = 0 ... Does anyone have an idea of why syslog-ng cannot read my pipe? I am open to ideas... -Matt

2 1

Update for syslog-ng FAQ
by Kevin 14 Feb '05

14 Feb '05

The FAQ (http://www.campin.net/syslog-ng/faq.html) talks about syslog volume and mentions "sites that exceed five gigabytes per day of log data". Our log volume has been growing slowly over time, I recently checked my primary logger and noticed that the raw log volume for this past Monday (the 24-hour period from midnight to midnight) was 10,982,118,488 bytes, or 10.22GiB. Peak hourly volume was 913MiB. Peak one-second volume in that hour was 2,626 messages totaling 440KiB, no duplicates. System specs: OpenBSD on a Dell 2650 (single 2.8Ghz P4) running syslog-ng 1.6.X. Logs are written to a Dell PERC 3/Di SCSI RAID-0, as a 2-drive stripe. Kevin Kadow

2 4

syslog-ng.conf.5 patch
by Jose Pedro Oliveira 14 Feb '05

14 Feb '05

Balasz, Two more patches: 1) doc/syslog-ng.conf.5 The syslog-ng.conf.5 manpage patch *only* updates the information regarding the Global Options. Problems: The global option "long_hostnames" is only documented in the contrib/syslog-ng.conf.doc file (not in the SGML file). The global option "check_hostname" is only documented in the SGML file (not in the syslog-ng.conf.doc). 2) contrib/syslog-ng.conf.doc The syslog-ng.conf.doc patch corrects the source type of /proc/kmsg (pipe -> file). Shouldn't this file be better located in the doc directory? (if this file is relocated, the contrib/README file will also need to be updated). Regards, jpo -- José Pedro Oliveira mailto: jpo(a)di.uminho.pt * http://gsd.di.uminho.pt/~jpo diff -ruN syslog-ng-1.6.6+20050208-orig/contrib/syslog-ng.conf.doc syslog-ng-1.6.6+20050208/contrib/syslog-ng.conf.doc --- syslog-ng-1.6.6+20050208-orig/contrib/syslog-ng.conf.doc 2003-04-10 11:51:00.000000000 +0100 +++ syslog-ng-1.6.6+20050208/contrib/syslog-ng.conf.doc 2005-02-09 03:33:53.370583904 +0000 @@ -283,7 +283,7 @@ # source src # { # internal(); -# pipe("/proc/kmsg" log_prefix("kernel: ")); +# file("/proc/kmsg" log_prefix("kernel: ")); # tcp(ip(127.0.0.1) port(4800) keep-alive(yes)); # udp(); # unix-stream("/dev/log"); @@ -304,7 +304,7 @@ { internal(); }; source s_kernel - { pipe("/proc/kmsg" log_prefix("kernel: ")); }; + { file("/proc/kmsg" log_prefix("kernel: ")); }; source s_tcp { tcp(port(4800) keep-alive(yes) max_connections(100)); }; diff -ruN syslog-ng-1.6.6+20050208-orig/doc/syslog-ng.conf.5 syslog-ng-1.6.6+20050208/doc/syslog-ng.conf.5 --- syslog-ng-1.6.6+20050208-orig/doc/syslog-ng.conf.5 1999-07-10 16:58:32.000000000 +0100 +++ syslog-ng-1.6.6+20050208/doc/syslog-ng.conf.5 2005-02-09 03:23:45.813946000 +0000 @@ -89,11 +89,130 @@ Where optn can be any of the following: -* mark(n) set mark frequency to n seconds - -* sync(n) set sync frequence to n lines (can be overridden locally) - -* long_hostnames(yes|no) set long hostname generation on or off +.TP +.B bad_hostname(regex) +A regexp which matches hostnames which should not be taken as such. + +.TP +.B chain_hostnames(yes|no) +Enable or disable the chained hostname format. + +.TP +.B check_hostname(yes|no) +Enable or disable whether the hostname contains valid characters. + +.TP +.B create_dirs(yes|no) +Enable or disable directory creation for destination files. + +.TP +.B dir_owner(uid) +User id. + +.TP +.B dir_group(gid) +Group id. + +.TP +.B dir_perm(perm) +Permission value (octal mask). + +.TP +.B dns_cache(yes|no) +Enable or disable DNS cache usage. + +.TP +.B dns_cache_expire(n) +Number of seconds while a successful lookup is cached. + +.TP +.B dns_cache_expire_failed(n) +Number of seconds while a failed lookup is cached. + +.TP +.B dns_cache_size(n) +Number of hostnames in the DNS cache. + +.TP +.B gc_busy_threshold(n) +Sets the threshold value for the garbage collector, when syslog-ng is +busy. GC phase starts when the number of allocated objects reach this +number. Default: 3000. + +.TP +.B gc_idle_threshold(n) +Sets the threshold value for the garbage collector, when syslog-ng is +idle. GC phase starts when the number of allocated objects reach this +number. Default: 100. + +.TP +.B group(gid) +Group id. + +.TP +.B keep_hostname(yes|no) +Enable or disable hostname rewriting. + +.TP +.B log_fifo_size(n) +The number of lines fitting to the output queue. + +.TP +.B log_msg_size(n) +Maximum length of message in bytes. + +.TP +.B long_hostnames(yes|no) +Set long hostname generation on or off. This options appears to only +really have an affect on the local system, which removes the source of +the log. As an example, normally the local logs will state +src@hostname, but with this feature off, the source is not reported. + +.TP +.B mark(n) +The number of seconds between two MARK lines. +NOTE: not implemented yet. + +.TP +.B owner(uid) +User id. + +.TP +.B perm(perm) +Permission value (octal mask). + +.TP +.B stats(n) +The number of seconds between two STATS. + +.TP +.B sync(n) +The number of lines buffered before written to file +(can be overridden locally). + +.TP +.B time_reap(n) +The time to wait before an idle destination file is closed. + +.TP +.B time_reopen(n) +The time to wait before a died connection is reestablished. + +.TP +.B use_dns(yes|no) +Enable or disable DNS usage. syslog-ng blocks on DNS queries, so +enabling DNS may lead to a Denial of Service attack. To prevent DoS, +protect your syslog-ng network endpoint with firewall rules, and make +sure that all hosts, which may get to syslog-ng is resolvable. + +.TP +.B use_fqdn(yes|no) +Add Fully Qualified Domain Name instead of short hostname. + +.TP +.B use_time_recvd(yes|no) +Use the time a message is received instead of the one specified in the +message. .SH FILES /etc/syslog-ng/syslog-ng.conf

2 4

Compiling syslog-ng-1.9.3 on OpenBSD 3.6?
by Kevin 12 Feb '05

12 Feb '05

I encounter a strange error related to the variable 'timezone' when attempting to compile syslog-ng-1.9.3 on OpenBSD 3.6. Suggestions appreciated. First problem I found was easily addressed: main.c main.c:39: wait.h: No such file or directory Replacing line 39 (#include <wait.h>) with these three lines solved that issue: #include <sys/time.h> #include <sys/resource.h> #include <sys/wait.h> After resolving that issue, I encounter the following: echo './'`logmsg.c logmsg.c: In function `log_stamp_format': logmsg.c:68: warning: assignment makes integer from pointer without a cast logmsg.c: In function `log_msg_parse': logmsg.c:234: warning: assignment makes integer from pointer without a cast logmsg.c:237: invalid operands to binary - logmsg.c:265: warning: assignment makes integer from pointer without a cast logmsg.c: In function `log_msg_init': logmsg.c:470: warning: assignment makes integer from pointer without a cast *** Error code 1 These lines are as follows: 68: target_zone_offset = timezone; 234: self->stamp.zone_offset = timezone; /* assume local timezone */ 237: self->stamp.time.tv_sec = mktime(&tm) - timezone + self->stamp.zone_offset; 265: self->stamp.zone_offset = timezone; /* assume local timezone */ 470: self->recvd.zone_offset = timezone; In OpenBSD, timezone is a function, char *timezone(int, int), not a variable. it appears that older versions of syslog-ng included options in configure for "checking for global timezone variable", but not in 1.9.3? Thanks, Kevin Kadow

3 2

Error compiling syslog-ng1.6.6 on HP-UX 11.23 IPF
by Jenny-Lou Sequeira 11 Feb '05

11 Feb '05

Hi, I get the following error when running "make" for syslog-ng1.6.6 on HP-UX 11.23 IPF. I do not get the error for 1.6.5. gcc -g -O2 -Wall -I/usr/local/include/libol -D_GNU_SOURCE -o syslog-ng main.o sources.o center.o filters.o destinations.o log.o cfgfile.o cfg-grammar.o cfg-lex.o affile.o afsocket.o afunix.o afinet.o afinter.o afuser.o afstreams.o afprogram.o afremctrl.o nscache.o utils.o syslog-names.o macros.o getopt.o getopt1.o -lnsl /usr/local/lib/libol.a -lnsl -lxnet -Wl,-Bstatic -lfl -lwrap -Wl,-Bdynamic ld: Bad keyword argument: static Fatal error. collect2: ld returned 1 exit status *** Error exit code 1 Stop. Thanks, Jenny-Lou

2 1

Am I an idiot? (tcp destinations)
by Jay 'Whip' Grizzard 10 Feb '05

10 Feb '05

Okay, perhaps I'm an idiot, but with a freshly downloaded and compiled syslog-ng 1.6.6, I get the following: syslogng.conf (one line long): destination d_tcp { tcp("10.1.2.3") port(1999); localport(999)); }; (which is, you'll notice, copied directly out of the docs) But I get: : [elfchief@bluetail /tmp]% /sbin/syslog-ng --cfgfile=/tmp/syslogng.conf --syntax-only : syntax error at 1 : Parse error reading configuration file, exiting. (line 1) ... and no matter what I put in there (I tried a lot of permutations, and even cut&pasted bits of other people's configs I saw in the mailing list archives), I can't seem to get the option to parse. Am I missing something fundamental here? Given that this is the one feature I'd really like to use syslog-ng for, it'd be nice if I could get it to work! I appreciate any thoughts anyone cares to share... -jay

4 6

Syslog -> Oracle Database
by Shane Presley 08 Feb '05

08 Feb '05

So I am considering moving my syslog-ng entries into Oracle. I've found the scripts to do that, but still working with our DBAs on it. One question I had, do we need an Oracle client on the syslog-ng server? With MySQL you can connect to a remote SQL server using the script directly. The Oracle scripts also seem to reference an Oracle Server and port. But my Oracle DBA's seem to think they have to install the Oracle client. Anyone done this before to a remote Oracle server? Thanks Shane

2 3

new Fedora/Redhat init script
by Philip J. Hollenback 08 Feb '05

08 Feb '05

Attached is a new init script I wrote. It's based on the existing init.d.Redhat and init.d.Redhat-7.3 scripts. The main deficiency I wanted to correct was that the existing scripts do not properly implement 'reload' by sending a HUP to the running syslog-ng. Other than that I just tried to clean the init script up and make it consistent with other Redhat-style init scripts. I'm calling this script init.d.Fedora since there were already those existing scripts. However, I believe my script should work on any RedHat or Fedora system, so I would ask that you remove the existing scripts. Unless, of course, there's some functionality I'm missing in my script. Thanks, P. -- Philip J. Hollenback Telemetry Investments phollenback(a)telemetry-investments.com

7 11

Compilation problem with syslog-ng-1.9.3.
by Peter 08 Feb '05

08 Feb '05

Hello. I have a problem. I'm trying to compile syslog-ng-1.9.3. But after succesful configuration I have a linking error: i686-pc-linux-gnu-gcc -O2 -march=pentium4 -fomit-frame-pointer -pipe -mcpu=pentium4 -Wall -g -o syslog-ng main.o misc.o utils.o messages.o gsockaddr.o syslog-names.o macros.o logmsg.o cfg.o logpipe.o driver.o sgroup.o dgroup.o center.o filter.o logsource.o logreader.o logwriter.o affile.o afunix.o afinet.o afsocket.o afstreams.o afuser.o afprog.o afinter.o cfg-lex.o cfg-grammar.o fdwrite.o fdread.o memtrace.o templates.o children.o -lnsl -Wl,-static -lfl -lglib-2.0 -levtlog -Wl,-call_shared /usr/lib/gcc-lib/i686-pc-linux-gnu/3.3.5/../../../../i686-pc-linux-gnu/bin/ld: cannot find -lglib-2.0 collect2: ld returned 1 exit status make[2]: *** [syslog-ng] Error 1 make[2]: Leaving directory `/var/tmp/portage/syslog-ng-1.9.3/work/syslog-ng-1.9.3/src' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/var/tmp/portage/syslog-ng-1.9.3/work/syslog-ng-1.9.3' make: *** [all] Error 2 Can anybody enlight me how I can avoid this error? I am sure I have this library: camobap ~ # ls -l /usr/lib/libglib-2.0.* -rwxr-xr-x 1 root root 805 Янв 30 22:01 /usr/lib/libglib-2.0.la lrwxrwxrwx 1 root root 22 Янв 30 22:01 /usr/lib/libglib-2.0.so -> libglib-2.0.so.0.400.8 lrwxrwxrwx 1 root root 22 Янв 30 22:01 /usr/lib/libglib-2.0.so.0 -> libglib-2.0.so.0.400.8 -rwxr-xr-x 1 root root 503304 Янв 30 22:01/usr/lib/libglib-2.0.so.0.400.8 Also I would like to note that many other applications that also depend on glib-2.0 compiled without problems. I can not find sollution by myself. Where should I look for sollution? Thank you in advance, _______ Peter.

2 1

1.6.6 release status
by Balazs Scheidler 05 Feb '05

05 Feb '05

Hi, Just a quick summary, how we are at: - there is one pending patch reporting truncated messages when log_msg_size() is exceeded, it can be in once Roberto confirms that my change is correct to his patch. Otherwise I've committed everything in, and made a manual snapshot with today's date which is being uploaded to our webserver right now. (takes a couple of minutes as there are over 10000 files to synchronize) Thanks Jose and Roberto for helping me preparing the release. -- Bazsi

3 3

syslog-ng 1.6.6 released
by Balazs Scheidler 04 Feb '05

04 Feb '05

Hi, I have released syslog-ng 1.6.6 with the user-visible changes file included at the end of this mail. It has been slightly been changed since the last snapshot to make the linking changes compatible with Solaris's linker. Otherwise the release is identical. Please note that I have changed GPG keys, this is signed with the new one. The fact that the release was signed with this key might attach some trust. My fingerprint is: pub 1024D/ADCF4138 2004-11-10 Balazs Scheidler <bazsi(a)balabit.hu> Key fingerprint = 8038 B76C B92A 661D E4EF 222D B613 44D0 ADCF 4138 sub 2048g/82B2CACD 2004-11-10 The release is available at the usual location at http://www.balabit.com/downloads/syslog-ng/ News for the 1.6.6 release Fri, 04 Feb 2005 13:06:22 +0100 * Fixed timezone handling problem on platforms which require an explicit tzset() call. * Fixed resolving the PRI macro which always expanded to an empty string. * Fixed possible hang during reload for one second. * Fixed syslog2ng script which sometimes generated three dots when generating priority ranges. (debug...emerg) * Added a notification about splitting message lines, if they are over the configured log_msg_size() limit. * Added a setsid() call to program destinations to avoid being in the same process group as the child, as some shells send signals to the whole process group they are in, effectively killing syslog-ng. (triggered on OpenBSD for example) * Added log_fifo_size option to TCP/UDP destinations. * Added relogger.pl to contrib/ which replays existing syslog files. * Updated documentation (manpages and SGML documentation) * Link to various libraries residing in /usr statically by default, also added an --enable-full-static configure option to make a fully static version. Happy logging, -- Bazsi

2 2

/dev/log on Linux is a datagram socket, not a stream socket
by Philip J. Hollenback 04 Feb '05

04 Feb '05

The syslog-ng reference manual (and the sample syslog-ng.conf for RedHat) indicate that /dev/log on linux is a stream socket. However, other utilities think that /dev/log is a datagram socket. To see this, configure syslog-ng to open /dev/log with unix_stream and then strace the logger command. logger tries to open /dev/log as a datagram socket first, fails, and then falls back to opening it as a stream socket: socket(PF_FILE, SOCK_DGRAM, 0) = 1 fcntl64(1, F_SETFD, FD_CLOEXEC) = 0 connect(1, {sa_family=AF_FILE, path="/dev/log"}, 16) = -1 EPROTOTYPE (Protocol wrong type for socket) close(1) = 0 socket(PF_FILE, SOCK_STREAM, 0) = 1 fcntl64(1, F_SETFD, FD_CLOEXEC) = 0 connect(1, {sa_family=AF_FILE, path="/dev/log"}, 16) = 0 send(1, "<13>Feb 2 11:19:50 phil: test m"..., 39, 0) = 39 rt_sigaction(SIGPIPE, {SIG_DFL}, NULL, 8) = 0 close(1) = 0 If, however, you open /dev/log with unix_dgram and then run logger, it's happier: socket(PF_FILE, SOCK_DGRAM, 0) = 1 fcntl64(1, F_SETFD, FD_CLOEXEC) = 0 connect(1, {sa_family=AF_FILE, path="/dev/log"}, 16) = 0 send(1, "<13>Feb 2 11:21:28 phil: test m"..., 45, 0) = 45 rt_sigaction(SIGPIPE, {SIG_DFL}, NULL, 8) = 0 close(1) = 0 This indicates to me that the syslog-ng documentation and sample syslog-ng.conf files should be changed to show you should open /dev/log with unix_dgram on linux, not with unix_stream. This is with the 2.4.22 kernel, maybe this is something that changed at some point? Thanks, P. -- Philip J. Hollenback Telemetry Investments phollenback(a)telemetry-investments.com

2 3

$DAY not working in filenames macros in 1.6.x as it did in 1.5.x
by Andrew Fort 03 Feb '05

03 Feb '05

Hi guys, I am running syslog-ng 1.5.15 on some debian stable machines. All good, all happy. I have a source: source network-hosts { udp(ip(0.0.0.0) port(8514)); tcp(ip(0.0.0.0) port(8514) max-connections(200)); }; I use filename macro expansion to auto-rotate my logs. My destinations look exactly like: destination logfiles-host { file("/var/log/HOSTS/$HOST/$YEAR/$MONTH/$YEAR$MONTH$DAY\_$HOST\_$PROGRAM" template("$ISODATE [$HOST] <$FACILITY> $MSG\n") template_escape(no) owner(root) group(logs) perm(0660) dir_perm(0770) create_dirs(yes)); }; the routing stanza goes: log { source(network-hosts); destination(logfiles-host); }; This works as I expect (even without escaping the _'s between $DAY, $HOST and $PROGRAM). My log paths look like: /var/log/HOSTS/mel1/2005/02/20050203_mel1_named /var/log/HOSTS/mel1/2005/02/20050203_mel1_sendmail Great. However, using 1.6.x (I've not tested 1.6.6, to be sure, only 1.6.4 and 1.6.5) on both Solaris and Linux, using the exact same config (i.e., when I upgraded, for example), using this destination: destination d_local { file("/var/log/LOCAL/$HOST/$YEAR/$MONTH/$YEAR$MONTH$DAY\_$HOST" template("$ISODATE [$HOST] <$FACILITY> $MSG\n") template_escape(no) perm(0660) dir_perm(0770) create_dirs(yes)); }; I get this: /var/log/LOCAL/mel1/2005/02/200502mel1 So $DAY is not expanding. Same thing occurs when I use file("/var/log/LOCAL/$HOST/$YEAR/$MONTH/$YEAR$MONTH$DAY_$HOST" Is this a problem with $DAY, the _, or something else? Let me know if I can provide further info to help debug this.. Anyone else run into this? I'd like to upgrade all my boxes to 1.6.{4,5,6} eventually but at the moment this is holding me back from moving to 1.6. Cheers, Andrew Fort

2 3

preparing syslog-ng 1.6.6
by Balazs Scheidler 03 Feb '05

03 Feb '05

Hi, As there are some changes since syslog-ng 1.6.5, I'd like to release a new version. Before doing so I'd like to ask you to test the latest snapshot a little bit. The most important and risky change seems to be this one: 2005-01-20 Balazs Scheidler <bazsi(a)bzorp.balabit> * src/main.c: fix possible log message losing during HUP as syslog-ng was not accepting messages for 1 second and these messages were dropped, processing changed so we still wait 1 second to flush buffers but also accept messages in this interval, so the window of losing messages becomes much smaller. (fixes: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=282771) Everything else is minor, and although they might cause build problems on some platforms, I doubt they'd cause real problems. The latest snapshot is here: http://www.balabit.com/downloads/syslog-ng/1.6/src-snapshot/ The summary of changes can be found in the file named ChangeLog within the tarball. If no show-stopper bugs are found, I'm going to release syslog-ng 1.6.6 this week. Thanks in advance. -- Bazsi

4 11

Bug in S_YEAR?
by janth＠moldung.no 03 Feb '05

03 Feb '05

Hi, I have a Solaris 8 host with syslog-ng ver 1.6.5 set up with the following syslog-ng.conf: options { use_time_recvd(no); # Use the time a message is received instead of the one specified in the message. sync(0); # The number of lines buffered before written to file. }; source src { udp(ip("0.0.0.0") port(514)); tcp(ip("0.0.0.0") port(514) max-connections(100)); sun-streams("/dev/log" door("/etc/.syslog_door")); # local solaris syslog internal(); # syslog-ng interne meldinger }; source s_syslog_ng { internal(); # syslog-ng interne meldinger }; destination dst_template { file("/syslog/$R_YEAR/$R_MONTH/$R_DAY/$HOST" template("$R_YEAR-$R_MONTH-$R_DAY $R_HOUR:$R_MIN:$R_SEC | $R_UNIXTIME | $S_YEAR-$S_MONTH-$S_DAY $S_HOUR:$S_MIN:$S_SEC | $S_UNIXTIME | $HOST | $HOST_FROM | $FULLHOST | $SOURCEIP | $FACILITY.$PRIORITY | $TAG | $PROGRAM | $MSG\n") ); }; destination d_syslog_ng { file("/syslog/syslog-ng.log" sync(0) log_fifo_size(10) perm(0644) ); }; log { source (s_syslog_ng); destination (d_syslog_ng); }; log { source(src); destination(dst_template); }; Then I have another host set up to use syslog-ng as loghost. This host has purposely wrong time (1973-01-02), as can be seen here: joe@testhost / $ logger "test. local time on `uname -n` is `date +'%Y-%m-%d %H:%m:%S'`" joe@loghost / $ tail -1 /syslog/2005/02/01/testhost 2005-02-01 11:24:47 | 1107253487 | 2005-01-02 21:04:39 | 1104696279 | testhost | testhost | testhost/testhost | 192.168.1.2 | user.notice | 0d | joe | joe: [ID 702911 user.notice] test. local time on testhost is 1973-01-02 21:01:39 It seems as S_YEAR is set to YEAR (or R_YEAR, I have not checked the source). I hope someone can fix this before syslog-ng 1.6.6... Regards Jan Thomas Moldung

2 1

syslog-ng 1.6.5: manpage patch
by Jose Pedro Oliveira 03 Feb '05

03 Feb '05

Balazs, This patch adds the missing command line options to the syslog-ng.8 manpage. The patch also applies to syslog-ng 1.9.x (he default syslog-ng.conf location still has to be corrected). Regards, jpo -- José Pedro Oliveira mailto: jpo(a)di.uminho.pt * http://gsd.di.uminho.pt/~jpo diff -ruN syslog-ng-1.6.5-orig/doc/syslog-ng.8 syslog-ng-1.6.5/doc/syslog-ng.8 --- syslog-ng-1.6.5-orig/doc/syslog-ng.8 1999-07-10 16:58:32.000000000 +0100 +++ syslog-ng-1.6.5/doc/syslog-ng.8 2005-01-30 00:05:48.906523000 +0000 @@ -7,7 +7,7 @@ .B syslog-ng [ -.B \-dvV +.B \-dFsvV ] [ .B \-f <\fIconfig-filename\fP> @@ -17,6 +17,12 @@ ] [ .B \-C <\fIchroot-dir\fP> +] [ +.B \-u +<\fIuser\fP> +] [ +.B \-g +<\fIgroup\fP> ] .SH DESCRIPTION .PP @@ -64,38 +70,72 @@ \fBsyslog-ng\fP will also save all the current values as a "checkpoint". .SH OPTIONS .TP -.B \-d +.B \-C \0<directory>, \-\-chroot=<directory> +Chroot to directory. +.TP +.B \-d, \-\-debug Set "debug mode". Prints out various messages to aid in debugging and stops it from becoming a daemon. To activate debugging in the yacc parser, either use this option twice or combine with \fB-v\fP. .TP -.B \-f \0<filename> +.B \-F, \-\-foreground +Don't fork into background. +.TP +.B \-f \0<filename>, \-\-cfgfile=<filename> Instead of reading the normal \fB/etc/syslog-ng/syslog-ng.conf\fP file for configuration information, use the filename given instead. .TP -.B \-p \0<filename> +.B \-g \0<group>, \-\-group=<group> +Switch to group. +.TP +.B \-p \0<filename>, \-\-pidfile=<filename> Write the current PID information to the specified file. +Defaults to \fB/var/run/syslog-ng.pid\fP. +.TP +.B \-s, \-\-syntax\-only +Only read and parse the configuration file. See also the option \fB-f\fP. .TP -.B \-v +.B \-u \0<user>, \-\-group=<user> +Switch to user. +.TP +.B \-v, \-\-verbose Enable verbose mode. Process will not become a daemon. Prints out fewer messages, compared to \fB-d\fP. .TP -.B \-V +.B \-V, \-\-version Print the version number. .SH DIAGNOSTICS It is expected that \fBsyslog-ng\fP will run as root, however, if not running on a priviledged port of it it owned its own log directories, etc, it might run as a non-root user. .SH SEE ALSO -logger(1), syslog(3), services(5), syslog.conf(5), syslog-ng.conf(5) +.BR syslog-ng.conf (5), +.BR logger (1), +.BR syslog (2), +.BR syslog (3), +.BR services (5), +.BR syslog.conf (5), +.BR klogd (8), +.BR syslogd (8), +.BR sysklogd (8) .SH FILES -/etc/syslog-ng/syslog-ng.conf -/etc/syslog-ng/filename.sha1 -/etc/syslog-ng/filename.H0 -/etc/syslog-ng/filename.salt -.br -/var/run/syslog.pid -.br +.TP +.I /etc/syslog-ng/syslog-ng.conf +Configuration file for +.BR syslog\-ng . +See +.BR syslog\-ng.conf (5) +for more information. +.TP +.I /etc/syslog-ng/filename.sha1 +.TP +.I /etc/syslog-ng/filename.H0 +.TP +.I /etc/syslog-ng/filename.salt +.TP +.I /var/run/syslog-ng.pid +The file containing the process id of +.BR syslog-ng . .SH BUGS .PP If you find any, please send email to me at bazsi(a)balabit.hu

3 6

syslog-ng 1.9.2/eventlog 0.2.2: packaging notes
by Jose Pedro Oliveira 03 Feb '05

03 Feb '05

Balazs, I have a couple of requests to make regarding the packaging of the above packages (for Red Hat/Fedora systems). 1) copyright information I believe both packages are being distributed under the GPL license. Could you mention this in the README files and maybe include the COPYING file in the tarballs? 2) the CFLAGS aren't being inherited The CFLAGS generated by the configure files don't honour the environment CFLAGS variable. Small patches are attached to this email. 3) eventlog static lib Would it be possible to also generate the eventlog static library by default? 4) syslog-ng 1.9.2 Would it be possible to also generate a static linked syslog-ng? Right know it only uses dynamic libraries (glib2.so and libevtlog.so). It would be nice to avoid any dynamic libraries located under /usr as the content of this directory can not yet be available (mounted) at the time the daemon syslog-ng is loaded. Best regards, jpo -- José Pedro Oliveira diff -ruN syslog-ng-1.9.2+20050112+1424-orig/configure.in syslog-ng-1.9.2+20050112+1424/configure.in --- syslog-ng-1.9.2+20050112+1424-orig/configure.in 2005-01-05 12:00:11.000000000 +0000 +++ syslog-ng-1.9.2+20050112+1424/configure.in 2005-01-15 03:22:25.989658032 +0000 @@ -71,7 +71,7 @@ PKG_CHECK_MODULES(GLIB, glib-2.0) PKG_CHECK_MODULES(EVTLOG, eventlog) -CFLAGS="-Wall -g" +CFLAGS="$CFLAGS -Wall -g" DEPS_CPPFLAGS="$CPPFLAGS $GLIB_CFLAGS $EVTLOG_CFLAGS" CPPFLAGS="$DEPS_CPPFLAGS -D_GNU_SOURCE" DEPS_LIBS="$LIBS $GLIB_LIBS $EVTLOG_LIBS" diff -ruN eventlog-0.2.2-orig/configure.in eventlog-0.2.2/configure.in --- eventlog-0.2.2-orig/configure.in 2005-01-05 12:02:18.000000000 +0000 +++ eventlog-0.2.2/configure.in 2005-01-15 03:21:00.591640504 +0000 @@ -5,7 +5,7 @@ AM_CONFIG_HEADER(config.h) AM_DISABLE_STATIC -CFLAGS="-Wall -g" +CFLAGS="$CFLAGS -Wall -g" CPPFLAGS="-D_GNU_SOURCE" AC_ARG_ENABLE(dlsym-wrapper,

5 14

There's no need to run klogd on current linux kernels
by Philip J. Hollenback 03 Feb '05

03 Feb '05

The syslog-ng reference manual http://www.balabit.com/products/syslog_ng/reference/reference.html#AEN279 contains the following: > NOTE: on Linux, the klogd daemon reads kernel messages, and > forwards them to the syslogd process. klogd preprocesses kernel > messages and replaces addresses with symbolic names (from > /boot/System.map). If you don't want to lose this functionality > you'll have to run klogd with syslog-ng as well. That info is obsolete. We run 2.4.22 and 2.6.10 kernels at my company, and both provide symbolic names in backtraces. You can verify this by looking at "dmesg" after an oops - it contains symbolic names, and that is the raw kernel log output. Please adjust the syslog-ng documentation to indicate this. Thanks, P. -- Philip J. Hollenback Telemetry Investments phollenback(a)telemetry-investments.com

2 1

why does syslog-ng open /proc/kmsg read/write?
by Philip J. Hollenback 02 Feb '05

02 Feb '05

While setting up syslog-ng 1.6.5 on Fedora Core 1 linux, I noticed a discrepancy between it and regular syslog: syslog-ng opens /proc/kmsg read/write. Here's my source entry in syslog-ng.conf: source local { pipe ("/proc/kmsg" log_prefix("kernel: ")); unix-stream ("/dev/log"); udp(); internal(); }; And lsof says: COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME syslog-ng 11320 root 3u REG 0,2 0 4112 /proc/kmsg The 'u' in the FD field means the file is open read/write. Regular syslog opens /proc/kmsg read-only. This seems to be the correct behavior, as you only ever read from that file. I notice syslog-ng also opens /dev/log read/write, so I suspect it opens all input files with that mode. Does someone have an explanation for this, or is it a bug in syslog-ng? Thanks, P. -- Philip J. Hollenback Telemetry Investments phollenback(a)telemetry-investments.com

2 2

Hernan Cortez/ADEXUS is out of the office.
by Hernan Cortez 01 Feb '05

01 Feb '05

I will be out of the office starting 31-01-2005 and will not return until 14-02-2005. Estare de vacaciones entre el 19/01 y el 30/01, si es urgente favor comuniquese con Marco Jara al Anexo 1746 e-mail: mjara(a)adexus.cl

1 0

making syslog_NG redundant / NFS / HA possibilities
by Oliver Tönsmann 31 Jan '05

31 Jan '05

Hi, we like to setup a central logging service for our distributed datacenter environment. Our idea is to use syslog_ng as an HA (high available) service running on two separate servers. The syslog clients (Solaris machines, network components, firewall appliances) should post the log data via UDP to a loadbalancer which will forward the log data to the two syslog_ng hosts. Now we have the problem how to merge the log data from the two syslog_ng hosts to one location. Most of our data is stored on centralised storage subsystems and this storage is distributed via NFS (NetApp file server). We like to store the logdata also into nfs directories. Following figure describes a possible dataflow: syslog clients --> LB --> --> syslog_ng host1 --> syslog_ng host2 --> NFS:/.../log/$HOST/$FACILITY/$PRIORITY/one_logfile_for_both_syslog_hosts Performance and nfs overhead is not a problem in our environment and nfs works very reliable due to our HA structures; logdata per day ~2-3 GB. Is there a possibilty to let syslog_ng hosts write the logdata simultaenously into the same logfile (NFS)? What are the disadvantages? What are your experiences in using syslog_ng as an HA service? What would be a good practice to setup syslog_ng and to make it high available? Ideas and helpful suggestions are welcome - Thanxx. Oliver ------------------------------------------------ Oliver Toensmann Universitaet Bielefeld - Hochschulrechenzentrum Email: Oliver.Toensmann(a)uni-bielefeld.de

2 1