- syslog-ng - lists.balabit.hu

colored messages
by Gaydov Victor 11 Dec '09

11 Dec '09

Hello. I wrote a small patch to add colorization to syslog-ng. The idea is to apply different color schemes to messages (that are going to a tty, for example) depending on their type and importance and using standard syslog-ng filters. IMHO this makes tty log much more readable. There is a new keyword: 'scheme'. "scheme { ... }" construction binds colors to message fields and "scheme()" field of "log {...}" adds scheme filter that colorizes the output (before it will be sent to destination). The filter uses ANSI escape sequences but it's easy to extend it to anything else. Color support is enabled with defining 'ENABLE_COLOR' or using '--enable-colors' in configure script. If you'll find it useful you can use it :) The patch and example (draft) syslog-ng.conf are attached. -- Gaydov Victor. Practically perfect people never permit sentiment to muddle their thinking. -- Mary Poppins

5 6

Syslogd and syslog-ng
by Rocco Scappatura 11 Dec '09

11 Dec '09

Hello, I'm configuring a centralized loghost (SLES 10 SP3) based on syslog-ng, for collecting mail log of different clients. Several machine which are entitled to log its mail logs on the central loghost (Debian), still use old syslogd as logger system. I configured one of this to send its mail logs to the loghost putting: mail.* @syslogmi01.local in /etc/syslog.conf file and restarted the service. Moreover on the loghost I set: source mail_gateway_src { udp(ip("0.0.0.0") port(514)); }; filter ernesto_mail_f { facility(mail); }; destination d_ernesto_mail { file("/var/log/ernesto_mail"); }; log { source(mail_gateway_src); filter(ernesto_mail_f); destination(d_ernesto_mail); }; And restarted it. But no file are created and so no log are saved on the loghost. With tcpdump I see the the log comes on the loghost, but syslog doesn't want to catch them. # tcpdump -vvv host 192.168.252.107 and port 514 tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes 18:04:57.046952 IP (tos 0x0, ttl 64, id 3157, offset 0, flags [DF], proto: UDP (17), length: 358) 192.168.252.107.syslog > syslogmi01.local.syslog: SYSLOG, length: 330 Facility mail (2), Severity info (6) Msg: postfix/smtpd[30946]: NOQUEUE: reject: RCPT from u[|syslog] 18:04:57.046962 IP (tos 0x0, ttl 64, id 3158, offset 0, flags [DF], proto: UDP (17), length: 360) 192.168.252.107.syslog > syslogmi01.local.syslog: SYSLOG, length: 332 I saw that the logs coming from Debian machine (so using syslogd) miss of timestamp and host name that generates host. Could be this a source of incompatibility between syslogd and syslog-ng? Or there is something that I'm wrong? Thanks in advance, rocsca

5 10

re-opening it's logs
by Tim Rupp 11 Dec '09

11 Dec '09

will syslog-ng re-open it's log files if it finds that the size of the log has suddenly decreased or the inode has changed or something similar that may be caused by a logrotate? Or should I just get in the habit of sighup'ing the process? I have a log file on AFS space that seems to have stopped being written to, but log files on the local system are still chugging along just fine, so I think an errant rotate script is taking place on that remote system and moving the file and syslog-ng isn't noticing. My last log entry in the syslog-ng log is this Dec 7 15:56:02 clogger syslog-ng[23479]: Initializing destination file writer; template='/afs/files/data/k5logs/tmp/kdc.log', filename='/afs/files/data/k5logs/tmp/kdc.log' but, and this is my next question, that timestamp is ~8 hours off. It's 9:47 am here and syslog-ng is writing Dec 8 01:38:01 into the syslog-ng.log file. `date` command on the system is reporting [root@clogger tarupp]# date Tue Dec 8 09:43:16 CST 2009 and other log files have a correct date in them. The related log config entries are source s_internal { internal(); }; destination d_syslog { file("/logging/syslog-ng/syslog-ng.log"); }; log { source(s_internal); destination(d_syslog); }; a snippet of the syslog-ng.log file is Dec 8 01:44:01 clogger syslog-ng[23479]: Closing log transport fd; fd='1043' Dec 8 01:44:01 clogger syslog-ng[23479]: Destination timed out, reaping; template='/logging/syslog-ng/$HOST/secure', filename='/logging/syslog-ng/fnd0763/secure' and there are a bunch of these going into the file. The "seconds" in the timestamp also do not change even though time is. For example, that 01 seconds could stay there for a minute or so for each log entry and then a Dec 8 01:45:31 clogger syslog-ng[23479]: Reaping unused destination files; template='/logging/syslog-ng/$HOST/messages' will come along and increment the seconds by 1; not to the new time though. If I had to venture a guess, maybe the time_reap value is too low? (ours is set to the default) and maybe we have such a large number of one-shot hosts that syslog-ng is getting backed up writing to that log file? Any ideas? Thanks, Tim

2 1

Re: [syslog-ng] Macro expansion in program destination
by chris packham 11 Dec '09

11 Dec '09

This is intentional behaviour. Syslog-ng starts any "program()" when syslog-ng is started and it expects that the program will perform read from stdin and block waiting for data. This protects syslog-ng againts the overhead of forking a new process everytime a matching log message is seen. You can still use filters so I'd suggest modifying your program so that it can have the $S_DAY fed to it at the start of each log message. >>> Evan Rempel <erempel(a)uvic.ca> 12/04/09 4:07 PM >>> Is it possible to use macro expansion in the program destination. I wanted to write something like destination mydest{ program("/some/path/to/filter/program -options >> /path/log.$S_DAY"));}; I was hoping that syslog-ng would not open this destination until the first message was ready for this destination. I was hoping that syslog-ng would open a new one when the $S_DAY changed, allowing me to have each day of logs processed by my program with outputs based on day. Does anyone know if macro expantion can be used on program destinations? If not, can someone explain what I am overlooking that makes this a bad thing? Thanks, -- Evan Rempel ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html

4 7

Undefined symbol: yylex, 3.1beta1
by Marvin Nipper 11 Dec '09

11 Dec '09

I decided to try to at least build the 3.1beta1 package, to be sure that there were no new "compilation oddities" introduced. It seems to have fallen over while trying to compile the pdbtool component: gcc -DHAVE_CONFIG_H -I. -I.. -I/usr/local/include/glib-2.0 -I/usr/local/lib/glib-2.0/include -I/usr/local/include/eventlog -I/usr/local/ssl/include -DLIBNET_LIL_ENDIAN -D_GNU_SOURCE -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -g -O2 -Wall -MT pdbtool.o -MD -MP -MF .deps/pdbtool.Tpo -c -o pdbtool.o pdbtool.c mv -f .deps/pdbtool.Tpo .deps/pdbtool.Po gcc -g -O2 -Wall -o pdbtool pdbtool.o libsyslog-ng.a -lpthread -ldoor -lsocket -lrt -lnsl -lfl -L/usr/local/lib -lglib-2.0 -L/usr/local/lib -levtlog -L/usr/local/ssl/lib -lssl -lcrypto -lsocket -lnsl -ldl -lz -lsocket -lnsl -lnet -lresolv -lpthread Undefined first referenced symbol in file yylex /usr/sfw/lib/gcc/i386-pc-solaris2.10/3.4.3/../../../libfl.so ld: fatal: Symbol referencing errors. No output written to pdbtool collect2: ld returned 1 exit status *** Error code 1 make: Fatal error: Command failed for target `pdbtool' Current working directory /var/opt/packages/syslog-ng-3.1beta1/src *** Error code 1 The following command caused the error: failcom='exit 1'; \ for f in x $MAKEFLAGS; do \ case $f in \ *=* | --[!k]*);; \ *k*) failcom='fail=yes';; \ esac; \ done; \ dot_seen=no; \ target=`echo all-recursive | sed s/-recursive//`; \ list='src tests doc contrib debian tgz2build'; for subdir in $list; do \ echo "Making $target in $subdir"; \ if test "$subdir" = "."; then \ dot_seen=yes; \ local_target="$target-am"; \ else \ local_target="$target"; \ fi; \ (CDPATH="${ZSH_VERSION+.}:" && cd $subdir && make $local_target) \ || eval $failcom; \ done; \ if test "$dot_seen" = "no"; then \ make "$target-am" || exit 1; \ fi; test -z "$fail" make: Fatal error: Command failed for target `all-recursive' Current working directory /var/opt/packages/syslog-ng-3.1beta1 *** Error code 1 make: Fatal error: Command failed for target `all' Note that I just built 3.0.5 (today, as well), with no problems at all. I'm not done googling, but at least one site, seems to indicate that this might be a make-related issue: http://mail.opensolaris.org/pipermail/tools-gcc/2006-May/000121.html This is Solaris 10 x86, running the original (albeit patched) gcc/make components. Here's the summary from the configure that preceded the make activities: syslog-ng Open Source Edition 3.1beta1 configured Compiler options: compiler : gcc compiler options : -g -O2 -Wall -I/usr/local/include/glib-2.0 -I/usr/local/lib/glib-2.0/include -I/usr/local/include/eventlog -I/usr/local/ssl/include -DLIBNET_LIL_ENDIAN -D_GNU_SOURCE -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 linker flags : -lpthread prefix : /usr/local linking mode : dynamic Features: Sun STREAMS support : yes Sun Door support : yes Debug symbols : no GCC profiling : no Memtrace : no IPV6 support : yes spoof-source support : yes tcp-wrapper support : no SSL support : yes SQL support : no Linux capability support : no PCRE support : no Env wrapper support : no Not dying (or needing) to get 3.1 running. Just submitting this to try to sort out any compilation problems while this is all still in beta. APOLOGIES (in advance) if I've just done something moronically stupid! Let me know if there's some other input that it would be useful to provide, and/or if you have any thoughts/suggestions as to what might be happening. THANKS! Marvin This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.

2 1

Syslog UDP extra packet?
by Syslog Beginner 11 Dec '09

11 Dec '09

Hello Syslog Experts, I have a question on the syslog-ng. I use syslog-ng 3.0.2. I tried to setup syslog-ng to forward syslog messages to other host. I tested 2 destination driver, udp() and syslog() as per below. test 1: udp("10.x.x.x" port(514)) test 2: syslog("10.x.x.x" transport("udp") port(514)); I found that with udp() driver, syslog-ng just forwards the incoming log messages to external host. No problem. However, with the syslog() driver, I found that syslog-ng generate 2 udp packets packet1... contains only 4 bytes in payload, I think this is message length?? packet2... is the actual syslog udp packet. Is it possible to disable the first packet? this just create the overhead unnecessarily? Please advise. Thanks. Cheers,

2 1

Re: [syslog-ng] Question about verisign certs
by Jimmy McDonald 11 Dec '09

11 Dec '09

The verify works fine. The client gives the following error now that things have been configured to get the verify to return OK tss4s003 syslog-ng[9739]: Syslog connection accepted; fd='8', client='AF_INET(10.139.64.126:3766)', local='AF_INET(xx.xx.xx.xx:8514)' Dec 9 16:24:25 tss4s003 syslog-ng[9739]: Certificate validation failed; subject='OU=Class 3 Public Primary Certification Authority, O=VeriSign\, Inc., C=US', issuer='OU=Class 3 Public Primary Certification Authority, O=VeriSign\, Inc., C=US', error='invalid CA certificate', depth='2' Dec 9 16:24:25 tss4s003 syslog-ng[9739]: SSL error while reading stream; tls_error='SSL routines:SSL3_GET_CLIENT_CERTIFICATE:no certificate returned' Dec 9 16:24:25 tss4s003 syslog-ng[9739]: I/O error occurred while reading; fd='8', error='Connection reset by peer (131)' Dec 9 16:24:25 tss4s003 syslog-ng[9739]: Syslog connection closed; fd='8', client='AF_INET(10.139.64.126:3766)', local='AF_4:27:03 PM: Chris: INET(xx.xx.xx.xx:8514)' Sent from my iPhone... So expect typos. On Dec 6, 2009, at 10:24 AM, Balazs Scheidler <bazsi(a)balabit.hu> wrote: > On Fri, 2009-12-04 at 11:20 -0500, Jimmy McDonald wrote: >> I have syslog-ng installed and configured for mutual authentication. >> When the client was using a self signed cert it worked. The client >> now >> has a verisign cert. >> >> Imported the CA and an intermediate cert and the public cert so the >> chain is complete but the error on authentication says: invalid CA >> certificate, depth=2 >> >> I have tried putting the pub cert in cert.d with the ca and >> intermediate in ca.d. I also tried putting the intermediate in cert.d >> >> I made a hash for the ca and the intermediate. >> >> I'm not really sure what else to try. Any help would be greatly >> appreciated. > > Can you ensure that "openssl verify" validates your cert in case > syslog-ng doesn't? > > -- > Bazsi > > > ______________________________________________________________________________ > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng > Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng > FAQ: http://www.campin.net/syslog-ng/faq.html >

2 1

syslog-ng 3.1 administrator guide draft
by Robert Fekete 09 Dec '09

09 Dec '09

Dear syslog-ng users, A draft version of the 3.1 administrator guide is available at http://people.balabit.hu/frobert/syslog-ng-v3.1admin-guide-en-1ed-draft.pdf Its mainly interesting if you are giving pattern databases a try: Section 8.6.2.3. Creating pattern databases now describes the latest V3 format, and in Appendix 1 you will find a man page for pdbtool. Also, some experimental/unsupported/unchecked/for-testing-only pattern databases are now available at http://www.balabit.com/downloads/files/patterndb-snapshot/, but Marci will write in more detail about that. Have fun! Robert

1 0

syslog-ng OSE 3.0.5 released
by Balazs Scheidler 09 Dec '09

09 Dec '09

Dear syslog-ng users, I've uploaded syslog-ng OSE 3.0.5 to our webservers recently which has some quite important bugfixes. I'd recommend any serious deployment of syslog-ng 3.x to upgrade to this release. The results of the last four months were accumulated into this release. Some statistics for those who like them: 37 files changed, 885 insertions(+), 468 deletions(-) In this timespan, altogether 36 patches were produced by the following people: Anthony Lineham (1) Balazs Scheidler (25) Marton Illes (2) Tamas Pal (3) Tevesz Andras (4) Viktor Tusa (1) Some of the bugreporters and testers also deserve credit, who were: Tamas Monos Sandor Geller Imre Lazar Arkadiusz Miśkiewicz Elan Ruusamäe Thank you for your contribution. Also please note that a new feature release of syslog-ng was published in recent weeks, numbered version 3.1beta1. That beta version has improvements in various areas you might want to check out. We'll only include bugfixes in syslog-ng 3.0.x in the future. The list of changes: 3.0.5 Thu, 03 Dec 2009 18:02:56 +0100 Bugfixes: * Fixed a possible hang that caused syslog-ng to completely stop processing messages, in case SQL destination is enabled in configure. The bug does not need a working SQL destination. * Fixed a possible 100% CPU usage in case an udp destination went down. * Fixed host_override/program_override options that were broken in 3.0.4. * Fixed the SQL destination to avoid dropping the output queue in case a HUP was received. * Fixed a possible segmentation fault when parsing syntactically incorrect patterndb files. * Fixed ESTRING parser to include the ending quote character in the result. * Report an error in case there's no terminating brace in template macro references, e.g. no closing brace in ${macro} format. * Fixed FreeBSD init script to add $prefix/sbin to the PATH environment variable. This problem caused syslog-ng not to start up on boot. * Fixed FreeBSD /dev/klog message reading, follow_freq() needs to default to zero in this case (just like Linux /proc/kmsg). * Fixed a possible segfault in SIGHUP processing if an included file contains a syntax error. * RFC5424 over UDP erroneously required frame length, just like TCP, this is now fixed. * Fixed HP-UX 11iv2 compilation issue related to pread/pwrite compile errors. * The flow-control flag was erroneously enabled if any of the log statements contained either "final" or "fallback". * Fixed rewrite processing, in some cases a rewritten value could be propagated to wrong branches in the log processing tree. Other changes: * Added support for Cisco ASA timestamps. * Added version checks for PCRE in the configure script. * Integrate with system LSB init scripts a bit better, with 3.0.5 the init script status reports should nicely integrate with the host system. Happy logging, -- Bazsi

1 0

Lock up problems when using unix-dgram("/dev/log") type on
by Lance Laursen 09 Dec '09

09 Dec '09

Hello, I'm having problems with machines eventually hanging on all processes that write to /dev/log when using unix-dgram("/dev/log") with Syslog-NG 3.0.4. The servers run fine for a while and hum along as expected. Unfortunately the success does not last, with various programs completely hanging after an undetermined time. Having an existing root shell as this happens allows me to kill syslog-ng, freeing up all locks. Repro'ing this is...well, annoying. I have 300+ servers running this build of syslog-ng fine, all using unix-stream(). The 4 servers that are locking up are the only ones I have running unix-dgram(). Completely fresh ubuntu 8.04 installs with syslog-ng 3.0.4, identical to all other boxes aside from the one syslog-ng option. I've got strace output that is hanging after programs try to write to /dev/log as well. I'm currently doing a repro by running "while true ; do logger -p local0.info ...longest_message_possible... ; sleep 1s ; done" in non-exact science and have managed to pile things up after just over 120 messages, or two minutes. I can still hop around as root, but all programs that try to write to /dev/log pile up. The pile up seems to be log-size/throughput based, not time-based after some rudimentary tests - though it could be something random that is triggering it while my crappy tests are running. My next test plans to have small log messages in very rapid succession. I'm running: # uname -a Linux tny0032 2.6.24-24-generic #1 SMP Tue Jul 7 19:10:36 UTC 2009 x86_64 GNU/Linux # cat /etc/debian_version lenny/sid (ubuntu 8.04) Here's my source definition: # all known message sources source s_all { internal(); unix-dgram("/dev/log"); file("/proc/kmsg" program_override("kernel: ")); }; Here's some strace output that locks after trying to write to /dev/log: # strace su - lance ... ... stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=118, ...}) = 0 socket(PF_FILE, SOCK_DGRAM, 0) = 3 fcntl(3, F_SETFD, FD_CLOEXEC) = 0 connect(3, {sa_family=AF_FILE, path="/dev/log"}, 110) = 0 sendto(3, "<85>Dec 4 02:12:43 su[7086]: pa"..., 148, MSG_NOSIGNAL, NULL, 0 # strace logger -p local0.info lalala produces the same lock-point as above. I thought dgram should be connectionless? I'm not sure how syslog-ng could be locking up resources. Has anyone seen this before? I will continue looking for a better repro case, if anyone has any ideas though shout. I am using unix-dgram solely because it does not break to a new log entry on NewLines. I was encountering a problem where, using unix-stream, lighttpd's multi-line log output was getting broken up into multiple syslog lines. This would have been fine, except when the new line is broken out and made into new log entries, the $hostname and $program fields get stripped out, leaving me with just $date $msg. This basically negated the ability to filter and relay logs effectively. I can elaborate further here if requested, but making unix-stream behave the same as unix-dgram with regards to multi-line log messages would solve all my problems. Thanks, -Lance

3 3