- syslog-ng - lists.balabit.hu

[Bug 32] New: syslog-ng stops listening after receiving a zero length udp packet
by bugzilla＠bugzilla.balabit.com 17 Feb '09

17 Feb '09

https://bugzilla.balabit.com/show_bug.cgi?id=32 Summary: syslog-ng stops listening after receiving a zero length udp packet Product: syslog-ng Version: 3.0.x Platform: Other OS/Version: Other Status: NEW Severity: major Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: ffs(a)ccn.net Type of the Report: bug Estimated Hours: 0.0 Created an attachment (id=12) --> (https://bugzilla.balabit.com/attachment.cgi?id=12) afsocket EOF patch Hi, if syslog-ng has an udp() source configured it stops listening after receiving a zero length udp packet. In debug mode it logs 'EOF occurred while reading'. After this message no udp messages get processed anymore. After a restart syslog-ng is logging again. As this bug is reproducible it could be abused for DOS attacks also. Tested on: Solaris 8/9 Attached is a patch for version 3.0.1. There might be a better way to solve this problem, but it's working for me. Thank you for syslog-ng! Florian -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

3 7

[Bug 33] New: syslog-ng 2.0.10 compile error over solaris
by bugzilla＠bugzilla.balabit.com 17 Feb '09

17 Feb '09

https://bugzilla.balabit.com/show_bug.cgi?id=33 Summary: syslog-ng 2.0.10 compile error over solaris Product: syslog-ng Version: 2.1.x Platform: PC OS/Version: Linux Status: NEW Severity: major Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: girl111_2002(a)hotmail.com Type of the Report: bug Estimated Hours: 0.0 gcc -g -O2 -Wall -o loggen loggen.o -lpthread -lresolv -lnsl -lrt -lsocket -ldoor -lfl -lglib-2.0 -L/usr/local/lib -levtlog -lwrap Undefined first referenced symbol in file yylex /usr/sfw/lib/gcc/sparc-sun-solaris2.10/3.4.3/../../../libfl.so deny_severity /usr/sfw/lib/gcc/sparc-sun-solaris2.10/3.4.3/../../../libwrap.so allow_severity /usr/sfw/lib/gcc/sparc-sun-solaris2.10/3.4.3/../../../libwrap.so ld: fatal: Symbol referencing errors. No output written to loggen collect2: ld returned 1 exit status *** Error code 1 make: Fatal error: Command failed for target `loggen' Current working directory /opt/software/syslog-ng/syslog-ng-2.0.10/tests/loggen *** Error code 1 The following command caused the error: set fnord $MAKEFLAGS; amf=$2; \ dot_seen=no; \ target=`echo all-recursive | sed s/-recursive//`; \ list='unit functional loggen'; for subdir in $list; do \ echo "Making $target in $subdir"; \ if test "$subdir" = "."; then \ dot_seen=yes; \ local_target="$target-am"; \ else \ local_target="$target"; \ fi; \ (cd $subdir && make $local_target) \ || case "$amf" in *=*) exit 1;; *k*) fail=yes;; *) exit 1;; esac; \ done; \ if test "$dot_seen" = "no"; then \ make "$target-am" || exit 1; \ fi; test -z "$fail" make: Fatal error: Command failed for target `all-recursive' Current working directory /opt/software/syslog-ng/syslog-ng-2.0.10/tests *** Error code 1 The following command caused the error: set fnord $MAKEFLAGS; amf=$2; \ dot_seen=no; \ target=`echo all-recursive | sed s/-recursive//`; \ list='src tests doc contrib solbuild tgzbuild'; for subdir in $list; do \ echo "Making $target in $subdir"; \ if test "$subdir" = "."; then \ dot_seen=yes; \ local_target="$target-am"; \ else \ local_target="$target"; \ fi; \ (cd $subdir && make $local_target) \ || case "$amf" in *=*) exit 1;; *k*) fail=yes;; *) exit 1;; esac; \ done; \ if test "$dot_seen" = "no"; then \ make "$target-am" || exit 1; \ fi; test -z "$fail" make: Fatal error: Command failed for target `all-recursive' Current working directory /opt/software/syslog-ng/syslog-ng-2.0.10 *** Error code 1 make: Fatal error: Command failed for target `all' -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 1

grouping of sources
by Joe Shaw 16 Feb '09

16 Feb '09

Hi, Is there a way to create a "group" of sources for log rules? This would be quite helpful when using the new include functionality for configuration files. To give you an idea of my setup: I have two files. One is a site config, which will be the same for every host. It contains global configuration and well-known destinations. The other file differs per-host, with different sources. I would like to be able to set up log rules for routing messages from those disparate sources to the well-known destinations inside the site config, but there doesn't seem to be any way to do it. Here's a pseudo-example. Site config: options { ... }; destination d_whatever { ... }; Host 1 config: source s_one { ... }; log { source(s_one); destination(d_whatever); }; Host 2 config: source s_two_alpha { ... }; source s_two_beta { ... }; log { source(s_two_alpha); source(s_two_beta); destination(d_whatever); }; Ideally, I'd be able to do something like this: Site config: options { ... }; destination d_whatever { ... }; log { source(s_aggregate); destination(d_whatever); }; Host 1 config: source s_one { ... }; source s_aggregate { source(s_one); }; Host 2 config: source s_two_alpha { ... }; source s_two_beta { ... }; source s_aggregate { source(s_two_alpha); source(s_two_beta); }; This setup maintains a nice, clean abstraction of the logging rules away from the source data. Is there any way to do something like this? Thanks, Joe

2 5

File descriptor leaks with 3.0.1
by chris packham 16 Feb '09

16 Feb '09

Hi, We're running a test where syslog-ng gets frequently sighup-ed (about every 5 seconds) and what we are finding is that after a while (sorry don't know specifically how may times) we run into a problem where syslog-ng starts experiencing errors reading its config. We are making use of the new 'include' feature (which is brilliant by the way). Here's a snippet of the internal() log 2009 Feb 17 00:05:01 syslog.notice syslog-ng[838]: Configuration reload request received, reloading configuration; 2009 Feb 17 00:05:04 syslog.notice syslog-ng[838]: Configuration reload request received, reloading configuration; 2009 Feb 17 00:05:37 syslog.notice syslog-ng[838]: Configuration reload request received, reloading configuration; 2009 Feb 17 00:05:43 syslog.notice syslog-ng[838]: Configuration reload request received, reloading configuration; 2009 Feb 17 00:05:46 syslog.err syslog-ng[838]: Error opening include file; filename='/etc/syslog-conf.d/trace-cfg', depth='1' 2009 Feb 17 00:05:46 syslog.notice syslog-ng[838]: Configuration reload request received, reloading configuration; 2009 Feb 17 00:05:47 syslog.err syslog-ng[838]: Error accepting new connection; error='Too many open files (24)' 2009 Feb 17 00:05:47 syslog.err syslog-ng[838]: Error accepting new connection; error='Too many open files (24)' /proc/$(pidof syslog-ng)/fd is also hinting that the included files aren't being closed. lr-x------ 1 root root 64 Feb 17 00:05 4042 -> /tmp/etc/syslog-conf.d/trace-cfg lr-x------ 1 root root 64 Feb 17 00:05 4043 -> /tmp/etc/syslog-conf.d/1_standalone lr-x------ 1 root root 64 Feb 17 00:05 4044 -> /tmp/etc/syslog-conf.d/buffered lr-x------ 1 root root 64 Feb 17 00:05 4045 -> /tmp/etc/syslog-conf.d/console lr-x------ 1 root root 64 Feb 17 00:05 4046 -> /tmp/etc/syslog-conf.d/email lr-x------ 1 root root 64 Feb 17 00:05 4047 -> /tmp/etc/syslog-conf.d/host lr-x------ 1 root root 64 Feb 17 00:05 4048 -> /tmp/etc/syslog-conf.d/permanent lr-x------ 1 root root 64 Feb 17 00:05 4049 -> /tmp/etc/syslog-conf.d/stats lr-x------ 1 root root 64 Feb 17 00:05 4050 -> /tmp/terminal lr-x------ 1 root root 64 Feb 17 00:05 4051 -> /tmp/etc/syslog-conf.d/trace-cfg lr-x------ 1 root root 64 Feb 17 00:05 4052 -> /tmp/etc/syslog-conf.d/1_standalone lr-x------ 1 root root 64 Feb 17 00:05 4053 -> /tmp/etc/syslog-conf.d/buffered lr-x------ 1 root root 64 Feb 17 00:05 4054 -> /tmp/etc/syslog-conf.d/console lr-x------ 1 root root 64 Feb 17 00:05 4055 -> /tmp/etc/syslog-conf.d/email lr-x------ 1 root root 64 Feb 17 00:05 4056 -> /tmp/etc/syslog-conf.d/host lr-x------ 1 root root 64 Feb 17 00:05 4057 -> /tmp/etc/syslog-conf.d/permanent lr-x------ 1 root root 64 Feb 17 00:05 4058 -> /tmp/etc/syslog-conf.d/stats lr-x------ 1 root root 64 Feb 17 00:05 4059 -> /tmp/terminal Also seems to be present in the latest version checked out from the git repo but I'll do some more digging. Thanks, Chris

1 0

Timezone problem?!
by Charles Chan 16 Feb '09

16 Feb '09

Recently I use the syslog-ng for log collection environment encountered a time zone problem,hope somebody can help me.Thanks The environment is like this: The apache server(PDT timezone,UTC-8) forword apache log through syslog to the syslog-ng server（CST timezone，UTC＋8）. The server with same time zones no problem between the forward. Syslog-ng test configuration is as follows: destination int_apache (file ( "/ disk3/cookie_logs/pid1 / $ YEAR-$ MONTH-$ DAY / $ (HOST). apache.log" time_zone (-08:00) template ( "$ DAY-$ HOUR $ S_DAY-$ S_HOUR $ R_DAY-$ R_HOUR $ $ ISODATE $ MSGONLY \ n ") template_escape (no));); The original apache log is: 58.240.69.161 - - - [14/Jan/2009: 22:27:47 -0800] "GET / www.example.com/HTTP/1.1" 200 2028 "-" "Mozilla/4.0 ( compatible; MSIE 7.0; Windows NT 5.1) " The syslog-ng server generated log is: 14-22 14-22 14-22 2009-01-14T22 :27:47-08: 00 58.240.69.161 - - - [14/Jan/2009: 22:27:47 -0800] "GET / www.example. com / HTTP/1.1 "200 2028" - "" Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1) " Originally generated documents should be kept in /disk3/cookie_logs/pid1/2009-01-14 However, the actual file is stored in the /disk3/cookie_logs/pid1/2009-01-15 It can be seen, in fact, syslog-ng configuration item file() use the $DAY and template() use the $DAY not with a identical Macros. Is this the syslog-ng a bug? or I configuration errors. Who can help me answer you?Thanks!

2 8

Snare and MSGONLY macro
by Stewart James 16 Feb '09

16 Feb '09

Hi all, I have had an ongoing problem with snarecore and syslog-ng, this is also happening with loglogics lasso agent too. For some services it makes sense that the system just store the messages without the syslog header. For example IIS logs, apache logs and other misc application logs. Wanting to save having to perform any post processing I figured I would setup a specific listener for those applications, allowing syslog to log to /someplace/with/lots/of/storage/YYYY/MM/DD/raw.program.servername. Using both the windows agents mentioned above, the macros $PROGRAM and $MSGONLY break. $PROGRAM is expanded into some ungodly form (MSWinEventLog\0110\011System\011325\011Mon....) and the MSGONLY macro is garbled. My logic tells me this is due to sysl;og-ng not detecting the syslog header being sent by snare or lasso properly... or it is snare and lasso sending the incorrect header format. I can survive without the $PROGRAM macro working correctly. The MSGONLY macro is a bit more of a pain though, I would rather use the native capabilities of syslog-ng. I have just updated to 2.0.9 (from Debian Lenny) to confirm this is still a problem for me, I see 3.0.x is out but not sure if this is something that would be corrected if I went down that path. Thanks, Stewart

1 0

Parsing messages! help!
by ramesh p 15 Feb '09

15 Feb '09

How to replace the syslog source IP address with the correct loopback IP address before forwarding syslog messages to someother application? PLz give suggestion. How to parse and filter? Thanks in advance! Regards, Rams.

2 2

Compiling syslog-ng 3.01 on CentOS 5.2
by Liam Kirsher 15 Feb '09

15 Feb '09

Hi -- Just thought this might be useful to someone who is compiling on CentOS. There were a few minor issues that came up, but otherwise it went pretty smoothly. Documentation says to use this -- PKG_CONFIG_PATH=/usr/local/lib/pkgconfig:$PKG_CONFIG_PATH However, using that I ended up with a colon at the end, which didn't work, and which I didn't catch immediately... PKG_CONFIG_PATH=/usr/local/lib/pkgconfig: Also, I needed to export -- export PKG_CONFIG_PATH Maybe it could be export PKG_CONFIG_PATH=$PKG_CONFIG_PATH:/usr/local/lib/pkgconfig I did have to install newer versions of libdbi and libdbi-drivers, from http://libdbi.sourceforge.net/ For libdbi-drivers, remember to specify which ones you want during configuration: ./configure --with-pgqsl After that, this worked: LIBDBI_CFLAGS=-I/usr/local/include LIBDBI_LIBS="-L/usr/local/lib -ldbi" ./configure --sysconfdir=/etc/syslog-ng/ --enable-tcp-wrapper --enable-sql --enable-dynamic-linking (Make sure to link with libs/dirs for my newly installed libdbi which were in /usr/local/include and /usr/local/lib, not the system default ones if they were already installed.) Note that this installs the executables in the default location, which is /usr/local, but looks for the configuration file in the standard place, /etc/syslog-ng/. I adjusted the /etc/init.d/syslog-ng file so it would use the executable in /usr/local/sbin: exec="/usr/local/sbin/syslog-ng" Also, I needed to run ldconfig so that syslog-ng would find libevtlog.so.0 in /usr/local/lib. At that point, I was able to start using the previous (v. 1.6) configuration file, albeit with warnings. This is a Linux system, and initially I added --enable-linux-caps to the configure line, although I wasn't sure what that would do. I got an error and so, removed it. What exactly are linux-caps? Should I go back and try to re-compile with that option? Liam -- Liam Kirsher PGP: http://liam.numenet.com/pgp/

2 1

Missing libraries when trying to compile 3.0
by Tim Boyer 15 Feb '09

15 Feb '09

I'm trying to get 3.0 going - I'm running 2.2.6 currently. configure says I'm missing some libraries: ./configure ... checking for static OpenSSL libraries... no configure: error: static OpenSSL libraries not found (libssl.a and libcrypto.a), either link OpenSSL statically using the --enable-dynamic-linking, or install a static OpenSSL [root@princeton syslog-ng-3.0.1]# ...but the libraries are there: [root@princeton syslog-ng-3.0.1]# locate libssl.a libcrypto.a /usr/lib/libcrypto.a /usr/lib/libssl.a /usr/lib64/libcrypto.a /usr/lib64/libssl.a However, I used the --enable-dynamic-linking for the configure. It configured and compiled just fine. When I went to run, I got this: [root@princeton syslog-ng-3.0.1]# service syslog-ng start Starting system logger: /usr/local/sbin/syslog-ng: error while loading shared libraries: libevtlog.so.0: cannot open shared object file: No such file or directory [FAILED] even though _those_ libraries are there: [root@princeton syslog-ng-3.0.1]# ls -la /usr/local/lib/libevtlog.* -rw-r--r-- 1 root root 65048 Feb 11 11:50 /usr/local/lib/libevtlog.a -rwxr-xr-x 1 root root 822 Feb 11 11:50 /usr/local/lib/libevtlog.la lrwxrwxrwx 1 root root 18 Feb 11 11:50 /usr/local/lib/libevtlog.so -> libevtlog.so.0.0.0 lrwxrwxrwx 1 root root 18 Feb 11 11:50 /usr/local/lib/libevtlog.so.0 -> libevtlog.so.0.0.0 -rwxr-xr-x 1 root root 45369 Feb 11 11:50 /usr/local/lib/libevtlog.so.0.0.0 I'm not sure where to go from here - pointers in the right direction greatly appreciated. -- Tim Boyer Denman Tire Corporation tim(a)denmantire.com

3 3

Using tail option
by James B Horwath 15 Feb '09

15 Feb '09

All: This question refers to syslog-ng 2.0.10 running on AIX. If I use a global syslog-ng client file and define 2 text files that do not exist on every system. What happens when syslog-ng is run on a system that does not contain the files? Does it report errors, have performance problems? Example: source s_tail1 { file("/var/apache/log/access_log" follow_freq(1) flags(no-parse)); }; source s_tail2 { file("/var/apache/log/error_log" follow_freq(1) flags(no-parse)); }; Please advise, Jim ----------------------------------------- This message, and any attachments to it, may contain information that is privileged, confidential, and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are notified that any use, dissemination, distribution, copying, or communication of this message is strictly prohibited. If you have received this message in error, please notify the sender immediately by return e-mail and delete the message and any attachments. Thank you.

2 1

Ip filtering
by Chris Bland 15 Feb '09

15 Feb '09

Hi guys, I am having a problem filtering based on ip address. My predecessor had a config file with over 240 hosts. I just upgraded to 2.0.10-1 and starting getting YACC overflow stack errors. I condensed the config file by using regex. If I stop using DNS everything works fine. What I am trying to do is filter based on ip address and have each log stored in a directory with the servers dns name. My config looks like this options { sync (0); time_reopen (10); log_fifo_size (1000); long_hostnames (off); keep_hostname (yes); use_dns (yes); use_fqdn (no); create_dirs (no); }; destination servers { file("/var/log/syslog-ng/hosts/$HOST_FROM/$YEAR/$MONTH/$YEAR-$MONTH-$DAY" owner(root) group(staff) perm(0650) dir_perm(0750) create_dirs(yes)); }; destination teaneck { file("/var/log/syslog-ng/network/TEANECK/$HOST_FROM/$YEAR/$MONTH/$YEAR-$MONTH-$DAY" owner(root) group(staff) perm(0650) dir_perm(0750) create_dirs(yes)); }; destination dslswitches { file("/var/log/syslog-ng/network/TEANECK/Dslswitches/$HOST_FROM/$YEAR/$MONTH/$YEAR-$MONTH-$DAY" owner(root) group(staff) perm(0650) dir_perm(0750) create_dirs(yes)); }; filter f_teaneck { netmask("132.238.11.0/24"); }; filter f_usas { host("132.238.(7|8).*$"); }; filter f_dslswitches { host("132.238.21.40"); }; log { source(s_network); filter(f_usas); destination(servers); }; log { source(s_network); filter(f_teaneck); destination(teaneck); }; log { source(s_network); filter(f_dslswitches); destination(dslswitches); }; The netmask filters work by my host filters don't -Chris

2 1

typo in v.3.0 admin guide
by Philipp 14 Feb '09

14 Feb '09

Hi please note that in the online available syslog-ng-v3.0-guide-admin-en.pdf on page 54 this line seems to have typos in the template() part: destination d_file { file("/var/log/messages" template("$ISODATE $HOST $MSG\n"; template_escape(no))); }; Best regards, Philipp

2 2

question on logging templates: strange log lines
by Philipp 14 Feb '09

14 Feb '09

Dear list users In the near future I would like to migrate from syslog-ng 2 to 3.x. For this reason I've set up a linux testbox. Now, I have a problem I lack to see a solution for. The syslog-ng testing config: @version:3.0 ## GLOBAL options { chain_hostnames(no); keep_hostname(yes); flush_lines(0); stats_freq(3600); use_dns(persist_only); dns_cache_hosts(/etc/hosts); }; ## SOURCES source src { unix-stream("/dev/log"); }; source src-kernel { file("/proc/kmsg" program_override("kernel: ")); }; source src-syslog-ng { internal(); }; ## LOGGING TEMPLATES template t_filetmpl { template("$DATE $FACILITY.$PRIORITY $FULLHOST $MSGHDR$MSG\n"); template_escape(no); }; ## DESTINATIONS destination local4 { file("/var/log/local4.log" template(t_filetmpl)); }; ## FILTERS filter f_local4 { facility(local4); }; ## LOGS log { source(src); filter(f_local4); destination(local4); }; For testing I use the logger command: ` logger -p local4.info -t test testnachricht2`. This configuration causes, and I do not see why, lines like these in the /var/log/local4.log file. [empty line] testnachricht2 test: donald.info local4Feb 11 14:26:01Feb 11 14:26:01 local4.info donald test: testnachricht2 However, if I replace the line below "## DESTINATIONS" with destination local4 { file("/var/log/local4.log" template("$DATE $FACILITY.$PRIORITY $FULLHOST $MSGHDR$MSG\n")); }; all is perfect. The logged line looks like this: Feb 11 14:30:22 local4.info donald test: testnachricht2 Please could you tell me what I've missed? Thanks in advance, Philipp

2 3

Strange template definition problem
by Peter Eckel 14 Feb '09

14 Feb '09

Hi, I am currently using the Open Source edition of syslog-ng version 3.0.1 on RedHat Linux 5.3 and 3.9. So far, there are not too many problems, but one strangeness occurs when defining templates. When I define the template inside the 'destination' configuration, everything works as expected: # + # Definition 1: Works fine. # - source s_local { unix-stream( "/dev/log" ); file( "/proc/kmsg" ); udp( ip( "127.0.0.1" ) port(514) ); tcp( ip( "127.0.0.1" ) port(514) keep-alive(yes) ); }; destination d_test { file( "/opt/sec/var/log/test" template( "$ISODATE -- $FACILITY -- $PRIORITY -- $HOST -- $MESSAGE\n----------\n" ) ); }; log { source(s_local); destination(d_test); }; The log entries look like this: >2009-01-22T19:33:35+01:00 -- authpriv -- notice -- linux4 -- LOG5[17309:3086130064]: syslog_in connected from 192.168.42.129:41105 >---------- >2009-01-22T19:33:35+01:00 -- authpriv -- notice -- linux4 -- LOG5[17309:3086060432]: syslog_in connected from 192.168.42.133:33950 >---------- >2009-01-22T19:33:35+01:00 -- authpriv -- notice -- linux4 -- LOG5[17309:3085990800]: syslog_in connected from 192.168.42.128:43163 >---------- So far, so good. But when I try global templates, I get a rather strange result: # + # Definition 2: Fails miserably. # - source s_local { unix-stream( "/dev/log" ); file( "/proc/kmsg" ); udp( ip( "127.0.0.1" ) port(514) ); tcp( ip( "127.0.0.1" ) port(514) keep-alive(yes) ); }; template t_test { template( "$ISODATE -- $FACILITY -- $PRIORITY -- $HOST -- $MESSAGE\n----------\n" ); }; destination d_test { file( "/opt/sec/var/log/test" template( t_test ) ); }; log { source(s_local); destination(d_test); }; What I'm getting is this (including marked blank lines): > >---------- >-- LOG5[17309:3086130064]: syslog_in connected from 192.168.42.129:41105 -- linux4 -- notice -- authpriv2009-01-22T19:33:35+01:0020 >09-01-22T19:33:35+01:00 -- authpriv -- notice -- linux4 -- LOG5[17309:3086130064]: syslog_in connected from 192.168.42.129:41105 >---------- > >---------- >-- LOG5[17309:3086060432]: syslog_in connected from 192.168.42.133:33950 -- linux4 -- notice -- authpriv2009-01-22T19:33:35+01:0020 >09-01-22T19:33:35+01:00 -- authpriv -- notice -- linux4 -- LOG5[17309:3086060432]: syslog_in connected from 192.168.42.133:33950 >---------- > >---------- >-- LOG5[17309:3085990800]: syslog_in connected from 192.168.42.128:43163 -- linux4 -- notice -- authpriv2009-01-22T19:33:35+01:0020 >09-01-22T19:33:35+01:00 -- authpriv -- notice -- linux4 -- LOG5[17309:3085990800]: syslog_in connected from 192.168.42.128:43163 It looks a bit as if all elements of the filled template are output twice: First in reverse order, then again from start in correct order. This is reproducable on several machines. The only difference is the use of the global template in the second case instead of the inline template in the first. Am I making some stupid mistake, or is this a bug in syslog-ng? Best regards and thanks in advance for any suggestions, Peter. -- ------------------------------------------------------------------------- Peter Eckel System- und Netzwerkmanagement Postfach 1463 D-61284 Bad Homburg v.d.H. Tel.: +49 6172 969495 Mobil: +49 172 6812107 Fax: +49 6172 969845 EMail: Peter.Eckel(a)Eckel-EDV.de -------------------------------------------------------------------------

2 4

patches ported from syslog-ng PE 3.0.2
by Balazs Scheidler 14 Feb '09

14 Feb '09

Hi, Just as I outlined in my previous mail, here goes the list of patches that were ported to syslog-ng OSE from the PE 3.0.2 release. You can find more information in the git tree at http://git.balabit.hu/, but feel free to ask specific questions if you need further information. Balazs Scheidler (30): don't use long double in the parser [tgzbuild] fixed --with-pidfile-dir configure option Fixed: slng_start checks whether syslog-ng already running before starting a new session. (fixes: #16011) Fixed: logic error when checking whether an instance already running. (fixes: #16011) Fixed: On SuSE -n no longer echoed (fixes: #16025) Fixed: obscure missing file error on HP-UX (fixes: #16710) Fixed: Init scripts link /var/run/syslog.pid to syslog-ng's pid. (fixes: #16710) Fixed: Solaris 10 method script links /var/run/syslog.pid to syslog-ng's pid. (fixes: #16710) [config parser] fixed a failed assert in case an include directory does not exist [security] user names were not resolved properly, do it right this time fixed compilation warnings fixed LogMessageMatch structure layout on big-endian 32 bit machines (like PA-RISC) Fixed: init.d script uses different syslogd pid on redhat (fixes: #16710) Fixed: init script don't start syslog-ng when it doesn't run (fixes: #16011) really only move to the newly rotated logfile if the new one already has data [logproto] remember "encoding" value, so that log messages can contain it [logproto] properly handle incomplete encodings (fixes: #15680) [logproto] don't let the conversion buffer increase to unlimited sizes Fixed: init script on solaris 8 links /etc/syslog.pid to pidfile. (fixes: #16710) [security] enable SSL X.509 policy check [tlscontext] reorganize cert validation somewhat, fixed "required-untrusted" (fixes: #16671) [tlscontext] added basicConstraints validation even for trusted CA certificates [afsocket] fixed a possible reference of invalid file descriptors [main] never initialize the configuration in the supervisor process kill the syslog-ng process if reverting to the old config fails fixed compilation error on FreeBSD [templates] use the correct timezone offset when time_zone_info_get_offset() returns -1 [config parser] accept newly introduced keywords as identifiers (fixes: #17653) [config parser] removed unused KW_FRAMED keyword [configure] change default linking mode to "dynamic" Tamas Pal (1): Fixed: on linux, dash no longer prinst -e (fixes: #17559) -- Bazsi

1 0

syslog-ng OSE patch flow
by Balazs Scheidler 14 Feb '09

14 Feb '09

Hi, In order to improve transparency in the syslog-ng development process, I'm going to try to improve the communication about the changes that go into syslog-ng OSE. Currently I see three major ways to get patches into syslog-ng: * Patches that get implemented because one of the list members request it are already posted on this list. * Patches that are submitted to bugzilla.balabit.com are also posted here, because of the bugzilla -> email gateway that I set up. * A third portion of patches that are the results of BalaBit's internal development projects were not posted here so far, simply because I took the internal git tree and ported patches to the open source branch that were relevant. I hope to improve the latter point. At first I thought about "submitting" the patches openly to this mailing list, but then I thought it'd be a mail storm every time I started doing the patch porting. (for instance just today I ported over 30 patches) Also it is not the way I do the work: I use git directly, doing this based on emails would be more difficult for me. Therefore I decided to post a single mail, every time I port patches back-and-forth. This mail would contain a summary for the patches I ported. The summary would be generated using "git-shortlog", more details are always available in the public git repositories at "git.balabit.hu". Currently the author information of my git trees are not perfect, but some changes were already made to improve the accuracy of these fields. Hopefully by the end of the next syslog-ng PE cycle, these would contains the names for the guys who actually worked on this stuff. That's the basic idea. I'm going to submit the current list of patches in a separate mail. -- Bazsi

1 0

3.01 facility(syslog) returns an error
by Tim Boyer 13 Feb '09

13 Feb '09

Continuing with my 3.01 install. My .conf files contains on line 61: filter fprinceton_syslog { facility(syslog); }; [root@princeton src]# ./syslog-ng -s syntax error in /usr/local/etc/syslog-ng.conf at line 61. However, if I change that to the numeric equivalent: filter fprinceton_syslog { facility(5); }; [root@princeton src]# ./syslog-ng -s [root@princeton src]# the error goes away. -- Tim Boyer Denman Tire Corporation

2 2

syslog-ng logging with wrong year
by Mayashankar Mishra 10 Feb '09

10 Feb '09

I have syslog-ng 2.1.1 running, still i face the problem when the date changed from 31 Dec 2008 to 1 Jan 2009, some of the log files still log as 1 Jan 2008 Is it resolved in higher version, and if yes which ? Mayashankar Mishra Consultant E-mail : mmishra(a)isabel.eu Tel : +32 (0)2 403.18.84 Fax : +32 Isabel NV/S.A. Keizerinlaan 13-15 Boulevard de l'Imp?ratrice 1000 Brussels - Belgium RPR Bruxelles / RPM Brussel: BE 0455 530 509 www.isabel.eu<http://www.isabel.eu/> www.zoomit.eu<http://www.zoomit.eu/> Zoomit is a Registered Trademark of Isabel NV/S.A. Disclaimer : www.isabel.eu/gps/en/disclaimer/mailing.php<http://www.isabel.eu/gps/en/disclaimer/mailing.php>

2 2

Still not working at the most basic level
by lance raymond 10 Feb '09

10 Feb '09

ok, so can anyone say how to troubleshoot the most basic scenario to start. 1 server, 1 client, have client write logs to server? I will show both config files, please note tI have tried both UDP and TCP and neither work. I did notice doing an lsof |grep syslog that the port does change from udp to tcp so I know it's reading the config file, I just don't know how to turn on some debugging or another way to trace the log as it's still being written to the client. ok, so *server config is;* #source s_remote { udp(); }; source s_remote { tcp(); }; ## This will create seprate file for each client on central log server and log http messages destination d_clients { file("/var/log/web.$HOST.log"); }; log { source(s_remote); destination(d_clients); }; ################################################################# options { sync (0); time_reopen (10); log_fifo_size (1000); long_hostnames (off); use_dns (no); use_fqdn (no); create_dirs (no); keep_hostname (yes); }; source s_sys { file ("/proc/kmsg" log_prefix("kernel: ")); unix-stream ("/dev/log"); internal(); #udp(ip(0.0.0.0) port(514)); }; ## This will log local http messages to defined file destination send_http_logs { file("/var/log/web.log"); }; filter send_http_logs { program("httpd.*"); }; log { source(s_sys); filter(send_http_logs); destination(send_http_logs); }; and the server shows the following running process; root 30945 0.0 0.0 7240 636 ? Ss 11:24 0:00 /sbin/syslog-ng -p /var/run/syslogd.pid *Client side:* options { sync (0); time_reopen (10); log_fifo_size (1000); long_hostnames(on); use_dns(yes); dns_cache(yes); use_fqdn(no); create_dirs (yes); keep_hostname (yes); perm(0640); dir_perm(0750); }; source s_sys { file ("/proc/kmsg" log_prefix("kernel: ")); unix-stream ("/dev/log"); internal(); }; #destination send_http_logs { udp("192.168.2.54" port(514)); }; destination send_http_logs { tcp("192.168.2.54" port(514)); }; filter send_http_logs { program("httpd.*"); }; log { source(s_sys); filter(send_http_logs); destination(send_http_logs); }; and the running process are; root 1607 0.0 0.0 6216 904 ? Ss 2008 0:48 syslogd -m 0 root 28374 0.0 0.0 7368 564 ? Ss 11:24 0:00 /sbin/syslog-ng -p /var/run/syslogd.pid Not sure if the client is supposed to have 2 processes or not, but any help is really appreciated.

2 3

[patch] Fix build with --disable-ssl
by Joe Shaw 09 Feb '09

09 Feb '09

Hi, Here's a patch to fix building without SSL. Without it, tlstransport.h is included by tlstransport.c and things break because SSLisms are undefined. Thanks, Joe

3 8

Cannot compile loggen
by fredzy padzy 09 Feb '09

09 Feb '09

Hi everyone ! Ok, i'm quite ashamed to ask this ... but i'm not very good in compiling, and i'm sure it must be easy to solve But i do not succeed ! I discovered that php-syslog-ng provide loggen, written by bazsi. I need this tool since i must benchmark my syslog-ng server. The problem is that our customized version of this OS lacks a lot of librairies and i cannot compile loggen ( FI, i use a pkg of syslog-ng for solaris, since i couldn't compile it, because of too many librairies that where missing) i tried to compil loggen, but didn't succeed :( I tried a simple "gcc loggen.c -o loggen" and i was told : loggen.c:1:20 config.h:No such file or dorectory In which package should i find this config.h file ? because a lot of other tools (like perl) have one The make command give me another error like it doesn't find any "../../ configure.in" which is normal i think. How did you guys managed to compil it ? thanks in advance :)

1 0

Compile 3.0.1 on RHEL 5 with sql support
by Henk van Lingen 09 Feb '09

09 Feb '09

Hi, I'm new to syslog-ng and I'm trying to compile 3.0.1 on a RHEL 5.3 system. I want to use the log to PostgreSQL features so I'm trying to enable sql support. For this I've installed the libdbi packages: root@:~-# yum list '*libdbi*' Loaded plugins: rhnplugin, security Installed Packages libdbi.i386 0.8.1-2.1 installed libdbi.x86_64 0.8.1-2.1 installed libdbi-dbd-pgsql.x86_64 0.8.1a-1.2.2 installed libdbi-devel.i386 0.8.1-2.1 installed libdbi-devel.x86_64 0.8.1-2.1 installed libdbi-drivers.x86_64 0.8.1a-1.2.2 installed However, a configure gives: SQL support : no And a "./configure --enable-dynamic-linking --enable-sql" gives a /home/henk/syslog-ng-3.0.1/src/afsql.c:672: undefined reference to `dbi_initialize' in the make phase. Any ideas what or which lib I am missing? BTW: a configure without --enable-dynamic-linking gives a configure: error: static OpenSSL libraries not found (libssl.a and libcrypto.a) although I have those files both in /usr/lib and /usr/lib64 I have PKG_CONFIG_PATH=/usr/local/lib/pkgconfig, as is needed for eventlog. Regards, -- Henk van Lingen, (o- -+ Netwerk & Telefonie, ICT Service Center /\ | Jenalaan 18a, kamer 1.40 v_/_ http://henk.vanlingen.net/ http://www.tuxtown.net/netiquette/

3 3

Issue installing syslog-ng 3.0.1 on debian etch
by steven 06 Feb '09

06 Feb '09

Hi there I'm actually struggling. 'Configure' is doing fine, but 'make' always drops me at : if gcc -DHAVE_CONFIG_H -I. -I. -I.. -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include -I/usr/include/eventlog -D_BSD_SOURCE -D__BSD_SOURCE -D__FAVOR_BSD -DHAVE_NET_ETHERNET_H -D_GNU_SOURCE -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -g -O2 -Wall -MT logmatcher.o -MD -MP -MF ".deps/logmatcher.Tpo" -c -o logmatcher.o logmatcher.c; \ then mv -f ".deps/logmatcher.Tpo" ".deps/logmatcher.Po"; else rm -f ".deps/logmatcher.Tpo"; exit 1; fi logmatcher.c: In function 'log_matcher_pcre_re_compile': logmatcher.c:509: error: 'PCRE_NEWLINE_ANYCRLF' undeclared (first use in this function) logmatcher.c:509: error: (Each undeclared identifier is reported only once logmatcher.c:509: error: for each function it appears in.) make[3]: *** [logmatcher.o] Error 1 make[3]: Leaving directory `/home/swn/syslog-ng-3.0.1/src' make[2]: *** [all-recursive] Error 1 make[2]: Leaving directory `/home/swn/syslog-ng-3.0.1' make[1]: *** [all] Error 2 make[1]: Leaving directory `/home/swn/syslog-ng-3.0.1' make: *** [build-stamp] Error 2 I tried on both i386 and amd64 architecture. If needed I can provide my config.log. Evtlog, libnet1 and libwrap* are installed. Oh yes, and of course this happens only when I --enable-pcre. Otherwise it compiles perfectly. Any ideas ? Thank you very much Steven

2 2

Re: [syslog-ng] [patch] log messages duplicated with syslog-ng version 3.0.1
by chris packham 06 Feb '09

06 Feb '09

OK thanks Bazsi. We'll pull down your changes. I must have missed the earlier message. >>> Balazs Scheidler <bazsi(a)balabit.hu> 02/05/09 11:11 PM >>> On Mon, 2009-02-02 at 11:18 +1300, chris packham wrote: > Hi, > > We were looking at migrating our system to use syslog-ng 3.0.1 and found that with our configuration we ended up with duplicated log messages. After some investigation this turned out to be a problem in the way templates were compiled. > > Attached is a patch that addresses this. you are right, but the same issue was reported the Peter Eckel. I've fixed this as a reply to his message, and also pushed a similar patch: commit 60aeab750e952b3a69714ea47e2aa31683c761fd Author: Balazs Scheidler <bazsi(a)balabit.hu> Date: Thu Feb 5 10:50:45 2009 +0100 fixed the references for non-inline templates this patch fixes template expansion for non-inline templates. The parser code caused these templates to be compiled as many times as they were referenced, which in turn concatenated the previously compiled value and the currently compiled value. -- Bazsi ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html

1 0

1st post on some basic 101 setup
by lance raymond 05 Feb '09

05 Feb '09

Basic scenario is we have 5 web servers in a cluster which I wish to centrally log. I was pointed to syslog-ng and the linuxquestions guys 50+ readings and not 1 response so found this list and will give it a try. Basically all the webservers are running FC5, and rpm -qa shows the following version; syslog-ng-1.6.12-1.fc5 I see some newer versions, not sure if I should/have to, but really just want to log the different vhosts apache logs to a central server. I read a few diff things a few places, and so far have the following conf files. *Server.conf* options { sync (0); time_reopen (10); log_fifo_size (1000); long_hostnames (off); use_dns (no); use_fqdn (no); create_dirs (no); keep_hostname (yes); }; source s_sys { file ("/proc/kmsg" log_prefix("kernel: ")); unix-stream ("/dev/log"); internal(); udp(ip(0.0.0.0) port(514)); }; destination send_http_logs { file("/var/log/web.log"); }; filter send_http_logs { program("httpd.*"); }; log { source(s_sys); filter(send_http_logs); destination(send_http_logs); }; *Client.conf* options { sync (0); time_reopen (10); log_fifo_size (1000); long_hostnames (off); use_dns (no); use_fqdn (no); create_dirs (no); keep_hostname (yes); }; source s_sys { file ("/proc/kmsg" log_prefix("kernel: ")); unix-stream ("/dev/log"); internal(); # udp(ip(0.0.0.0) port(514)); }; destination send_http_logs { udp("192.168.2.54" port(514)); }; filter send_http_logs { program("httpd.*"); }; log { source(s_sys); filter(send_http_logs); destination(send_http_logs); }; I restart the services and look for that web.log file and never see anything, so any help is appreciated.

2 2