March 2012 - syslog-ng - lists.balabit.hu

[Bug 157] New: syslog-ng 3.2.x/3.3.x: test_sql.py and 64-bit systems
by bugzilla＠bugzilla.balabit.com 01 Jun '12

01 Jun '12

https://bugzilla.balabit.com/show_bug.cgi?id=157 Summary: syslog-ng 3.2.x/3.3.x: test_sql.py and 64-bit systems Product: syslog-ng Version: 3.3.x Platform: PC OS/Version: Mac OS Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: jpo(a)di.uminho.pt Type of the Report: --- Estimated Hours: 0.0 Created an attachment (id=45) --> (https://bugzilla.balabit.com/attachment.cgi?id=45) Add '/usr/lib64/dbd/' to the search paths Hi, In 64-bits systems we need to add lib64 directories to the libdbi/libdbdsqlite3 module search path in order to successfully run the test_sql functional test. Location of libdbdsqlite3.so in 64-bit Fedora/RHEL: ---------- $ rpm -ql libdbi-dbd-sqlite /usr/lib64/dbd/libdbdsqlite3.so ---------- Affected syslog-ng versions: At least the 3.2 and 3.3 branches (I haven't check the 3.4 development branch). Fix: The attached patch adds the '/usr/lib64/dbd/' directory to the search path. Adding '/usr/local/lib64/dbd' should also make sense (but it is not done by the attached patch). /jpo -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 5

[Bug 143] New: syslog-ng supervisor dies on SIGHUP
by bugzilla＠bugzilla.balabit.com 31 May '12

31 May '12

https://bugzilla.balabit.com/show_bug.cgi?id=143 Summary: syslog-ng supervisor dies on SIGHUP Product: syslog-ng Version: 3.3.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: bugzilla.syslogng(a)feystorm.net Type of the Report: --- Estimated Hours: 0.0 The process which supervises syslog-ng and restarts it in the event of a crash dies when it receives a SIGHUP. This is a problem as when sending a HUP to syslog-ng to have it reload its config, the HUP will get sent to both the supervisor process and the worker process if doing something like `pkill -HUP syslog-ng`. I attached a gdb to the supervisor and its not a segfault or anything, it just quits. I'm guessing the supervisor doesnt have a signal handler for HUP, so it uses the default action which is to quit. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 5

Problem with Syslog-NG 3.2.5 on Aix 7.1 ... It coredumps..
by Jonathan Kaufman 16 May '12

16 May '12

Hello All, I am hoping someone out there has a working installation of 3.2.5 on Aix 7.1 and can help me get mine working. I am currently trying to use a self-compiled version of 3.2.5 on an Power7 CPU Aix 7.1 TL1 SP3 box. I was able to get Syslog-NG to compile on Aix 7.1 after I mangled a new libtool release into the source so it would compile shared objects. Unfortunately, when I start syslog-ng it core dumps repeatedly. Core dumps start filling up /sftw/syslog-ng/var and a ps -ef shows only the "supervising syslog-ng process". a dbx syslog-ng gives this output... [root@mlwitt71]:[/sftw/syslog-ng/sbin] > dbx syslog-ng Type 'help' for help. [using memory image in core.10092546.01150742] reading symbolic information ... Segmentation fault in alloca at 0xd2970930 ($t1) 0xd2970930 (alloca+0x8) 800c0000 lwz r0,0x0(r12) (dbx) where alloca() at 0xd2970930 evt_str_append_escape_bs() at 0xd2970520 evtrec_format_plain() at 0xd296feb4 evt_format() at 0xd2970104 msg_event_send(e = 0x00000005), line 166 in "messages.c" main_loop_run(cfg = (nil)), line 148 in "main.c" main(argc = 1, argv = 0x2ff227f4), line 470 in "main.c" (dbx) I have also tried using the Syslog-ng rpm's from perzl & Bull. Both of them core dump as well, so I went back to trying to get a self-compiled release to work as I have some control over that. I used IBM VisualAge C/C++ v11.1.0.9 as the C compiler. [root@mlwitt71]:[/sftw/syslog-ng/sbin] > lslpp -L | grep vac vac.Bnd 11.1.0.1 C F XL C for AIX Media Defined vac.C 11.1.0.9 C F IBM XL C Compiler vac.aix53.lib 11.1.0.9 C F XL C for AIX Libraries for AIX vac.include 11.1.0.9 C F IBM XL C Compiler Include vac.lib 11.1.0.9 C F XL C for AIX Libraries vacpp.Bnd 11.1.0.1 C F IBM XL C/C++ Media Defined vacpp.cmp.aix53.lib 11.1.0.9 C F IBM XL C/C++ Libraries for AIX vacpp.cmp.core 11.1.0.9 C F IBM XL C/C++ Compiler vacpp.cmp.include 11.1.0.9 C F IBM XL C/C++ Compiler Include vacpp.cmp.lib 11.1.0.9 C F IBM XL C/C++ Libraries vacpp.cmp.rte 11.1.0.9 C F IBM XL C/C++ Compiler vacpp.cmp.tools 11.1.0.9 C F IBM XL C/C++ Tools vacpp.tnb 11.1.0.1 C F IBM XL C/C++ Evaluation I mangled libtool 2.4.2 into the source directories so it would detect/compile shared libraries (it wouldn't otherwise). I compiled and installed eventlog 0.2.12 in /sftw/syslog-ng, I also compiled OpenSSL 1.0.0g into it staticly. I am initially trying to start it using the default configuration files. There were no errors during the compile, and a syslog-ng -s did NOT coredump. I believe syslog-ng is using the following libraries and their locations (dynamically linked). [root@mlwitt71]:[/sftw/syslog-ng/sbin] > ldd syslog-ng syslog-ng needs: /sftw/syslog-ng/lib/libsyslog-ng.a(libsyslog-ng.so.0) /usr/lib/libnsl.a(shr.o) /opt/freeware/lib/libgmodule-2.0.so /opt/freeware/lib/libglib-2.0.so /usr/lib/libpthread.a(shr_xpg5.o) /usr/lib/libc.a(shr.o) /sftw/syslog-ng/lib/libevtlog.a(libevtlog.so.0) /usr/lib/librtl.a(shr.o) /opt/freeware/lib/libpcre.a(libpcre.so.0) /usr/lib/libthread.a(shr.o) /usr/lib/libpthreads_compat.a(shr.o) /usr/lib/libpthreads.a(shr_xpg5.o) /usr/lib/libtli.a(shr.o) /opt/freeware/lib/libglib-2.0.a(libglib-2.0.so.0) /opt/freeware/lib/libintl.a(libintl.so.1) /usr/lib/libiconv.a(shr4.o) /usr/lib/libpthreads.a(shr_comm.o) /unix /usr/lib/libcrypt.a(shr.o) /usr/lib/libpthreads.a(shr.o) /usr/lib/libc.a(pse.o) and the configure command when compiling from source was: ./configure --prefix=/sftw/syslog-ng \ --disable-spoof-source \ --enable-dynamic-linking \ --enable-debug \ --enable-ssl and I added -g to the CFLAGS I am using glib2-2.28.6-1 from perzl as well.. I plan on removing the --enable-debug if I can get it working Any ideas? Jonathan Kaufman

8 10

[Bug 170] New: feature request: include only load *.conf files
by bugzilla＠bugzilla.balabit.com 07 May '12

07 May '12

https://bugzilla.balabit.com/show_bug.cgi?id=170 Summary: feature request: include only load *.conf files Product: syslog-ng Version: 3.3.x Platform: Other OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: erempel(a)uvic.ca Type of the Report: enhancement Estimated Hours: 0.0 When an rpm package is used to place a configuration file in an include directory and the package is later updated, a file of the name *.conf.rpmnew or *.conf.rpmsave but the contents of this file will be a conflict with the *.conf file and syslog-ng will not start. Syslog-ng should only load the *.conf files so that it can work seamlessly with the rpm method. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 6

[Bug 156] New: rewrite rule not working on Linux
by bugzilla＠bugzilla.balabit.com 20 Apr '12

20 Apr '12

https://bugzilla.balabit.com/show_bug.cgi?id=156 Summary: rewrite rule not working on Linux Product: syslog-ng Version: 3.3.x Platform: Other OS/Version: Linux Status: NEW Severity: major Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: ne24(a)georgetown.edu Type of the Report: bug Estimated Hours: 0.0 Hi, I have syslog-ng 3.3.3 running Red Hat Enterprise Linux Server release 6.2 (Santiago). I compiled with the following flags. ./configure --prefix=/usr/local --enable-spoof-source --enable-pcre --enable-ssl \ --enable-debug I have the following rewrite rule: rewrite n_dst_router { subst("^(%(?:ASA|PIX|FWSM)\-\d\-\d{6}):", "", value("MESSAGE"), type("pcre"), flags("global")); }; But it is not working. The line below minus the IP addresses shows that it is not replacing the %FWSM-6-305011 with a blank. Jan 7 15:59:06 HOSTNAME-REMOVED %FWSM-6-305011: Built dynamic tcp translation from GRT:IP-ADDRESS-REMOVED/53328 to BACKBONE-VRF:IP-ADDRESS-REMOVED/29361 I will also attach my whole syslog-ng.conf file so that you would see the whole setup. Here is the output of the debug output and as you can see, it looks like it is doing what it is supposed to, but it does not write it to the file with the change. Incoming log entry; line='<163>%FWSM-3-710003: udp access denied by ACL from IP-ADDRESS-REMOVED/35390 to BACKBONE-VRF:IP-ADDRESS-REMOVED/46897\x0a' Rewrite expression evaluation result; value='MESSAGE', new_value='udp access denied by ACL from IP-ADDRESS-REMOVED/35390 to BACKBONE-VRF:IP-ADDRESS-REMOVED/46897' I searched the bug reports but could not find anything related to this issue. I would appreciate it your help on this. Thank you Nadim El-Khoury -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 2

[Bug 163] New: afmongo does not send log
by bugzilla＠bugzilla.balabit.com 20 Apr '12

20 Apr '12

https://bugzilla.balabit.com/show_bug.cgi?id=163 Summary: afmongo does not send log Product: syslog-ng Version: 3.4.x Platform: PC OS/Version: Linux Status: NEW Severity: critical Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: whille(a)163.com Type of the Report: bug Estimated Hours: 0.0 i'm using syslog-ng to store mongo, about 2k logs/sec. About half an hour, no message sent out. trying to locate it, I set log_fifo_size(16), it's easy to reproduce the bug. i found codes related: afmongodb_dd_queue(LogPipe *s, LogMessage *msg, const LogPathOptions *path_options, gpointer user_data) { ... g_mutex_lock(self->queue_mutex); self->last_msg_stamp = cached_g_current_time_sec (); queue_was_empty = log_queue_get_length(self->queue) == 0; g_mutex_unlock(self->queue_mutex); log_queue_push_tail(self->queue, msg, path_options); g_mutex_lock(self->suspend_mutex); if (queue_was_empty && !self->writer_thread_suspended) { g_mutex_lock(self->queue_mutex); log_queue_set_parallel_push(self->queue, 1, afmongodb_dd_queue_notify, self, NULL); g_mutex_unlock(self->queue_mutex); } g_mutex_unlock(self->suspend_mutex); } since in afmongodb_worker_thread(){ ... g_mutex_unlock(self->suspend_mutex); g_mutex_lock(self->queue_mutex); if (log_queue_get_length(self->queue) == 0) { g_cond_wait(self->writer_thread_wakeup_cond, self->queue_mutex); } g_mutex_unlock(self->queue_mutex); ... } the whole rough process i think is, worker wait fo afmongodb_dd_queue_notify(), if 0 queue. But in multi-thread race condition, queue_was_empty offend simplicity. Here's a situation: afmongodb_dd_queue() afmongodb_dd_queue() afmongodb_dd_queue() { // worker thread afmongodb_worker_thread{ whille(){... // run in a while circle afmongodb_worker_thread{ whille(){... afmongodb_worker_thread{ whille(){... all queue are empty now, g_cond_wait(self->writer_thread_wakeup_cond, self->queue_mutex); log_queue_push_tail queue_was_empty is FALSE, so afmongodb_dd_queue_notify is not called. afmongodb_dd_queue() // a queue has been put to tail last time, so queue_was_empty will be still FALSE. i'm trying to get queue length later and judge it as <=1, instead of ==0, for a queue just put to tail. wonder if it work, though a bit ugly. afmongodb_dd_queue(){ ... if(!self->writer_thread_suspended){ g_mutex_lock(self->queue_mutex); if (log_queue_get_length(self->queue)<=1){ log_queue_set_parallel_push(self->queue, 1, afmongodb_dd_queue_notify, self, NULL); } g_mutex_unlock(self->queue_mutex); } -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 10

race condition in destination driver deinit?
by Michael Hocke 07 Apr '12

07 Apr '12

-----BEGIN PGP SIGNED MESSAGE----- Me again.... Hi guys, I've been running 3.3.3 on Solaris 10 x86 for quite a bit now. I've got two boxes, both running the same OS release and the same release of syslog-ng. One of them (box A) has a destination configured that doesn't really exist so I've been getting Mar 13 14:46:15 flowmon-sys syslog-ng[20118]: I/O error occurred while writing; fd='24', error='Connection refused (146)' which is perfectly fine and I just ignored it. The other Solaris box (box B) does not have this destination configured. It now happens that after a random number of HUP signals syslog-ng on box A crashes with a segmentation fault and the following backtrace: (gdb) bt #0 0xfef2a7a8 in log_dest_driver_release_queue_method (self=0x80d2958, q=0x83e58955, user_data=0x0) at driver.c:80 #1 0xfef2aa5c in log_dest_driver_deinit_method (s=0x80d2958) at driver.c:80 #2 0xfe914a14 in afsocket_dd_deinit (s=0x80d2958) at afsocket.c:109 #3 0xfef29cd6 in log_dest_group_deinit (s=0x8074198) at dgroup.c:59 #4 0xfef24a21 in log_center_deinit (self=0x8090cc0) at center.c:67 #5 0xfef25493 in cfg_deinit (cfg=0x80d2848) at cfg.c:90 #6 0xfef406b2 in main_loop_reload_config_apply () at mainloop.c:364 #7 0xfef40a42 in main_loop_io_worker_sync_call (func=<value optimized out>) at mainloop.c:364 #8 0x08047900 in ?? () I was able to reproduce this over and over again. Sometimes it happens after 11 HUPs, sometimes after 35, but it eventually does crash. Since the only difference between box A and box B is the one additional destination I suspect that this is the cause for this segfault. Maybe the HUP signal came at a time when syslog-ng was trying to send something to the non-existing destination? I'll try to collect some more data. If anybody could give me some direction on where exactly I should look into, I'll be happy to do that. - - Michael -----BEGIN PGP SIGNATURE----- Version: PGP Desktop 10.0.3 (Build 1) Charset: us-ascii wsBVAwUBT1+ZRZbfnpCg64TVAQEiVgf7BmePJa6Va396QRyBLNPpOHzAvi9p3n07 uVvvYOcMHUFwixZoC0BXs+21EfSkMOmROQWupXBTcEbyBMOf0+HXDsuO7mfZDznU AZzLBzhgaDPf0stx55PqOssKTf28QLtxs1gUmJSb2DgmK9WjtMsovAlS83pQOa6O oQIaK3SBIUtrbijdx7vLvS0VUcqhDzJI9TJ7R+A0x3i9dAWKIvDb7wLW4CvB9k19 7bmmXFwpAClGZ6DS1CdIt4eHuDiuIoes2uXNhh5xReZpCh89FHzv8LxWxVnph3ms 6FctjAt4JMQTGnZRHN7QidYAGwjjabqtGjKWvXtRfoBCdUfLfC+wAA== =JjOa -----END PGP SIGNATURE-----

2 1

[PATCH] afsocket: Do not die if a destination is not resolvable at startup.
by Gergely Nagy 06 Apr '12

06 Apr '12

If a destination cannot be resolved at init time, do not panic, just bail out, but otherwise pretend we succeeded: after time_reopen, we'll retry anyway! And on that note, teach afsocket_dd_start_connect() to call afsocket_dd_apply_transport() if our fd is -1 (ie, not connected). Reported-by: Peter Palfrader <weasel(a)debian.org> Signed-off-by: Gergely Nagy <algernon(a)balabit.hu> --- modules/afsocket/afinet.c | 2 +- modules/afsocket/afsocket.c | 8 +++++--- 2 files changed, 6 insertions(+), 4 deletions(-) diff --git a/modules/afsocket/afinet.c b/modules/afsocket/afinet.c index f83ff38..a7d3635 100644 --- a/modules/afsocket/afinet.c +++ b/modules/afsocket/afinet.c @@ -498,7 +498,7 @@ afinet_dd_apply_transport(AFSocketDestDriver *s) if ((self->bind_ip && !resolve_hostname(&self->super.bind_addr, self->bind_ip)) || !resolve_hostname(&self->super.dest_addr, self->super.hostname)) - return FALSE; + return TRUE; afinet_set_port(self->super.dest_addr, self->dest_port ? : default_dest_port, self->super.flags & AFSOCKET_DGRAM ? "udp" : "tcp"); diff --git a/modules/afsocket/afsocket.c b/modules/afsocket/afsocket.c index ae9c5c2..095f5a5 100644 --- a/modules/afsocket/afsocket.c +++ b/modules/afsocket/afsocket.c @@ -1103,6 +1103,11 @@ afsocket_dd_start_connect(AFSocketDestDriver *self) gchar buf1[MAX_SOCKADDR_STRING], buf2[MAX_SOCKADDR_STRING]; main_loop_assert_main_thread(); + + if (self->fd == -1) + if (!afsocket_dd_apply_transport(self)) + return FALSE; + if (!afsocket_open_socket(self->bind_addr, !!(self->flags & AFSOCKET_STREAM), &sock)) { return FALSE; @@ -1169,10 +1174,7 @@ afsocket_dd_init(LogPipe *s) /* these fields must be set up by apply_transport, so let's check if it indeed did */ g_assert(self->transport); - g_assert(self->bind_addr); - g_assert(self->dest_addr); g_assert(self->hostname); - g_assert(self->dest_name); if (cfg) { -- 1.7.9

2 3

[Bug 171] New: Unwanted IP adress Folder
by bugzilla＠bugzilla.balabit.com 04 Apr '12

04 Apr '12

https://bugzilla.balabit.com/show_bug.cgi?id=171 Summary: Unwanted IP adress Folder Product: syslog-ng Version: 3.1.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: julien.groselle(a)gmail.com Type of the Report: bug Estimated Hours: 0.0 Hi, I work for a lange society, and we use syslog-ng for 5 years now. We have a centralized server with storage tank to keep logs securly. Concretely, we have 2 centralized servers syslog-ng in chrooted environement, and 50 clients servers. Since we use TLS transport in place of stunnel workaround, we have many issues : - First of all, many logs aren't writen in $HOST folder but in IPADDRESS folder. So, to be clear, this is an exemple : # ls drwxr-x--- 8 root adm 4,0K 1 mars 00:07 10.0.0.1 drwxr-x--- 53 root adm 4,0K 19 mars 00:35 host1 I assume that host1 have 10.0.0.1 IP address and # tree 192.168.100.79/2012-03/ 10.0.0.1/2012-03/ ├── 02-user-10.0.0.1.log.bz2 ├── 06-user-10.0.0.1.log.bz2 ├── 07-user-10.0.0.1.log.bz2 ├── 08-user-10.0.0.1.log.bz2 ├── 09-user-10.0.0.1.log.bz2 ├── 12-user-10.0.0.1.log.bz2 ├── 13-user-10.0.0.1.log.bz2 ├── 14-user-10.0.0.1.log.bz2 ├── 15-user-10.0.0.1.log.bz2 ├── 16-user-10.0.0.1.log.bz2 └── 19-user-10.0.0.1.log # tree host1/2012-03/ |grep 19- ├── 19-apache.access-host1.log ├── 19-apache.error-host1.log ├── 19-authpriv-host1.log ├── 19-auth-host1.log ├── 19-cron-host1.log ├── 19-daemon-host1.log ├── 19-kern-host1.log ├── 19-mail-host1.log ├── 19-nagios-host1.log ├── 19-puppetd-host1.log ├── 19-syslog-host1.log └── 19-user-host1.log (we have this problem with many servers) In facility "user" for host 10.0.0.1 in fact i have log for snmptrapd... But why ?? We have config for snmpd but not for snmptrapd... So i have tried to define a default facility => failed After i have tried many dns and hostnames options => failed As anyone here have a way to search for me ? If you need more details, i'm your's. Kind regards. -- JG -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 6

Ongoing 3.3.4 memory leaks
by Marvin Nipper 02 Apr '12

02 Apr '12

Hi. Sorry, looking for some help. Still trying to chase some memory leaks that I see on Solaris 10. I'll crank up syslog-ng, and it'll start running at around 7 or 8MB, and interestingly, it will run that way sometimes for several hours, and then (for reasons unknown), I'll take a look at it a bit later, and suddenly it will have jumped to 26MB (or later on, maybe 75MB). I've already applied the (older) logwriter.c fix and the (new) Gergely driver.h/afsocket.c fix, in an attempt to resolve known issues. Certainly the logwriter fix corrected a super-nasty, continuous leak problem, but this other leak problem still lingers. MY PROBLEM is that I'm wanting to use the debug symbols in syslog-ng, so that I can have the Solaris tools specifically show the exact location where these leaks are happening, BUT, whereas a non-debugging version of syslog-ng executes properly, when I configure syslog-ng with -enable-debug, the resulting executable will no longer start. It immediately exits, coughing up what amounts to a syntax error (even though I'm using the exact same command-line invocation that I use on the non-debugged executable): #/etc/init.d/syslog-ng start syslog-ng service starting. mkfifo: File exists mkfifo: File exists Usage: /usr/local/sbin/syslog-ng { start | stop } # Something about enabling debugging is impacting the operational behavior of the resulting binary. Anyway, I can use (for Solaris) libumem and mdb to narrow this down to a particular module, but without the debug symbols, I can't really get to a specific chunk of code, so it would obviously help to get that working. If anyone has any ideas, I certainly would appreciate the input. THANKS Marvin Nipper This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.

3 3