January 2012 - syslog-ng - lists.balabit.hu

patterndb that kills syslong-ng with a segfault.
by Russell Fulton 16 Jan '12

16 Jan '12

I tried to rewrite the program macro with a <value> clause and this caused syslog-ng to die. syslog-ng-3.2.4 I no longer have the example but can reconstruct it if you have problems reproducing the problem. R

2 1

[PATCH (3.4) 0/5] afmongodb cleanups & replica set support
by Gergely Nagy 16 Jan '12

16 Jan '12

Following this email, a few patches will come, various minor touch-ups (such as the cleanup of the default value-pairs config, and the removal of the TODO file) and replica-set support for afmongodb. The replica-set support is an incompatible change compared to 3.3, as it removes the host() and port() options from the driver, replacing both with a servers() option. After the patches, the driver will support replica sets, which means that when connecting to a replica set, and the primary goes down, the driver will reconnect to the new master automatically. Most of this is handled transparently by the underlying libmongo-client library, so the syslog-ng side doesn't actually get much more complicated. These patches have been sitting in my syslog-ng module collection repo since about july, and I kinda forgot about them. It's about time I submit them. (And consequently, the module-collection repo will be abandoned, as it pretty much has been for the past few months anyway. Turns out it's easier for me to maintain module updates within my syslog-ng repo, especially now that I have 3.3 and 3.4 within the same git repo.) All of these are available from the feature/3.4/afmongodb/lmc-sync branch of my git repository.

2 7

Syslog-ng 3.2 connection timeout with firewall
by André Larose 16 Jan '12

16 Jan '12

Hi, I have two syslog-ng 3.2, one client and one server. The two are separated by a firewall and a load balancer. I noticed that after some inactivity I was not able to receive logs from my client. So I started some tcpdump on both servers to check the traffic. From what I see the firewall will close the connection after some time, so when the client sends traffic it gets dropped. I added the keep-alive(yes) and so_keep-alive(yes) and the mark_freq(60) to the configs. But I still do not see keepalive packets with tcpdump. Am I missing other parameters to have "keepalive" traffic sent ? Thank you in advance. ############################################################################# # # Configuration file syslog-ng.conf.client # @version: 3.2 @include "scl.conf" # add global settings options { # Disable DNS usage to gain the process time use_dns(no); flush_lines(1); flush_timeout(500); mark(59); }; source s_TCP { tcp( port(555) flags(no-parse) ); }; source s_syslogng { internal(); }; destination d_syslogng { file("/opt/syslog-ng/logs/$YEAR$MONTH$DAY.syslog-ng.log"); }; destination d_TCP { tcp ( "server1" port(556) keep-alive(yes) so_keepalive(yes) flags(no_multi_line) flush_lines(1) flush_timeout(500) log_fifo_size(4096) template("$MSG\n") template_escape(no) ); }; .... ############################################################################# # # Configuration file syslog-ng.conf.server # @version: 3.2 @include "scl.conf" # add global settings options { # Disable DNS usage to gain the process time use_dns(no); flush_lines(1); flush_timeout(500); mark(59); }; source s_TCP_555 { tcp( port(555) flags(no-parse) max-connections(200) use_dns(no) ); }; source s_TCP_556 { tcp( port(556) flags(no-parse) so_keepalive(yes) use_dns(no) ); }; source s_syslogng { internal(); }; destination d_syslogng { file("/opt/syslog-ng/logs/$YEAR$MONTH$DAY.syslog-ng.log"); }; destination d_local { file("/opt2/syslog-ng/logs/$YEAR$MONTH$DAY.TNT.Messages.log" log_fifo_size(8192) template("$MSG\n") template_escape(no) ); }; ...

2 1

[PATCH (3.4)] center: Fix a few syntax errors.
by Gergely Nagy 16 Jan '12

16 Jan '12

Fix two syntax errors (an unclosed comment, and a double-closing-bracer) that came in with a previous patch. Signed-off-by: Gergely Nagy <algernon(a)balabit.hu> --- lib/center.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/lib/center.c b/lib/center.c index 986ab75..2fc3554 100644 --- a/lib/center.c +++ b/lib/center.c @@ -320,6 +320,7 @@ log_center_instantiate_process_pipe_line(LogCenter *self, LogProcessRule *rule) * * The next member pointer is not holding a reference, but can be * assumed to be kept alive as long as the configuration is running. + */ LogPipe * log_center_init_pipe_line(LogCenter *self, LogConnection *conn, GlobalConfig *cfg, gboolean toplevel, gboolean flow_controlled_parent) { @@ -468,7 +469,6 @@ log_center_init_pipe_line(LogCenter *self, LogConnection *conn, GlobalConfig *cf log_multiplexer_add_next_hop(fork_mpx, sub_pipe); break; } - } default: { g_assert_not_reached(); -- 1.7.7.3

2 1

Re: [syslog-ng] syslog-ng 3.1.2 segmentation fault in Ubuntu 10.10 (Luis Pugoy)
by Luis Pugoy 16 Jan '12

16 Jan '12

> > > Message: 1 > Date: Mon, 5 Dec 2011 19:54:42 +0800 > From: Luis Pugoy <lpugoy(a)insynchq.com> > Subject: Re: [syslog-ng] syslog-ng 3.1.2 segmentation fault in Ubuntu > 10.10 > To: syslog-ng(a)lists.balabit.hu > Message-ID: > <CA+WGxEhKYMQXbYTVUZVScjaXDdjRETGSCk5nZaisW9p-UOQWoA(a)mail.gmail.com > > > Content-Type: text/plain; charset="iso-8859-1" > > > > > Message: 3 > > Date: Mon, 05 Dec 2011 10:10:40 +0100 > > From: Gergely Nagy <algernon(a)balabit.hu> > > Subject: Re: [syslog-ng] syslog-ng 3.1.2 segmentation fault in Ubuntu > > 10.10 > > To: Syslog-ng users' and developers' mailing list > > <syslog-ng(a)lists.balabit.hu> > > Message-ID: <87aa77l6y7.fsf(a)algernon.balabit> > > Content-Type: text/plain > > > > Luis Pugoy <lpugoy(a)insynchq.com> writes: > > > > > Does anyone have any insight into this? The segfault is happening more > > > often now. :( > > > > I'd suggest either upgrading to 3.3 from my repo[1], or if an upgrade is > > not feasible, then if you could compile 3.1 with debug symbols, get it > > to drop a core, and do a backtrace, that would help tremendously. > > > > [1]: http://asylum.madhouse-project.org/projects/debian/ > > > > To do the latter, the following steps should work: > > > > $ apt-get source syslog-ng > > $ sudo apt-get build-dep syslog-ng > > $ cd syslog-ng-* > > $ DEB_BUILD_OPTIONS="debug nostrip" dpkg-buildpackage -us -uc -rfakeroot > > $ sudo dpkg -i ../syslog-ng_*.deb > > > > Then add --enable-core to SYSLOGNG_OPTS in /etc/default/syslog-ng, and > > when it segfaults, it'll make a core in (as far as I remember) /. Once > > that happened, do something like this: > > > > $ gdb -c /core /usr/bin/syslog-ng > > (gdb) thread apply all bt full > > > > This should give us a few hints. > > > > -- > > |8] > > > > I installed a newer version of the PCRE library. That seems to have worked > for now. If it happens again I will be sure to follow your instructions, > thanks. > Well, installing another version of the PCRE library did not solve my problem. syslog-ng is still segfaulting. I tried upgrading to 3.2.2 but that did nothing, so I tried to create the backtrace like you said. However I don't think it was helpful in my case. The following is the output of the backtrace: root@ip-10-34-150-77:/var/lib/syslog-ng# gdb -c core /usr/sbin/syslog-ng GNU gdb (GDB) 7.2-ubuntu Copyright (C) 2010 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html > This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>... Reading symbols from /usr/sbin/syslog-ng...done. [New Thread 14680] warning: Can't read pathname for load map: Input/output error. Reading symbols from /lib/librt.so.1...(no debugging symbols found)...done. Loaded symbols for /lib/librt.so.1 Reading symbols from /lib/libnsl.so.1...(no debugging symbols found)...done. Loaded symbols for /lib/libnsl.so.1 Reading symbols from /usr/lib/libgthread-2.0.so.0...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libgthread-2.0.so.0 Reading symbols from /lib/libglib-2.0.so.0...(no debugging symbols found)...done. Loaded symbols for /lib/libglib-2.0.so.0 Reading symbols from /usr/lib/libevtlog.so.0...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libevtlog.so.0 Reading symbols from /lib/libssl.so.0.9.8...(no debugging symbols found)...done. Loaded symbols for /lib/libssl.so.0.9.8 Reading symbols from /lib/libcrypto.so.0.9.8...(no debugging symbols found)...done. Loaded symbols for /lib/libcrypto.so.0.9.8 Reading symbols from /lib/libz.so.1...(no debugging symbols found)...done. Loaded symbols for /lib/libz.so.1 Reading symbols from /usr/lib/libnet.so.1...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libnet.so.1 Reading symbols from /lib/libwrap.so.0...(no debugging symbols found)...done. Loaded symbols for /lib/libwrap.so.0 Reading symbols from /usr/lib/libdbi.so.0...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libdbi.so.0 Reading symbols from /lib/libcap.so.2...(no debugging symbols found)...done. Loaded symbols for /lib/libcap.so.2 Reading symbols from /lib/libpcre.so.3...(no debugging symbols found)...done. Loaded symbols for /lib/libpcre.so.3 Reading symbols from /lib/libpthread.so.0...(no debugging symbols found)...done. Loaded symbols for /lib/libpthread.so.0 Reading symbols from /lib/libc.so.6...(no debugging symbols found)...done. Loaded symbols for /lib/libc.so.6 Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/ld-linux-x86-64.so.2 Reading symbols from /lib/libdl.so.2...(no debugging symbols found)...done. Loaded symbols for /lib/libdl.so.2 Reading symbols from /lib/libm.so.6...(no debugging symbols found)...done. Loaded symbols for /lib/libm.so.6 Reading symbols from /lib/libattr.so.1...(no debugging symbols found)...done. Loaded symbols for /lib/libattr.so.1 Reading symbols from /lib/libnss_compat.so.2...(no debugging symbols found)...done. Loaded symbols for /lib/libnss_compat.so.2 Reading symbols from /lib/libnss_nis.so.2...(no debugging symbols found)...done. Loaded symbols for /lib/libnss_nis.so.2 Reading symbols from /lib/libnss_files.so.2...(no debugging symbols found)...done. Loaded symbols for /lib/libnss_files.so.2 Reading symbols from /lib/libnss_dns.so.2...(no debugging symbols found)...done. Loaded symbols for /lib/libnss_dns.so.2 Reading symbols from /lib/libresolv.so.2...(no debugging symbols found)...done. Loaded symbols for /lib/libresolv.so.2 Core was generated by `/usr/sbin/syslog-ng -p /var/run/syslog-ng.pid --no-caps --enable-core'. Program terminated with signal 11, Segmentation fault. #0 0x00007fd8440c4e9d in ?? () from /lib/libpcre.so.3 (gdb) thread apply all bt full Thread 1 (Thread 14680): #0 0x00007fd8440c4e9d in ?? () from /lib/libpcre.so.3 No symbol table info available. Cannot access memory at address 0x7fff4a4e3fd8 (gdb) If it is in anyway helpful, I think the only usage of PCRE in my syslog-ng.conf is a rewrite rule to change the Unicode character separator line to a new line: subst("\\p{Zl}", "\n", value("MESSAGE") type("pcre") flags("utf8" "global")); Is it related to my problem in some way?

2 1

[PATCH] afmongodb: Only free the value pairs if there's any.
by Gergely Nagy 16 Jan '12

16 Jan '12

In afmongodb_dd_free() only free self->vp, if it is non-NULL. In case of a syntax error in the configuration file, it may very well be NULL still, and that would lead to us segfaulting shortly after printing the error message. Signed-off-by: Gergely Nagy <algernon(a)balabit.hu> --- modules/afmongodb/afmongodb.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/modules/afmongodb/afmongodb.c b/modules/afmongodb/afmongodb.c index cbd0d16..7009b33 100644 --- a/modules/afmongodb/afmongodb.c +++ b/modules/afmongodb/afmongodb.c @@ -495,7 +495,8 @@ afmongodb_dd_free(LogPipe *d) g_free(self->user); g_free(self->password); g_free(self->host); - value_pairs_free(self->vp); + if (self->vp) + value_pairs_free(self->vp); log_dest_driver_free(d); } -- 1.7.7.3

2 1

[PATCH] afmongodb: Fix a race condition in the queue handling.
by Gergely Nagy 16 Jan '12

16 Jan '12

There is a race condition between afmongodb_dd_queue() and afmongodb_worker_insert(), that can trigger an assertion when the rate of incoming messages is very high. The scenario that triggers this is like this: * We receive a message, put it in our internal queue, and notify the worker thread. * The worker thread resets the queue's parallel_push. * We receive another message, notice that the queue is empty, so we insert it, and set parallel_push at the same time. * The worker thread tries to pop the queue's head, but that triggers an assert, because parallel_push is set. The root cause of the problem is that afmongodb_dd_queue() modifies the queue without locking the queue_mutex. This patch corrects that problem. Reported-By: Costa Farber <costaf(a)wix.com> Signed-off-by: Gergely Nagy <algernon(a)balabit.hu> --- modules/afmongodb/afmongodb.c | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/modules/afmongodb/afmongodb.c b/modules/afmongodb/afmongodb.c index cbd0d16..a88aeaa 100644 --- a/modules/afmongodb/afmongodb.c +++ b/modules/afmongodb/afmongodb.c @@ -530,7 +530,9 @@ afmongodb_dd_queue(LogPipe *s, LogMessage *msg, const LogPathOptions *path_optio g_mutex_lock(self->suspend_mutex); if (queue_was_empty && !self->writer_thread_suspended) { + g_mutex_lock(self->queue_mutex); log_queue_set_parallel_push(self->queue, 1, afmongodb_dd_queue_notify, self, NULL); + g_mutex_unlock(self->queue_mutex); } g_mutex_unlock(self->suspend_mutex); } -- 1.7.7.3

2 1

limit of "log_fifo_size"
by 外川　康徳 16 Jan '12

16 Jan '12

Hello folks, Is there limit of "log_fifo_size" on syslog-ng output message quing? I do know that real limit size of handling log messages depends on network environment or machine spec, but I'd like to know it has limit figure on its specification. Regards, Yasunori Togawa

2 1

Antw: Re: v3.3 not flushing lines?
by Peter Eisenlohr 14 Jan '12

14 Jan '12

Hi, just tried to reproduce with a simple perl script: open FILE, ">/home/peter/test.log" or die $!; { my $ofh = select FILE; $| = 1; select $ofh; } while (<>) { chomp; print FILE "incoming: $_\n"; }; Now, when sending log input to syslog-ng via tcp(), each line I type immediatelly shows up in the output log file. => can not reproduce. That is, with version 3.3.1. Peter Dipl.-Inf. Peter Eisenlohr Geschäftsbereich Logistik INFORM GmbH, Pascalstr. 23, 52076 Aachen, Germany Telefon: (+49) 2408 9456-6000 E-Mail: peter.eisenlohr(a)inform-software.com, Web: http://www.inform-software.com INFORM Institut fur Operations Research und Management GmbH Registered AmtsG Aachen HRB1144 Gfhr. Adrian Weiler >>> Clayton Dukes destination d_logzilla { 13.01.12 16.03 Uhr >>> program("/var/www/logzilla/scripts/db_insert.pl" template("$R_YEAR-$R_MONTH-$R_DAY $R_HOUR:$R_MIN:$S_SEC\t$HOST\t$PRI\t$PROGRAM\t$MSGONLY\n") template_escape(yes) );

2 1

v3.3 not flushing lines?
by Clayton Dukes 13 Jan '12

13 Jan '12

Hey folks, I have a user experiencing an issue where some of the incoming logs are not getting piped to my perl script until the second time we generate events. I believe this is generally set using flush_lines(1), but it doesn’t seem to be honoring that setting in the config. I also tried adding flush_timeout(900), but that has no affect. In the output below, if I quit the telnet and start it again, the previously missing events are then received (but the current ones are missing) - basically, it doesn't appear to be flushing every single line. Possible other reason: has something changed between v2.x and 3.x where the program() destination would possibly not send an EOF signal - i.e. is the pipe now kept open vs. an individual call to the program each time in v2.x? To verify that this is happening: Term 1: tail -f /tmp/logzilla_import.txt Term 2: /usr/local/sbin/syslog-ng -f /etc/syslog-ng/syslog-ng.conf -Fdv Term 3: telnet 192.168.254.1 Term 1 results: 192.168.254.1 22 7 3732620769 .Jan 11 2012 19:29:02.284 EST: Telnet2: 1 1 251 1 3751981041 0 2012-01-11 19:29:02 2012-01-11 19:29:02 192.168.254.1 22 7 3732620769 .Jan 11 2012 19:29:02.284 EST: TCP2: Telnet sent WILL ECHO (1) 3751981041 0 2012-01-11 19:29:02 2012-01-11 19:29:02 Term 2 results: Incoming log entry; line='<183>6987: .Jan 11 2012 19:29:02.284 EST: TCP2: Telnet sent WILL ECHO (1)' Incoming log entry; line='<183>6988: .Jan 11 2012 19:29:02.284 EST: Telnet2: 2 2 251 3' Incoming log entry; line='<183>6989: .Jan 11 2012 19:29:02.284 EST: TCP2: Telnet sent WILL SUPPRESS-GA (3)' Incoming log entry; line='<183>6990: .Jan 11 2012 19:29:02.284 EST: Telnet2: 80000 80000 253 24' Incoming log entry; line='<183>6991: .Jan 11 2012 19:29:02.284 EST: TCP2: Telnet sent DO TTY-TYPE (24)' Incoming log entry; line='<183>6992: .Jan 11 2012 19:29:02.284 EST: Telnet2: 10000000 10000000 253 31' Incoming log entry; line='<183>6993: .Jan 11 2012 19:29:02.284 EST: TCP2: Telnet sent DO WINDOW-SIZE (31)' Incoming log entry; line='<183>6994: .Jan 11 2012 19:29:02.284 EST: TCP2: Telnet received DO ENCRYPTION (38)' Incoming log entry; line='<183>6995: .Jan 11 2012 19:29:02.284 EST: TCP2: Telnet sent WONT ENCRYPTION (38) (unimplemented)' Incoming log entry; line='<183>6996: .Jan 11 2012 19:29:02.292 EST: TCP2: Telnet received WILL ENCRYPTION (38)' Incoming log entry; line='<183>6997: .Jan 11 2012 19:29:02.292 EST: TCP2: Telnet sent DONT ENCRYPTION (38) (unimplemented)' Incoming log entry; line='<183>6998: .Jan 11 2012 19:29:02.292 EST: TCP2: Telnet received DO SUPPRESS-GA (3)' Incoming log entry; line='<183>6999: .Jan 11 2012 19:29:02.292 EST: TCP2: Telnet received WILL TTY-TYPE (24)' Incoming log entry; line='<183>7000: .Jan 11 2012 19:29:02.292 EST: Telnet2: Sent SB 24 1 ' Incoming log entry; line='<183>7001: .Jan 11 2012 19:29:02.292 EST: TCP2: Telnet received WILL WINDOW-SIZE (31)' Incoming log entry; line='<183>7002: .Jan 11 2012 19:29:02.292 EST: TCP2: Telnet received WILL TTY-SPEED (32) (refused)' Incoming log entry; line='<183>7003: .Jan 11 2012 19:29:02.292 EST: TCP2: Telnet sent DONT TTY-SPEED (32)' Incoming log entry; line='<183>7004: .Jan 11 2012 19:29:02.292 EST: TCP2: Telnet received WILL LOCAL-FLOW (33) (refused)' Incoming log entry; line='<183>7005: .Jan 11 2012 19:29:02.292 EST: TCP2: Telnet sent DONT LOCAL-FLOW (33)' Incoming log entry; line='<183>7006: .Jan 11 2012 19:29:02.292 EST: TCP2: Telnet received WILL LINEMODE (34)' Incoming log entry; line='<183>7007: .Jan 11 2012 19:29:02.292 EST: TCP2: Telnet sent DONT LINEMODE (34) (unimplemented)' Incoming log entry; line='<183>7008: .Jan 11 2012 19:29:02.292 EST: TCP2: Telnet received WILL NEW-ENVIRON (39)' Incoming log entry; line='<183>7009: .Jan 11 2012 19:29:02.292 EST: TCP2: Telnet sent DONT NEW-ENVIRON (39) (unimplemented)' Incoming log entry; line='<183>7010: .Jan 11 2012 19:29:02.292 EST: TCP2: Telnet received DO STATUS (5)' Incoming log entry; line='<183>7011: .Jan 11 2012 19:29:02.292 EST: TCP2: Telnet sent WONT STATUS (5) (unimplemented)' Incoming log entry; line='<183>7012: .Jan 11 2012 19:29:02.292 EST: TCP2: Telnet received DO ECHO (1)' Incoming log entry; line='<183>7013: .Jan 11 2012 19:29:02.292 EST: Telnet2: recv SB NAWS 132 63' Incoming log entry; line='<183>7014: .Jan 11 2012 19:29:02.292 EST: Telnet2: recv SB 24 0 LINUX' Incoming log entry; line='<183>7015: .Jan 11 2012 19:29:02.493 EST: TCP2: Telnet received WILL ENVIRONMENT (36) (refused)' Incoming log entry; line='<183>7016: .Jan 11 2012 19:29:02.493 EST: TCP2: Telnet sent DONT ENVIRONMENT (36)' ______________________________________________________________ Clayton Dukes ______________________________________________________________

3 5