Message: 1
Date: Mon, 5 Dec 2011 19:54:42 +0800
From: Luis Pugoy <lpugoy@insynchq.com>
Subject: Re: [syslog-ng] syslog-ng 3.1.2 segmentation fault in Ubuntu
       10.10
To: syslog-ng@lists.balabit.hu
Message-ID:
       <CA+WGxEhKYMQXbYTVUZVScjaXDdjRETGSCk5nZaisW9p-UOQWoA@mail.gmail.com>
Content-Type: text/plain; charset="iso-8859-1"

>
> Message: 3
> Date: Mon, 05 Dec 2011 10:10:40 +0100
> From: Gergely Nagy <algernon@balabit.hu>
> Subject: Re: [syslog-ng] syslog-ng 3.1.2 segmentation fault in Ubuntu
>        10.10
> To: Syslog-ng users' and developers' mailing list
>        <syslog-ng@lists.balabit.hu>
> Message-ID: <87aa77l6y7.fsf@algernon.balabit>
> Content-Type: text/plain
>
> Luis Pugoy <lpugoy@insynchq.com> writes:
>
> > Does anyone have any insight into this? The segfault is happening more
> > often now. :(
>
> I'd suggest either upgrading to 3.3 from my repo[1], or if an upgrade is
> not feasible, then if you could compile 3.1 with debug symbols, get it
> to drop a core, and do a backtrace, that would help tremendously.
>
>  [1]: http://asylum.madhouse-project.org/projects/debian/
>
> To do the latter, the following steps should work:
>
> $ apt-get source syslog-ng
> $ sudo apt-get build-dep syslog-ng
> $ cd syslog-ng-*
> $ DEB_BUILD_OPTIONS="debug nostrip" dpkg-buildpackage -us -uc -rfakeroot
> $ sudo dpkg -i ../syslog-ng_*.deb
>
> Then add --enable-core to SYSLOGNG_OPTS in /etc/default/syslog-ng, and
> when it segfaults, it'll make a core in (as far as I remember) /. Once
> that happened, do something like this:
>
> $ gdb -c /core /usr/bin/syslog-ng
> (gdb) thread apply all bt full
>
> This should give us a few hints.
>
> --
> |8]
>

 I installed a newer version of the PCRE library. That seems to have worked
for now. If it happens again I will be sure to follow your instructions,
thanks.

Well, installing another version of the PCRE library did not solve my problem. syslog-ng is still segfaulting. I tried upgrading to 3.2.2 but that did nothing, so I tried to create the backtrace like you said. However I don't think it was helpful in my case. The following is the output of the backtrace:

root@ip-10-34-150-77:/var/lib/syslog-ng# gdb -c core /usr/sbin/syslog-ng
GNU gdb (GDB) 7.2-ubuntu
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /usr/sbin/syslog-ng...done.
[New Thread 14680]

warning: Can't read pathname for load map: Input/output error.
Reading symbols from /lib/librt.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib/librt.so.1
Reading symbols from /lib/libnsl.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib/libnsl.so.1
Reading symbols from /usr/lib/libgthread-2.0.so.0...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libgthread-2.0.so.0
Reading symbols from /lib/libglib-2.0.so.0...(no debugging symbols found)...done.
Loaded symbols for /lib/libglib-2.0.so.0
Reading symbols from /usr/lib/libevtlog.so.0...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libevtlog.so.0
Reading symbols from /lib/libssl.so.0.9.8...(no debugging symbols found)...done.
Loaded symbols for /lib/libssl.so.0.9.8
Reading symbols from /lib/libcrypto.so.0.9.8...(no debugging symbols found)...done.
Loaded symbols for /lib/libcrypto.so.0.9.8
Reading symbols from /lib/libz.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib/libz.so.1
Reading symbols from /usr/lib/libnet.so.1...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libnet.so.1
Reading symbols from /lib/libwrap.so.0...(no debugging symbols found)...done.
Loaded symbols for /lib/libwrap.so.0
Reading symbols from /usr/lib/libdbi.so.0...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libdbi.so.0
Reading symbols from /lib/libcap.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/libcap.so.2
Reading symbols from /lib/libpcre.so.3...(no debugging symbols found)...done.
Loaded symbols for /lib/libpcre.so.3
Reading symbols from /lib/libpthread.so.0...(no debugging symbols found)...done.
Loaded symbols for /lib/libpthread.so.0
Reading symbols from /lib/libc.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib/libc.so.6
Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/ld-linux-x86-64.so.2
Reading symbols from /lib/libdl.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/libdl.so.2
Reading symbols from /lib/libm.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib/libm.so.6
Reading symbols from /lib/libattr.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib/libattr.so.1
Reading symbols from /lib/libnss_compat.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/libnss_compat.so.2
Reading symbols from /lib/libnss_nis.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/libnss_nis.so.2
Reading symbols from /lib/libnss_files.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/libnss_files.so.2
Reading symbols from /lib/libnss_dns.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/libnss_dns.so.2
Reading symbols from /lib/libresolv.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/libresolv.so.2
Core was generated by `/usr/sbin/syslog-ng -p /var/run/syslog-ng.pid --no-caps --enable-core'.
Program terminated with signal 11, Segmentation fault.
#0  0x00007fd8440c4e9d in ?? () from /lib/libpcre.so.3
(gdb) thread apply all bt full

Thread 1 (Thread 14680):
#0  0x00007fd8440c4e9d in ?? () from /lib/libpcre.so.3
No symbol table info available.
Cannot access memory at address 0x7fff4a4e3fd8
(gdb) 



If it is in anyway helpful, I think the only usage of PCRE in my syslog-ng.conf is a rewrite rule to change the Unicode character separator line to a new line: subst("\\p{Zl}", "\n", value("MESSAGE") type("pcre") flags("utf8" "global")); Is it related to my problem in some way?