<table cellspacing="0" cellpadding="0" border="0" ><tr><td valign="top" style="font: inherit;"><p><font face="Arial" size="2">(gdb) bt full</font>
<br><font face="Arial" size="2">#0 0xfed46df0 in __lwp_park () from /lib/libc.so.1</font>
<br><font face="Arial" size="2">No symbol table info available.</font>
<br><font face="Arial" size="2">#1 0xfed40c44 in cond_sleep_queue () from /lib/libc.so.1</font>
<br><font face="Arial" size="2">No symbol table info available.</font>
<br><font face="Arial" size="2">#2 0xfed40e08 in cond_wait_queue () from /lib/libc.so.1</font>
<br><font face="Arial" size="2">No symbol table info available.</font>
<br><font face="Arial" size="2">#3 0xfed41350 in cond_wait () from /lib/libc.so.1</font>
<br><font face="Arial" size="2">No symbol table info available.</font>
<br><font face="Arial" size="2">#4 0xfed4138c in pthread_cond_wait () from /lib/libc.so.1</font>
<br><font face="Arial" size="2">No symbol table info available.</font>
<br><font face="Arial" size="2">#5 0xff119d80 in g_async_queue_pop_intern_unlocked (queue=0x757e0, try=0, end_time=0x75618) at gasyncqueue.c:359</font>
<br><font face="Arial" size="2"> retval = (gpointer) 0xa15b8</font>
<br><font face="Arial" size="2"> __PRETTY_FUNCTION__ = "g_async_queue_pop_intern_unlocked"</font>
<br><font face="Arial" size="2">#6 0xff119e80 in g_async_queue_pop (queue=0x757e0) at gasyncqueue.c:398</font>
<br><font face="Arial" size="2"> retval = (gpointer) 0x757e0</font>
<br><font face="Arial" size="2"> __PRETTY_FUNCTION__ = "g_async_queue_pop"</font>
<br><font face="Arial" size="2">#7 0x0003e984 in
afinter_source_dispatch (source=0x8d260, callback=0x3e9dc
<afinter_source_dispatch_msg>, user_data=0x8d1e0)</font></p>
<p><font face="Arial" size="2"> at afinter.c:112</font>
<br><font face="Arial" size="2"> msg = (LogMessage *) 0xa0dc0</font>
<br><font face="Arial" size="2"> path_options = {flow_control = -1, matched = 0x0}</font>
<br><font face="Arial" size="2"> tv = {tv_sec = 1257363112, tv_usec = 441817}</font>
<br><font face="Arial" size="2">#8 0xff143564 in g_main_context_dispatch (context=0x8d158) at gmain.c:2144</font>
<br><font face="Arial" size="2">No locals.</font>
<br><font face="Arial" size="2">#9 0xff1459a4 in g_main_context_iterate (context=0x8d158, block=1, dispatch=1, self=0x76030) at gmain.c:2778</font>
<br><font face="Arial" size="2"> max_priority = 2147483647</font>
<br><font face="Arial" size="2"> timeout = 4000</font>
<br><font face="Arial" size="2"> some_ready = 1</font>
<br><font face="Arial" size="2"> nfds = 4</font>
<br><font face="Arial" size="2"> allocated_nfds = 1</font>
<br><font face="Arial" size="2"> fds = (GPollFD *) 0x788c8</font>
<br><font face="Arial" size="2"> __PRETTY_FUNCTION__ = "g_main_context_iterate"</font>
<br><font face="Arial" size="2">#10 0xff146050 in g_main_context_iteration (context=0x8d158, may_block=1) at gmain.c:2841</font>
<br><font face="Arial" size="2"> retval = 1</font>
<br><font face="Arial" size="2">#11 0x0001bc20 in main_loop_run (cfg=0xffbffbc8) at main.c:149</font>
<br><font face="Arial" size="2"> iters = 0</font>
<br><font face="Arial" size="2"> stats_timer_id = 0</font>
<br><font face="Arial" size="2">#12 0x0001c260 in main (argc=1, argv=0xffbffd44) at main.c:394</font>
<br><font face="Arial" size="2"> cfg = (GlobalConfig *) 0x794d0</font>
<br><font face="Arial" size="2"> rc = 0</font>
<br><font face="Arial" size="2"> ctx = (GOptionContext *) 0x76030</font>
<br><font face="Arial" size="2"> error = (GError *) 0x0</font>
</p><br><br>Igor M., M.Eng, P.Eng Network Architect<br><br>--- On <b>Mon, 11/9/09, Pallagi Zoltán <i><pzolee@balabit.hu></i></b> wrote:<br><blockquote style="border-left: 2px solid rgb(16, 16, 255); margin-left: 5px; padding-left: 5px;"><br>From: Pallagi Zoltán <pzolee@balabit.hu><br>Subject: Re: [syslog-ng] syslog-ng on solaris locks up after a while<br>To: imanassypov@rogers.com, "Syslog-ng users' and developers' mailing list" <syslog-ng@lists.balabit.hu><br>Date: Monday, November 9, 2009, 11:35 AM<br><br><div id="yiv198697965">
Igor Manassypov írta:
<blockquote type="cite">
<table border="0" cellpadding="0" cellspacing="0">
<tbody>
<tr>
<td style="font-family: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; font-size: inherit; line-height: inherit; font-size-adjust: inherit; font-stretch: inherit;" valign="top">
<p>Would this one make more sense?<br>
</p>
<p><font face="Arial" size="2"><br>
</font></p>
<p><font face="Arial" size="2">bash-3.00# ps -eaf | grep syslog</font>
<br>
<font face="Arial" size="2"> root 22562 22561 0 Nov 04
? 0:30 /usr/local/sbin/syslog-ng</font>
<br>
<font face="Arial" size="2"> root 22561 1 0 Nov 04
? 0:00 /usr/local/sbin/syslog-ng</font>
</p>
<p><font face="Arial" size="2">bash-3.00# truss -f -p 22562</font>
<br>
<font face="Arial" size="2">22562/2:
door_return(0x00000000, 0, 0x00000000, 0) (sleeping...)</font>
<br>
<font face="Arial" size="2">22562/1:
lwp_park(0x00000000, 0) (sleeping....)</font>
<br>
<font face="Arial" size="2">22562/1: Received signal
#11, SIGSEGV, in lwp_park() [default]</font>
<br>
<font face="Arial" size="2">22562/1: siginfo:
SIGSEGV pid=12717 uid=0</font>
<br>
<font face="Arial" size="2">22562/1:
lwp_park(0x00000000, 0) Err#4 EINTR</font>
</p>
<p><font face="Arial" size="2">Core was generated by
`/usr/local/sbin/syslog-ng'.</font>
<br>
<font face="Arial" size="2">Program terminated with signal 11,
Segmentation fault.</font>
<br>
<font face="Arial" size="2">[New process 88098 ]</font>
<br>
<font face="Arial" size="2">[New process 153634 ]</font>
<br>
<font face="Arial" size="2">#0 0xfed46df0 in __lwp_park ()
from /lib/libc.so.1</font>
<br>
<font face="Arial" size="2">#0 0xfed46df0 in __lwp_park ()
from /lib/libc.so.1</font>
</p>
<p><font face="Arial" size="2">bash-3.00# gdb syslog-ng core</font>
</p>
<font face="Arial" size="2">Core was generated by
`/usr/local/sbin/syslog-ng'.</font>
<br>
<font face="Arial" size="2">Program terminated with signal 11,
Segmentation fault.</font>
<br>
<font face="Arial" size="2">[New process 88098 ]</font>
<br>
<font face="Arial" size="2">[New process 153634 ]</font>
<br>
<font face="Arial" size="2">#0 0xfed46df0 in __lwp_park ()
from /lib/libc.so.1</font>
<br>
<font face="Arial" size="2">(gdb) </font></td>
</tr>
</tbody>
</table>
</blockquote>
Please show us output of "bt full" too<br>
<blockquote type="cite">
<table border="0" cellpadding="0" cellspacing="0">
<tbody>
<tr>
<td style="font-family: inherit; font-style: inherit; font-variant: inherit; font-weight: inherit; font-size: inherit; line-height: inherit; font-size-adjust: inherit; font-stretch: inherit;" valign="top"><br>
<br>
--- On <b>Tue, 11/3/09, Balazs Scheidler <i><a rel="nofollow" class="moz-txt-link-rfc2396E" ymailto="mailto:bazsi@balabit.hu" target="_blank" href="/mc/compose?to=bazsi@balabit.hu"><bazsi@balabit.hu></a></i></b>
wrote:<br>
<blockquote style="border-left: 2px solid rgb(16, 16, 255); margin-left: 5px; padding-left: 5px;"><br>
From: Balazs Scheidler <a rel="nofollow" class="moz-txt-link-rfc2396E" ymailto="mailto:bazsi@balabit..hu" target="_blank" href="/mc/compose?to=bazsi@balabit..hu"><bazsi@balabit..hu></a><br>
Subject: Re: [syslog-ng] syslog-ng on solaris locks up after a while<br>
To: <a rel="nofollow" class="moz-txt-link-abbreviated" ymailto="mailto:imanassypov@rogers.com" target="_blank" href="/mc/compose?to=imanassypov@rogers.com">imanassypov@rogers.com</a>, "Syslog-ng users' and developers' mailing
list" <a rel="nofollow" class="moz-txt-link-rfc2396E" ymailto="mailto:syslog-ng@lists.balabit.hu" target="_blank" href="/mc/compose?to=syslog-ng@lists.balabit.hu"><syslog-ng@lists.balabit.hu></a><br>
Cc: "Pallagi Zoltán" <a rel="nofollow" class="moz-txt-link-rfc2396E" ymailto="mailto:pzolee@balabit.hu" target="_blank" href="/mc/compose?to=pzolee@balabit.hu"><pzolee@balabit.hu></a>, <a rel="nofollow" class="moz-txt-link-abbreviated" ymailto="mailto:network@ci.com" target="_blank" href="/mc/compose?to=network@ci.com">network@ci.com</a><br>
Date: Tuesday, November 3, 2009, 2:11 PM<br>
<br>
<div class="plainMail">Hi,<br>
<br>
The problem is that you killed the supervisor process, which restarts<br>
syslog-ng in case it crashes.. However the hang is not in this part, but<br>
in its child.<br>
<br>
So by looking at the ps output, I'd say that in this situation you<br>
should have trussed 13621 and not its parent.<br>
<br>
On Tue, 2009-11-03 at 08:54 -0800, Igor Manassypov wrote:<br>
> Hi Zoltan,<br>
> <br>
> <br>
> Here are the traces:<br>
> <br>
> bash-3.00# ps -eaf | grep syslog<br>
> root 12694 12616 0 11:37:07 pts/1 0:00 grep syslog<br>
> root 13012 1 0 Oct 21 ? 0:00 syslog-ng -v<br>
> root 13013 13012 0 Oct 21 ? 0:41 syslog-ng -v<br>
> root 13620 1 0 Oct 08 ?<br>
> 0:00 /usr/local/sbin/syslog-ng<br>
> root 13621 13620 0 Oct 08 ?<br>
> 6:16 /usr/local/sbin/syslog-ng<br>
> bash-3.00# truss -f -p "13620"<br>
> 13620: waitid(P_PID, 13621, 0xFFBFF468, WEXITED|WTRAPPED)<br>
> (sleeping...)<br>
> <br>
> 13620: Received signal #11, SIGSEGV, in waitid() [default]<br>
> 13620: siginfo: SIGSEGV pid=12717 uid=0<br>
> 13620: waitid(P_PID, 13621, 0xFFBFF468, WEXITED|WTRAPPED) Err#4
EINTR<br>
> <br>
> Core was generated by `/usr/local/sbin/syslog-ng'.<br>
> Program terminated with signal 11, Segmentation fault.<br>
> [New process 79156 ]<br>
> #0 0xfed4ad80 in _waitid () from /lib/libc.so.1<br>
> (gdb) bt full<br>
> #0 0xfed4ad80 in _waitid () from /lib/libc.so.1<br>
> No symbol table info available.<br>
> #1 0xfecee038 in _waitpid () from /lib/libc.so.1<br>
> No symbol table info available.<br>
> #2 0xfed3a70c in waitpid () from /lib/libc.so.1<br>
> No symbol table info available.<br>
> #3 0x0003017c in g_process_start () at gprocess.c:1042<br>
> rc = 0<br>
> deadlock = 0<br>
> pid = 13621<br>
> __PRETTY_FUNCTION__ = "g_process_start"<br>
> #4 0x0001c214 in main (argc=1, argv=0xffbffd14) at main.c:371<br>
> cfg = (GlobalConfig *) 0x10034<br>
> rc = 310272<br>
> ctx = (GOptionContext *) 0x76030<br>
> error = (GError *) 0x0<br>
> <br>
> Please let me know if I can provide you with more information,<br>
> <br>
> Thanks!<br>
> <br>
> --- On Tue, 11/3/09, Pallagi Zoltán <<a rel="nofollow">pzolee@balabit.hu</a>>
wrote:<br>
> <br>
> From: Pallagi Zoltán <<a rel="nofollow">pzolee@balabit.hu</a>><br>
> Subject: Re: [syslog-ng] syslog-ng on solaris locks up
after a<br>
> while<br>
> To: <a rel="nofollow">imanassypov@rogers.com</a>,
"Syslog-ng users' and developers'<br>
> mailing list" <<a rel="nofollow">syslog-ng@lists.balabit.hu</a>><br>
> Received: Tuesday, November 3, 2009, 11:10 AM<br>
> <br>
> Hi Igor,<br>
> <br>
> Can you show me truss output or backtrace of the stuck<br>
> syslog-ng?:<br>
> truss:<br>
> <br>
> truss -f -p "syslog-ng pid"<br>
> <br>
> backtrace:<br>
> <br>
> kill -11 "syslog-ng pid" (syslog-ng will drop a core file)<br>
> gdb syslog-ng core<br>
> bt full<br>
> <br>
> Igor Manassypov írta: <br>
> > Hello,<br>
> > <br>
> > <br>
> > I am having an issue with a solaris installation of
the<br>
> > syslog-ng. It is configured such that all the logs are<br>
> > stored different per-ip folders. This is my
centralized<br>
> > logging device, so it is fairly heavily loaded with<br>
> > receiving logs from a few dozen hosts. The syslog-ng
process<br>
> > locks up every two to three weeks, with no messages
logging<br>
> > to any of the files. The only way of getting it back
is kill<br>
> > -9 the process and restart it.<br>
> > <br>
> > Is there any known issue of same sorts and is there
any<br>
> > other way around it other than recycling the daemon
every<br>
> > night?<br>
> > <br>
> > <br>
> > here is the version info:<br>
> > <br>
> > bash-3.00# syslog-ng --version<br>
> > syslog-ng 3.0.4<br>
> > Revision: ssh<br>
> > +git://<a rel="nofollow">bazsi@git.balabit</a>//var/scm/git/syslog-ng/syslog-ng-ose--mainline--3.0#master#1b5d618e301ad94aa20e692ffba16469dece8d10<br>
> > Compile-Date: Aug 11 2009 10:44:17<br>
> > Enable-Threads: on<br>
> > Enable-Debug: off<br>
> > Enable-GProf: off<br>
> > Enable-Memtrace: off<br>
> > Enable-Sun-STREAMS: on<br>
> > Enable-Sun-Door: on<br>
> > Enable-IPv6: off<br>
> > Enable-Spoof-Source: on<br>
> > Enable-TCP-Wrapper: off<br>
> > Enable-SSL: on<br>
> > Enable-SQL: on<br>
> > Enable-Linux-Caps: off<br>
> > Enable-Pcre: on<br>
> > <br>
> > bash-3.00# uname -a<br>
> > SunOS prelude 5.10 Generic_137137-09 sun4v sparc
SUNW,T5240<br>
> > Thanks!<br>
> > <br>
> > -igor<br>
> > <br>
> > Igor Manassypov., M.Eng, P.Eng, CCIE 23032, CCVP
Network<br>
> > Architect<br>
> > <br>
> >
____________________________________________________________<br>
> > <br>
> >
______________________________________________________________________________<br>
> > Member info: <a rel="nofollow" target="_blank" href="https://lists.balabit.hu/mailman/listinfo/syslog-ng">https://lists.balabit.hu/mailman/listinfo/syslog-ng</a><br>
> > Documentation: <a rel="nofollow" target="_blank" href="http://www.balabit.com/support/documentation/?product=syslog-ng">http://www.balabit.com/support/documentation/?product=syslog-ng</a><br>
> > FAQ: <a rel="nofollow" target="_blank" href="http://www.campin.net/syslog-ng/faq.html">http://www.campin.net/syslog-ng/faq.html</a><br>
> > <br>
> > <br>
> <br>
> <br>
>
______________________________________________________________________________<br>
> Member info: <a rel="nofollow" target="_blank" href="https://lists.balabit.hu/mailman/listinfo/syslog-ng">https://lists.balabit.hu/mailman/listinfo/syslog-ng</a><br>
> Documentation: <a rel="nofollow" target="_blank" href="http://www.balabit.com/support/documentation/?product=syslog-ng">http://www.balabit.com/support/documentation/?product=syslog-ng</a><br>
> FAQ: <a rel="nofollow" target="_blank" href="http://www.campin.net/syslog-ng/faq.html">http://www.campin.net/syslog-ng/faq..html</a><br>
> <br>
-- <br>
Bazsi<br>
<br>
<br>
</div>
</blockquote>
</td>
</tr>
</tbody>
</table>
<pre><hr size="4" width="90%"><br>______________________________________________________________________________<br>Member info: <a rel="nofollow" class="moz-txt-link-freetext" target="_blank" href="https://lists.balabit.hu/mailman/listinfo/syslog-ng">https://lists.balabit.hu/mailman/listinfo/syslog-ng</a><br>Documentation: <a rel="nofollow" class="moz-txt-link-freetext" target="_blank" href="http://www.balabit.com/support/documentation/?product=syslog-ng">http://www.balabit.com/support/documentation/?product=syslog-ng</a><br>FAQ: <a rel="nofollow" class="moz-txt-link-freetext" target="_blank" href="http://www.campin.net/syslog-ng/faq.html">http://www.campin.net/syslog-ng/faq.html</a><br><br> </pre>
</blockquote>
<br>
</div></blockquote></td></tr></table>