<table cellspacing="0" cellpadding="0" border="0" ><tr><td valign="top" style="font: inherit;">Hi Balazs,<br><br><br>Thanks for your prompt reply. Can you please direct me to the link where I can obtain the patch?<br><br>Thanks!<br><br>-igor<br><br>Igor M., M.Eng, P.Eng Network Architect<br><br>--- On <b>Thu, 11/12/09, Balazs Scheidler <i><bazsi@balabit.hu></i></b> wrote:<br><blockquote style="border-left: 2px solid rgb(16, 16, 255); margin-left: 5px; padding-left: 5px;"><br>From: Balazs Scheidler <bazsi@balabit.hu><br>Subject: Re: [syslog-ng] syslog-ng on solaris locks up after a while<br>To: imanassypov@rogers.com, "Syslog-ng users' and developers' mailing list" <syslog-ng@lists.balabit.hu><br>Cc: "Pallagi Zoltán" <pzolee@balabit.hu><br>Date: Thursday, November 12, 2009, 11:11 AM<br><br><div class="plainMail">Hi,<br><br>This seems to be the same issue as the one fixed by this patch:<br><br>Author: Balazs Scheidler <<a
ymailto="mailto:bazsi@balabit.hu" href="/mc/compose?to=bazsi@balabit..hu">bazsi@balabit.hu</a>> 2009-08-30 11:41:24<br>Committer: Balazs Scheidler <<a ymailto="mailto:bazsi@balabit.hu" href="/mc/compose?to=bazsi@balabit.hu">bazsi@balabit.hu</a>> 2009-08-30 11:41:24<br>Parent: 1ad4da07d5305ba0140ac385d661ab6de25fc5f3 ([patterndb] estring parser length calculation must include ending quote)<br>Child: c2e8aa58763a89cab58d05fb7a2b2a18021413b4 ([logmsg] added support for ASA timestamps)<br>Branches: master, remotes/balabit/master, remotes/origin/master<br>Follows: v3.0.4<br>Precedes: <br><br> [afinter] don't block on the internal_msg_queue even in the threaded case (fixes: pub#48)<br> <br> A hang was reported in bugzilla ticket #48 which seems to have<br> been caused by MARK messages interfering with local messages:<br> <br> * if the MARK is due in
the same poll iteration as a local message<br> * the MARK timeout is checked and the internal source is marked as having<br> input available<br> * then the local message comes in pushing the mark timeout further ahead<br> in time<br> * then the internal() dispatch callback checks the mark timeout again,<br> but at this time it is already in the future -><br> * the dispatch callback falls back to fetching the internal message from<br> internal_msg_queue, assuming it was that which caused the dispatch<br> callback to be scheduled<br> * this blocks indefinitely.<br> <br> The solution is very simple: use g_async_queue_try_pop() instead of<br> g_async_queue_pop(), the dispatch code
already takes care about a<br> NULL message value.<br><br><br>On Tue, 2009-11-10 at 05:09 -0800, Igor Manassypov wrote:<br>> (gdb) bt full <br>> #0 0xfed46df0 in __lwp_park () from /lib/libc.so.1 <br>> No symbol table info available. <br>> #1 0xfed40c44 in cond_sleep_queue () from /lib/libc.so.1 <br>> No symbol table info available. <br>> #2 0xfed40e08 in cond_wait_queue () from /lib/libc.so.1 <br>> No symbol table info available. <br>> #3 0xfed41350 in cond_wait () from /lib/libc.so.1 <br>> No symbol table info available. <br>> #4 0xfed4138c in pthread_cond_wait () from /lib/libc.so.1 <br>> No symbol table info available. <br>> #5 0xff119d80 in g_async_queue_pop_intern_unlocked (queue=0x757e0,<br>> try=0, end_time=0x75618) at gasyncqueue.c:359 <br>> retval = (gpointer) 0xa15b8 <br>>
__PRETTY_FUNCTION__ = "g_async_queue_pop_intern_unlocked" <br>> #6 0xff119e80 in g_async_queue_pop (queue=0x757e0) at<br>> gasyncqueue.c:398 <br>> retval = (gpointer) 0x757e0 <br>> __PRETTY_FUNCTION__ = "g_async_queue_pop" <br>> #7 0x0003e984 in afinter_source_dispatch (source=0x8d260,<br>> callback=0x3e9dc <afinter_source_dispatch_msg>, user_data=0x8d1e0)<br>> <br>> at afinter.c:112 <br>> msg = (LogMessage *) 0xa0dc0 <br>> path_options = {flow_control = -1, matched = 0x0} <br>> tv = {tv_sec = 1257363112, tv_usec = 441817} <br>> #8 0xff143564 in g_main_context_dispatch (context=0x8d158) at<br>> gmain.c:2144 <br>> No locals. <br>> #9 0xff1459a4 in
g_main_context_iterate (context=0x8d158, block=1,<br>> dispatch=1, self=0x76030) at gmain.c:2778 <br>> max_priority = 2147483647 <br>> timeout = 4000 <br>> some_ready = 1 <br>> nfds = 4 <br>> allocated_nfds = 1 <br>> fds = (GPollFD *) 0x788c8 <br>> __PRETTY_FUNCTION__ = "g_main_context_iterate" <br>> #10 0xff146050 in g_main_context_iteration (context=0x8d158,<br>> may_block=1) at gmain.c:2841 <br>> retval = 1 <br>> #11 0x0001bc20 in main_loop_run (cfg=0xffbffbc8) at main.c:149 <br>> iters = 0 <br>> stats_timer_id = 0 <br>> #12
0x0001c260 in main (argc=1, argv=0xffbffd44) at main.c:394 <br>> cfg = (GlobalConfig *) 0x794d0 <br>> rc = 0 <br>> ctx = (GOptionContext *) 0x76030 <br>> error = (GError *) 0x0<br>> <br>> <br>> <br>> Igor M., M.Eng, P.Eng Network Architect<br>> <br>> --- On Mon, 11/9/09, Pallagi Zoltán <<a ymailto="mailto:pzolee@balabit.hu" href="/mc/compose?to=pzolee@balabit.hu">pzolee@balabit.hu</a>> wrote:<br>> <br>> From: Pallagi Zoltán <<a ymailto="mailto:pzolee@balabit.hu" href="/mc/compose?to=pzolee@balabit.hu">pzolee@balabit.hu</a>><br>> Subject: Re: [syslog-ng] syslog-ng on solaris locks up after a<br>>
while<br>> To: <a ymailto="mailto:imanassypov@rogers.com" href="/mc/compose?to=imanassypov@rogers.com">imanassypov@rogers.com</a>, "Syslog-ng users' and developers'<br>> mailing list" <<a ymailto="mailto:syslog-ng@lists.balabit.hu" href="/mc/compose?to=syslog-ng@lists.balabit.hu">syslog-ng@lists.balabit.hu</a>><br>> Date: Monday, November 9, 2009, 11:35 AM<br>> <br>> Igor Manassypov írta: <br>> > Would this one make more sense?<br>> > <br>> > <br>> > <br>> > bash-3.00# ps -eaf | grep syslog <br>>
> root 22562 22561 0 Nov 04 ?<br>> > 0:30 /usr/local/sbin/syslog-ng <br>> > root 22561 1 0 Nov 04 ?<br>> > 0:00 /usr/local/sbin/syslog-ng <br>> > <br>> > bash-3.00# truss -f -p 22562 <br>> > 22562/2: door_return(0x00000000, 0, 0x00000000, 0)<br>> > (sleeping...) <br>> > 22562/1: lwp_park(0x00000000, 0)<br>> > (sleeping....) <br>>
> 22562/1: Received signal #11, SIGSEGV, in<br>> > lwp_park() [default] <br>> > 22562/1: siginfo: SIGSEGV pid=12717 uid=0 <br>> > 22562/1: lwp_park(0x00000000, 0)<br>> > Err#4 EINTR <br>> > <br>> > Core was generated by `/usr/local/sbin/syslog-ng'. <br>> > Program terminated with signal 11, Segmentation fault. <br>> > [New process 88098 ] <br>> > [New process 153634 ] <br>>
> #0 0xfed46df0 in __lwp_park () from /lib/libc..so.1 <br>> > #0 0xfed46df0 in __lwp_park () from /lib/libc.so.1 <br>> > <br>> > bash-3..00# gdb syslog-ng core <br>> > <br>> > Core was generated by `/usr/local/sbin/syslog-ng'. <br>> > Program terminated with signal 11, Segmentation fault. <br>> > [New process 88098 ] <br>> > [New process 153634 ] <br>> > #0 0xfed46df0 in __lwp_park () from /lib/libc.so.1 <br>> > (gdb) <br>>
Please show us output of "bt full" too<br>> > <br>> > <br>> > --- On Tue, 11/3/09, Balazs Scheidler <<a ymailto="mailto:bazsi@balabit.hu" href="/mc/compose?to=bazsi@balabit.hu">bazsi@balabit.hu</a>><br>> > wrote:<br>> > <br>> > From: Balazs Scheidler <<a ymailto="mailto:bazsi@balabit..hu" href="/mc/compose?to=bazsi@balabit..hu">bazsi@balabit..hu</a>><br>> > Subject: Re: [syslog-ng] syslog-ng on solaris locks<br>> > up after a
while<br>> > To: <a ymailto="mailto:imanassypov@rogers.com" href="/mc/compose?to=imanassypov@rogers.com">imanassypov@rogers.com</a>, "Syslog-ng users' and<br>> > developers' mailing list"<br>> > <<a ymailto="mailto:syslog-ng@lists.balabit.hu" href="/mc/compose?to=syslog-ng@lists.balabit.hu">syslog-ng@lists.balabit.hu</a>><br>> > Cc: "Pallagi Zoltán" <<a ymailto="mailto:pzolee@balabit.hu" href="/mc/compose?to=pzolee@balabit.hu">pzolee@balabit.hu</a>>,<br>> > <a ymailto="mailto:network@ci.com"
href="/mc/compose?to=network@ci.com">network@ci.com</a><br>> > Date: Tuesday, November 3, 2009, 2:11 PM<br>> > <br>> > Hi,<br>> > <br>> > The problem is that you killed the supervisor<br>> > process, which restarts<br>> > syslog-ng in case it crashes.. However the hang is<br>> > not in this part,
but<br>> > in its child.<br>> > <br>> > So by looking at the ps output, I'd say that in this<br>> > situation you<br>> > should have trussed 13621 and not its parent.<br>> > <br>> > On Tue, 2009-11-03 at 08:54 -0800, Igor Manassypov<br>> > wrote:<br>> >
> Hi Zoltan,<br>> > > <br>> > > <br>> > > Here are the traces:<br>> > > <br>> > > bash-3.00# ps -eaf | grep syslog<br>> > > root 12694 12616 0 11:37:07 pts/1 0:00<br>> > grep syslog<br>> >
> root 13012 1 0 Oct 21 ? 0:00<br>> > syslog-ng -v<br>> > > root 13013 13012 0 Oct 21 ? 0:41<br>> > syslog-ng -v<br>> > > root 13620 1 0 Oct 08 ?<br>> > > 0:00 /usr/local/sbin/syslog-ng<br>>
> > root 13621 13620 0 Oct 08 ?<br>> > > 6:16 /usr/local/sbin/syslog-ng<br>> > > bash-3.00# truss -f -p "13620"<br>> > > 13620: waitid(P_PID, 13621, 0xFFBFF468, WEXITED|<br>> > WTRAPPED)<br>> > > (sleeping...)<br>> > > <br>> > > 13620:
Received signal #11, SIGSEGV, in<br>> > waitid() [default]<br>> > > 13620: siginfo: SIGSEGV pid=12717 uid=0<br>> > > 13620: waitid(P_PID, 13621, 0xFFBFF468, WEXITED|<br>> > WTRAPPED) Err#4 EINTR<br>> > > <br>> > > Core was generated by `/usr/local/sbin/syslog-ng'.<br>> > > Program terminated with signal 11, Segmentation<br>>
> fault.<br>> > > [New process 79156 ]<br>> > > #0 0xfed4ad80 in _waitid () from /lib/libc.so.1<br>> > > (gdb) bt full<br>> > > #0 0xfed4ad80 in _waitid () from /lib/libc.so.1<br>> > > No symbol table info available.<br>> > > #1 0xfecee038 in _waitpid () from /lib/libc.so.1<br>> >
> No symbol table info available.<br>> > > #2 0xfed3a70c in waitpid () from /lib/libc.so.1<br>> > > No symbol table info available.<br>> > > #3 0x0003017c in g_process_start () at<br>> > gprocess.c:1042<br>> > > rc = 0<br>> > > deadlock = 0<br>> > >
pid = 13621<br>> > > __PRETTY_FUNCTION__ = "g_process_start"<br>> > > #4 0x0001c214 in main (argc=1, argv=0xffbffd14)<br>> > at main.c:371<br>> > > cfg = (GlobalConfig *) 0x10034<br>> > > rc = 310272<br>> > > ctx = (GOptionContext *) 0x76030<br>>
> > error = (GError *) 0x0<br>> > > <br>> > > Please let me know if I can provide you with more<br>> > information,<br>> > > <br>> > > Thanks!<br>> > > <br>> > > --- On Tue, 11/3/09, Pallagi Zoltán<br>> >
<<a ymailto="mailto:pzolee@balabit.hu" href="/mc/compose?to=pzolee@balabit.hu">pzolee@balabit.hu</a>> wrote:<br>> > > <br>> > > From: Pallagi Zoltán <<a ymailto="mailto:pzolee@balabit.hu" href="/mc/compose?to=pzolee@balabit.hu">pzolee@balabit.hu</a>><br>> > > Subject: Re: [syslog-ng] syslog-ng on<br>> > solaris locks up after a<br>> > > while<br>>
> > To: <a ymailto="mailto:imanassypov@rogers.com" href="/mc/compose?to=imanassypov@rogers.com">imanassypov@rogers.com</a>, "Syslog-ng<br>> > users' and developers'<br>> > > mailing list" <<a ymailto="mailto:syslog-ng@lists.balabit.hu" href="/mc/compose?to=syslog-ng@lists.balabit.hu">syslog-ng@lists.balabit.hu</a>><br>> > > Received: Tuesday, November 3, 2009, 11:10<br>> > AM<br>> >
> <br>> > > Hi Igor,<br>> > > <br>> > > Can you show me truss output or backtrace<br>> > of the stuck<br>> > > syslog-ng?:<br>> > > truss:<br>> >
> <br>> > > truss -f -p "syslog-ng pid"<br>> > > <br>> > > backtrace:<br>> > > <br>> > > kill -11 "syslog-ng pid" (syslog-ng will<br>> > drop a core file)<br>>
> > gdb syslog-ng core<br>> > > bt full<br>> > > <br>> > > Igor Manassypov írta: <br>> > > > Hello,<br>> > > > <br>> > >
> <br>> > > > I am having an issue with a solaris<br>> > installation of the<br>> > > > syslog-ng. It is configured such that<br>> > all the logs are<br>> > > > stored different per-ip folders. This is<br>> > my centralized<br>> >
> > logging device, so it is fairly heavily<br>> > loaded with<br>> > > > receiving logs from a few dozen hosts.<br>> > The syslog-ng process<br>> > > > locks up every two to three weeks, with<br>> > no messages logging<br>> > > > to any of the files. The only way of<br>>
> getting it back is kill<br>> > > > -9 the process and restart it.<br>> > > > <br>> > > > Is there any known issue of same sorts<br>> > and is there any<br>> > > > other way around it other than recycling<br>> > the daemon
every<br>> > > > night?<br>> > > > <br>> > > > <br>> > > > here is the version info:<br>> > > > <br>> > > > bash-3.00# syslog-ng --version<br>> >
> > syslog-ng 3.0.4<br>> > > > Revision: ssh<br>> > > ><br>> > +git://<a ymailto="mailto:bazsi@git.balabit" href="/mc/compose?to=bazsi@git.balabit">bazsi@git.balabit</a>//var/scm/git/syslog-ng/syslog-ng-ose--mainline--3.0#master#1b5d618e301ad94aa20e692ffba16469dece8d10<br>> > > > Compile-Date: Aug 11 2009 10:44:17<br>> > >
> Enable-Threads: on<br>> > > > Enable-Debug: off<br>> > > > Enable-GProf: off<br>> > > > Enable-Memtrace: off<br>> > > > Enable-Sun-STREAMS: on<br>> > > > Enable-Sun-Door: on<br>> > > >
Enable-IPv6: off<br>> > > > Enable-Spoof-Source: on<br>> > > > Enable-TCP-Wrapper: off<br>> > > > Enable-SSL: on<br>> > > > Enable-SQL: on<br>> > > > Enable-Linux-Caps: off<br>> > > > Enable-Pcre:
on<br>> > > > <br>> > > > bash-3.00# uname -a<br>> > > > SunOS prelude 5.10 Generic_137137-09<br>> > sun4v sparc SUNW,T5240<br>> > > > Thanks!<br>> > > > <br>> > >
> -igor<br>> > > > <br>> > > > Igor Manassypov., M.Eng, P.Eng, CCIE<br>> > 23032, CCVP Network<br>> > > > Architect<br>> > > > <br>> > > ><br>> >
____________________________________________________________<br>> > > > <br>> > > ><br>> > ______________________________________________________________________________<br>> > > > Member info:<br>> > <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" target="_blank">https://lists.balabit.hu/mailman/listinfo/syslog-ng</a><br>> >
> > Documentation:<br>> > <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" target="_blank">http://www.balabit.com/support/documentation/?product=syslog-ng</a><br>> > > > FAQ:<br>> > <a href="http://www.campin.net/syslog-ng/faq.html" target="_blank">http://www.campin.net/syslog-ng/faq.html</a><br>> > > > <br>> > >
> <br>> > > <br>> > > <br>> > ><br>> > ______________________________________________________________________________<br>> > > Member info:<br>> > <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" target="_blank">https://lists.balabit.hu/mailman/listinfo/syslog-ng</a><br>> >
> Documentation:<br>> > <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" target="_blank">http://www.balabit.com/support/documentation/?product=syslog-ng</a><br>> > > FAQ: <a href="http://www.campin.net/syslog-ng/faq..html" target="_blank">http://www.campin.net/syslog-ng/faq..html</a><br>> > > <br>> > -- <br>> > Bazsi<br>> > <br>> >
<br>> > <br>> > <br>> > ____________________________________________________________<br>> > <br>> > ______________________________________________________________________________<br>> > Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" target="_blank">https://lists.balabit.hu/mailman/listinfo/syslog-ng</a><br>> > Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" target="_blank">http://www.balabit.com/support/documentation/?product=syslog-ng</a><br>> > FAQ: <a
href="http://www.campin.net/syslog-ng/faq.html" target="_blank">http://www.campin.net/syslog-ng/faq.html</a><br>> > <br>> > <br>> <br>> <br>> ______________________________________________________________________________<br>> Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" target="_blank">https://lists.balabit.hu/mailman/listinfo/syslog-ng</a><br>> Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" target="_blank">http://www.balabit.com/support/documentation/?product=syslog-ng</a><br>> FAQ: <a href="http://www.campin.net/syslog-ng/faq.html" target="_blank">http://www.campin.net/syslog-ng/faq.html</a><br>> <br>-- <br>Bazsi<br><br></div></blockquote></td></tr></table>