<table cellspacing="0" cellpadding="0" border="0" ><tr><td valign="top" style="font: inherit;">I see,<br><br>I will resend the trace once I catch it locked up.<br><br>Thanks for your help,<br><br>-igor<br><br>--- On <b>Tue, 11/3/09, Balazs Scheidler <i><bazsi@balabit.hu></i></b> wrote:<br><blockquote style="border-left: 2px solid rgb(16, 16, 255); margin-left: 5px; padding-left: 5px;"><br>From: Balazs Scheidler <bazsi@balabit.hu><br>Subject: Re: [syslog-ng] syslog-ng on solaris locks up after a while<br>To: imanassypov@rogers.com, "Syslog-ng users' and developers' mailing list" <syslog-ng@lists.balabit.hu><br>Cc: "Pallagi Zoltán" <pzolee@balabit.hu>, network@ci.com<br>Received: Tuesday, November 3, 2009, 2:11 PM<br><br><div class="plainMail">Hi,<br><br>The problem is that you killed the supervisor process, which restarts<br>syslog-ng in case it crashes. However the hang is not in this part, but<br>in its child.<br><br>So by
looking at the ps output, I'd say that in this situation you<br>should have trussed 13621 and not its parent.<br><br>On Tue, 2009-11-03 at 08:54 -0800, Igor Manassypov wrote:<br>> Hi Zoltan,<br>> <br>> <br>> Here are the traces:<br>> <br>> bash-3.00# ps -eaf | grep syslog<br>> root 12694 12616 0 11:37:07 pts/1 0:00 grep syslog<br>> root 13012 1 0 Oct 21 ? 0:00 syslog-ng -v<br>> root 13013 13012 0 Oct 21 ? 0:41 syslog-ng -v<br>> root 13620 1 0 Oct 08 ?<br>> 0:00 /usr/local/sbin/syslog-ng<br>> root 13621
13620 0 Oct 08 ?<br>> 6:16 /usr/local/sbin/syslog-ng<br>> bash-3.00# truss -f -p "13620"<br>> 13620: waitid(P_PID, 13621, 0xFFBFF468, WEXITED|WTRAPPED)<br>> (sleeping...)<br>> <br>> 13620: Received signal #11, SIGSEGV, in waitid() [default]<br>> 13620: siginfo: SIGSEGV pid=12717 uid=0<br>> 13620: waitid(P_PID, 13621, 0xFFBFF468, WEXITED|WTRAPPED) Err#4 EINTR<br>> <br>> Core was generated by `/usr/local/sbin/syslog-ng'.<br>> Program terminated with signal 11, Segmentation fault.<br>> [New process 79156 ]<br>> #0 0xfed4ad80 in _waitid () from /lib/libc.so.1<br>> (gdb) bt full<br>> #0 0xfed4ad80 in _waitid () from /lib/libc.so.1<br>> No symbol table info available.<br>> #1 0xfecee038 in _waitpid () from /lib/libc.so.1<br>> No symbol table info available.<br>> #2 0xfed3a70c
in waitpid () from /lib/libc.so.1<br>> No symbol table info available.<br>> #3 0x0003017c in g_process_start () at gprocess.c:1042<br>> rc = 0<br>> deadlock = 0<br>> pid = 13621<br>> __PRETTY_FUNCTION__ = "g_process_start"<br>> #4 0x0001c214 in main (argc=1, argv=0xffbffd14) at main.c:371<br>> cfg = (GlobalConfig *) 0x10034<br>> rc = 310272<br>> ctx = (GOptionContext *) 0x76030<br>> error = (GError *) 0x0<br>> <br>> Please let me know if I can provide you with more information,<br>> <br>> Thanks!<br>> <br>> --- On Tue, 11/3/09, Pallagi Zoltán <<a ymailto="mailto:pzolee@balabit.hu"
href="/mc/compose?to=pzolee@balabit.hu">pzolee@balabit.hu</a>> wrote:<br>> <br>> From: Pallagi Zoltán <<a ymailto="mailto:pzolee@balabit.hu" href="/mc/compose?to=pzolee@balabit.hu">pzolee@balabit.hu</a>><br>> Subject: Re: [syslog-ng] syslog-ng on solaris locks up after a<br>> while<br>> To: <a ymailto="mailto:imanassypov@rogers.com" href="/mc/compose?to=imanassypov@rogers.com">imanassypov@rogers.com</a>, "Syslog-ng users' and developers'<br>> mailing list" <<a ymailto="mailto:syslog-ng@lists.balabit.hu" href="/mc/compose?to=syslog-ng@lists.balabit.hu">syslog-ng@lists.balabit.hu</a>><br>> Received: Tuesday, November 3, 2009, 11:10 AM<br>>
<br>> Hi Igor,<br>> <br>> Can you show me truss output or backtrace of the stuck<br>> syslog-ng?:<br>> truss:<br>> <br>> truss -f -p "syslog-ng pid"<br>> <br>> backtrace:<br>> <br>> kill -11 "syslog-ng pid" (syslog-ng will drop a core file)<br>> gdb syslog-ng core<br>> bt full<br>> <br>> Igor Manassypov írta: <br>>
> Hello,<br>> > <br>> > <br>> > I am having an issue with a solaris installation of the<br>> > syslog-ng. It is configured such that all the logs are<br>> > stored different per-ip folders. This is my centralized<br>> > logging device, so it is fairly heavily loaded with<br>> > receiving logs from a few dozen hosts. The syslog-ng process<br>> > locks up every two to three weeks, with no messages logging<br>> > to any of the files. The only way of getting it back is kill<br>> > -9 the
process and restart it.<br>> > <br>> > Is there any known issue of same sorts and is there any<br>> > other way around it other than recycling the daemon every<br>> > night?<br>> > <br>> > <br>> > here is the version info:<br>> > <br>> > bash-3.00# syslog-ng --version<br>> > syslog-ng 3.0.4<br>> > Revision: ssh<br>> > +git://<a ymailto="mailto:bazsi@git.balabit"
href="/mc/compose?to=bazsi@git.balabit">bazsi@git.balabit</a>//var/scm/git/syslog-ng/syslog-ng-ose--mainline--3.0#master#1b5d618e301ad94aa20e692ffba16469dece8d10<br>> > Compile-Date: Aug 11 2009 10:44:17<br>> > Enable-Threads: on<br>> > Enable-Debug: off<br>> > Enable-GProf: off<br>> > Enable-Memtrace: off<br>> > Enable-Sun-STREAMS: on<br>> > Enable-Sun-Door: on<br>> > Enable-IPv6: off<br>> > Enable-Spoof-Source: on<br>> > Enable-TCP-Wrapper: off<br>> > Enable-SSL:
on<br>> > Enable-SQL: on<br>> > Enable-Linux-Caps: off<br>> > Enable-Pcre: on<br>> > <br>> > bash-3.00# uname -a<br>> > SunOS prelude 5.10 Generic_137137-09 sun4v sparc SUNW,T5240<br>> > Thanks!<br>> > <br>> > -igor<br>> > <br>> > Igor Manassypov., M.Eng, P.Eng, CCIE 23032, CCVP Network<br>> > Architect<br>> > <br>> >
____________________________________________________________<br>> > <br>> > ______________________________________________________________________________<br>> > Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" target="_blank">https://lists.balabit.hu/mailman/listinfo/syslog-ng</a><br>> > Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" target="_blank">http://www.balabit.com/support/documentation/?product=syslog-ng</a><br>> > FAQ: <a href="http://www.campin.net/syslog-ng/faq.html" target="_blank">http://www.campin.net/syslog-ng/faq.html</a><br>> > <br>> > <br>>
<br>> <br>> ______________________________________________________________________________<br>> Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" target="_blank">https://lists.balabit.hu/mailman/listinfo/syslog-ng</a><br>> Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" target="_blank">http://www.balabit.com/support/documentation/?product=syslog-ng</a><br>> FAQ: <a href="http://www.campin.net/syslog-ng/faq.html" target="_blank">http://www.campin.net/syslog-ng/faq.html</a><br>> <br>-- <br>Bazsi<br><br><br></div></blockquote></td></tr></table>