[syslog-ng] syslog-ng crashed

[Subrahmanyam Manuguri]

Thank you Bazsi for the reply.

The file and other destinations in my configuration file are fixed
during the boot up.  It is possible to change the configuration during
the system life through cli commands but i don't see anybody touching
the configuration during the boot up. I will double check that anyway.

Anyway, following is my configuration file's destination list. Am i
doing something wrong?

Appreciate your help very much.

++++++++++++++++++++++++++++++++++++++++++

destination dest_console { file("/dev/console"); };

# Console log, comment it to turn off console logging, done by log_console.sh
#log { source(src); filter(f_emerg); destination(dest_console); };
#log { source(kern_source); filter(f_emerg); destination(dest_console); };

# File destinations
destination dest { file("/var/log/messages"); };
destination kern_destination { file("/var/log/messages" fsync(yes) );};
destination sip_dest { file("/var/log/siplog" fsync(yes) template("$MSG\n")); };
destination dsp_dest { file("/var/log/dsplog" fsync(yes)
template("$ISODATE: $HOST: $PROGRAM: $MSG\n")); };

# File log objects
log     { source(src); filter(f_debug); destination(dest); };
log { source(kern_source); filter(f_debug); destination(kern_destination); };
log { source(sip_src); destination(sip_dest); };
log { source(dsp_src); destination(dsp_dest); };

+++++++++++++++++++++++++++++++++++++++++

Thanks,
Subra

On Sat, Feb 5, 2011 at 6:39 AM, Balazs Scheidler <bazsi at balabit.hu> wrote:
> On Mon, 2011-01-24 at 22:12 -0800, Subrahmanyam Manuguri wrote:
>> Hi,
>>
>> I am using the following version of syslog-ng server on a PowerPC
>> based system running Montavista Linux.
>>
>>
>> syslog-ng --version
>> syslog-ng 3.0.3
>> Revision: ssh+git://bazsi@git.balabit//var/scm/git/syslog-ng/syslog-ng-ose--mainline--3.0#master#08c9bf9d98e4e021756adc12c42605840140ba8b
>> Compile-Date: Jul 23 2009 14:23:17
>> Enable-Threads: off
>> Enable-Debug: off
>> Enable-GProf: off
>> Enable-Memtrace: off
>> Enable-Sun-STREAMS: off
>> Enable-Sun-Door: off
>> Enable-IPv6: on
>> Enable-Spoof-Source: off
>> Enable-TCP-Wrapper: on
>> Enable-SSL: on
>> Enable-SQL: off
>> Enable-Linux-Caps: on
>> Enable-Pcre: on
>>
>> Of late, I am running into the syslog-ng crashes during the boot up of
>> the system. At the boot up time the number of messages are
>> significantly more. I was able to collect the stack trace.
>>
>> (gdb) bt
>> #0  0x00000000 in ?? ()
>> #1  0x10024d0c in log_pipe_queue ()
>> #2  0x10024c1c in log_multiplexer_queue ()
>> #3  0x100304a8 in log_pipe_queue ()
>> #4  0x10030440 in log_source_group_queue ()
>> #5  0x1000f218 in log_pipe_queue ()
>> #6  0x1000f1b4 in log_pipe_forward_msg ()
>> #7  0x1000f218 in log_pipe_queue ()
>> #8  0x1000f1b4 in log_pipe_forward_msg ()
>> #9  0x10045030 in log_pipe_queue ()
>> #10 0x10044f08 in log_source_queue ()
>> #11 0x100318d0 in log_pipe_queue ()
>> #12 0x1003185c in log_reader_handle_line ()
>> #13 0x10031bf4 in log_reader_fetch_log ()
>> #14 0x100315b8 in log_reader_fd_dispatch ()
>> #15 0x0ff132e4 in g_main_context_dispatch () from /usr/lib/libglib-2.0.so.0
>> #16 0x0ff16c9c in ?? () from /usr/lib/libglib-2.0.so.0
>> #17 0x0ff17248 in g_main_context_iteration () from /usr/lib/libglib-2.0.so.0
>> #18 0x10006200 in main_loop_run ()
>> #19 0x100069c4 in main ()
>> (gdb)
>>
>> The configuration i am running with is
>>
>>
>> options {
>>        time_sleep(10);
>>        log_fetch_limit(250);
>>        log_fifo_size(2000);
>>        keep_timestamp(no);
>>        dns_cache(no);
>>        dns_cache_hosts(/etc/hosts);
>>        use_dns(persist_only);
>>        flush_timeout(200);
>>        perm(0644);
>>        stats_freq(3600);
>>        stats_level(0);
>>        ts_format(iso);
>>        time_reopen(10);
>>        create_dirs(yes);
>>        dir_perm(755);
>> };
>>
>> Can somebody please help me in tweaking the right parameters to avoid
>> the problem (if that solves the problem)?
>>
>> If this is a known problem, is there a patch available?
>>
>> Appreciate your help very much.
>
> Hmm.. your version of syslog-ng is quite outdated. I have some vague
> remembering that there was a similar bug fixed, but the only clue I've
> found was before 3.0.3 got released:
>
>    [logprocess] fixed log_process_queue to handle NULL "next" pointers (fixes: pub#49)
>
>    With a configuration which had a filter() but no destination, it can happen
>    that the next pointer of a filter() node points to nowhere.  This patch
>    handles that case without causing a crash.
>
>    Reported-By: Jan Schaumann
>
> The bug seems to be related to your configuration file though, since it seems
> to be dereferencing a NULL function pointer while going "towards" the destination
> side of your configuration. It is probably triggered by a single message, which
> occurs during bootup, and doesn't occur later.
>
> Can you try 3.0.10 (the latest in 3.0.x, or perhaps an even newer version) in
> the same environment? If it crashes, then debug symbols would help even more.
>
> Thanks for the report.
>
> --
> Bazsi
>
>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.campin.net/syslog-ng/faq.html
>
>