[syslog-ng] Parser-DB Memory Leak

[ILLES, Marton]

On Fri, 2009-04-24 at 16:20 -0500, Martin Holste wrote:
> Ok, looks like it's still leaking.  I haven't used valgrind before,
> but here is the output with leak-check=full:

> ==27470== 19,854 (18,775 direct, 1,079 indirect) bytes in 18,684
> blocks are definitely lost in loss record 11 of 13
> ==27470==    at 0x40053C0: malloc (vg_replace_malloc.c:149)
> ==27470==    by 0x4CE615: g_malloc (in /lib/libglib-2.0.so.0.1200.3)
> ==27470==    by 0x4E06A0: g_strndup (in /lib/libglib-2.0.so.0.1200.3)
> ==27470==    by 0x807D30B: r_find_node
> (in /usr/local/syslog-ng-3.0-git/sbin/syslog-ng)
> ==27470==    by 0x807D0F8: r_find_node
> (in /usr/local/syslog-ng-3.0-git/sbin/syslog-ng)
> ==27470==    by 0x807D258: r_find_node
> (in /usr/local/syslog-ng-3.0-git/sbin/syslog-ng)
> ==27470==    by 0x807D0F8: r_find_node
> (in /usr/local/syslog-ng-3.0-git/sbin/syslog-ng)
> ==27470==    by 0x807AA5F: log_pattern_database_lookup
> (in /usr/local/syslog-ng-3.0-git/sbin/syslog-ng)
> ==27470==    by 0x8062C0C: log_db_parser_process
> (in /usr/local/syslog-ng-3.0-git/sbin/syslog-ng)
> ==27470==    by 0x8061D8E: log_parser_process
> (in /usr/local/syslog-ng-3.0-git/sbin/syslog-ng)
> ==27470==    by 0x8062D70: log_parser_rule_process
> (in /usr/local/syslog-ng-3.0-git/sbin/syslog-ng)
> ==27470==    by 0x8062F4D: log_process_rule_process
> (in /usr/local/syslog-ng-3.0-git/sbin/syslog-ng)

Hmm, I tried to reproduce the problem, but so far without any luck. My
guess is that only some message trigger this problem probably because of
a partial match. Could you send me some example messages when you
experience this problem?

Marton

> Hopefully this helps!
> 
> --Martin
> 
> On Fri, Apr 24, 2009 at 1:29 PM, Martin Holste <mcholste at gmail.com>
> wrote:
>         Wow!  That was quick!  I'm running the new code now, but it
>         will take a few hours to see if the memory grows too large.
>         Thanks for the quick commit.
>         
>         --Martin
>         
>         
>         
>         On Fri, Apr 24, 2009 at 12:29 PM, ILLES, Marton
>         <illes.marton at balabit.hu> wrote:
>                 On Fri, 2009-04-24 at 18:28 +0200, ILLES, Marton
>                 wrote:
>                 > On Fri, 2009-04-24 at 10:29 -0500, Martin Holste
>                 wrote:
>                 > > I'm running tests with the db-parser() parser
>                 module and finding that
>                 > > the memory grows exponentially until the program
>                 finally dies with:
>                 > >
>                 > > ***MEMORY-ERROR***: syslog-ng[2769]: GSlice:
>                 failed to allocate 248
>                 > > bytes (alignment: 256): Cannot allocate memory
>                 > >
>                 >
>                 > snip
>                 >
>                 > > Should I recompile with memtrace on?
>                 > >
>                 >
>                 > Running it in valgrind would be even more useful
>                 also enabling core
>                 > might help as well.
>                 >
>                 > M
>                 
>                 
>                 I think I have got it:
>                 
>                 commit 778cb9b27366425153a6141d4966a1a464e97e79
>                 Author: Marton Illes <marci at balabit.hu>
>                 Date:   Fri Apr 24 19:24:40 2009 +0200
>                 
>                    fixed a memory leak in db-parser() dynamic value
>                 handling
>                 
>                    Reported by: Martin Holste
>                 
>                 diff --git a/src/logpatterns.c b/src/logpatterns.c
>                 index f1012eb..fda9088 100644
>                 --- a/src/logpatterns.c
>                 +++ b/src/logpatterns.c
>                 @@ -312,6 +312,7 @@
>                 log_pattern_database_lookup(LogPatternDatabase
>                 *self, LogMessage *msg)
>                                */
>                 
>                               g_ptr_array_free(match_names, TRUE);
>                 +              g_array_free(matches, FALSE);
>                               return ((LogDBResult *)
>                 msg_node->value);
>                             }
>                           g_ptr_array_free(match_names, TRUE);
>                 
>                 Also in my git tree, please give it a try.
>                 http://git.balabit.hu/?p=marci/syslog-ng-3.0.git;a=commit;h=778cb9b27366425153a6141d4966a1a464e97e79
>                 
>                 
>                 best,
>                 
>                 Marton
>                 
>                 --
>                 Key fingerprint = F78C 25CA 5F88 6FAF EA21 779D 3279
>                 9F9E 1155 670D
>                 
>                 
>                 ______________________________________________________________________________
>                 Member info:
>                 https://lists.balabit.hu/mailman/listinfo/syslog-ng
>                 Documentation:
>                 http://www.balabit.com/support/documentation/?product=syslog-ng
>                 FAQ: http://www.campin.net/syslog-ng/faq.html
>                 
>                 
>         
>         
> 
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.campin.net/syslog-ng/faq.html
> 
-- 
Key fingerprint = F78C 25CA 5F88 6FAF EA21 779D 3279 9F9E 1155 670D