[syslog-ng] Parser-DB Memory Leak
ILLES, Marton
illes.marton at balabit.hu
Sat Apr 25 19:08:22 CEST 2009
On Sat, 2009-04-25 at 17:01 +0200, ILLES, Marton wrote:
> On Fri, 2009-04-24 at 16:20 -0500, Martin Holste wrote:
> > Ok, looks like it's still leaking. I haven't used valgrind before,
> > but here is the output with leak-check=full:
>
> > ==27470== 19,854 (18,775 direct, 1,079 indirect) bytes in 18,684
> > blocks are definitely lost in loss record 11 of 13
> > ==27470== at 0x40053C0: malloc (vg_replace_malloc.c:149)
> > ==27470== by 0x4CE615: g_malloc (in /lib/libglib-2.0.so.0.1200.3)
> > ==27470== by 0x4E06A0: g_strndup (in /lib/libglib-2.0.so.0.1200.3)
> > ==27470== by 0x807D30B: r_find_node
> > (in /usr/local/syslog-ng-3.0-git/sbin/syslog-ng)
> > ==27470== by 0x807D0F8: r_find_node
> > (in /usr/local/syslog-ng-3.0-git/sbin/syslog-ng)
> > ==27470== by 0x807D258: r_find_node
> > (in /usr/local/syslog-ng-3.0-git/sbin/syslog-ng)
> > ==27470== by 0x807D0F8: r_find_node
> > (in /usr/local/syslog-ng-3.0-git/sbin/syslog-ng)
> > ==27470== by 0x807AA5F: log_pattern_database_lookup
> > (in /usr/local/syslog-ng-3.0-git/sbin/syslog-ng)
> > ==27470== by 0x8062C0C: log_db_parser_process
> > (in /usr/local/syslog-ng-3.0-git/sbin/syslog-ng)
> > ==27470== by 0x8061D8E: log_parser_process
> > (in /usr/local/syslog-ng-3.0-git/sbin/syslog-ng)
> > ==27470== by 0x8062D70: log_parser_rule_process
> > (in /usr/local/syslog-ng-3.0-git/sbin/syslog-ng)
> > ==27470== by 0x8062F4D: log_process_rule_process
> > (in /usr/local/syslog-ng-3.0-git/sbin/syslog-ng)
>
> Hmm, I tried to reproduce the problem, but so far without any luck. My
> guess is that only some message trigger this problem probably because of
> a partial match. Could you send me some example messages when you
> experience this problem?
The problem was when the parser did not have a name configured and
syslog-ng incorrectly checked this situation and always malloced 1 byte
for the name which was never freed. Thanks Martin for helping tracking
it down.
Here is the patch:
commit a5f7bb56dc1c83431f531bf5726512f28c0e75e8
Author: Marton Illes <marci at balabit.hu>
Date: Sat Apr 25 19:02:53 2009 +0200
fixed a memory leak around parser match storing, check for parser node name length
Reported by: Martin Holste
diff --git a/src/radix.c b/src/radix.c
index 2caa2a1..787daff 100644
--- a/src/radix.c
+++ b/src/radix.c
@@ -691,7 +691,7 @@ r_find_node(RNode *root, gchar *whole_key, gchar *key, gint keylen, GArray *matc
match->ofs = match->ofs + (key + i) - whole_key;
match->len = match->len + len;
}
- if (parser_node->name)
+ if (parser_node->name_len)
*match_name = g_strndup(parser_node->name, parser_node->name_len);
else
*match_name = NULL;
It is also pushed to my git tree.
cheers,
Marton
>
> Marton
>
> > Hopefully this helps!
> >
> > --Martin
> >
> > On Fri, Apr 24, 2009 at 1:29 PM, Martin Holste <mcholste at gmail.com>
> > wrote:
> > Wow! That was quick! I'm running the new code now, but it
> > will take a few hours to see if the memory grows too large.
> > Thanks for the quick commit.
> >
> > --Martin
> >
> >
> >
> > On Fri, Apr 24, 2009 at 12:29 PM, ILLES, Marton
> > <illes.marton at balabit.hu> wrote:
> > On Fri, 2009-04-24 at 18:28 +0200, ILLES, Marton
> > wrote:
> > > On Fri, 2009-04-24 at 10:29 -0500, Martin Holste
> > wrote:
> > > > I'm running tests with the db-parser() parser
> > module and finding that
> > > > the memory grows exponentially until the program
> > finally dies with:
> > > >
> > > > ***MEMORY-ERROR***: syslog-ng[2769]: GSlice:
> > failed to allocate 248
> > > > bytes (alignment: 256): Cannot allocate memory
> > > >
> > >
> > > snip
> > >
> > > > Should I recompile with memtrace on?
> > > >
> > >
> > > Running it in valgrind would be even more useful
> > also enabling core
> > > might help as well.
> > >
> > > M
> >
> >
> > I think I have got it:
> >
> > commit 778cb9b27366425153a6141d4966a1a464e97e79
> > Author: Marton Illes <marci at balabit.hu>
> > Date: Fri Apr 24 19:24:40 2009 +0200
> >
> > fixed a memory leak in db-parser() dynamic value
> > handling
> >
> > Reported by: Martin Holste
> >
> > diff --git a/src/logpatterns.c b/src/logpatterns.c
> > index f1012eb..fda9088 100644
> > --- a/src/logpatterns.c
> > +++ b/src/logpatterns.c
> > @@ -312,6 +312,7 @@
> > log_pattern_database_lookup(LogPatternDatabase
> > *self, LogMessage *msg)
> > */
> >
> > g_ptr_array_free(match_names, TRUE);
> > + g_array_free(matches, FALSE);
> > return ((LogDBResult *)
> > msg_node->value);
> > }
> > g_ptr_array_free(match_names, TRUE);
> >
> > Also in my git tree, please give it a try.
> > http://git.balabit.hu/?p=marci/syslog-ng-3.0.git;a=commit;h=778cb9b27366425153a6141d4966a1a464e97e79
> >
> >
> > best,
> >
> > Marton
> >
> > --
> > Key fingerprint = F78C 25CA 5F88 6FAF EA21 779D 3279
> > 9F9E 1155 670D
> >
> >
> > ______________________________________________________________________________
> > Member info:
> > https://lists.balabit.hu/mailman/listinfo/syslog-ng
> > Documentation:
> > http://www.balabit.com/support/documentation/?product=syslog-ng
> > FAQ: http://www.campin.net/syslog-ng/faq.html
> >
> >
> >
> >
> >
> > ______________________________________________________________________________
> > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> > Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> > FAQ: http://www.campin.net/syslog-ng/faq.html
> >
--
Key fingerprint = F78C 25CA 5F88 6FAF EA21 779D 3279 9F9E 1155 670D
More information about the syslog-ng
mailing list