[Bug 278] New: Many memory leak problems happen when do configuration reloading
https://bugzilla.balabit.com/show_bug.cgi?id=278 Summary: Many memory leak problems happen when do configuration reloading Product: syslog-ng Version: 3.5.x Platform: PC OS/Version: Linux Status: NEW Severity: major Priority: unspecified Component: syslog-ng AssignedTo: bazsi@balabit.hu ReportedBy: xufeng.zhang@windriver.com Type of the Report: --- Estimated Hours: 0.0 Created an attachment (id=95) --> (https://bugzilla.balabit.com/attachment.cgi?id=95) the config and script files to reproduce this problem I found many memory leak problems on syslog-ng_3.5.4.1, I have resolved some of them, but there are still some others need to be resolved: 1). mutex problem such as: ==1077== 40 bytes in 1 blocks are definitely lost in loss record 514 of 636 ==1077== at 0x4A05F58: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==1077== by 0x36882478F4: g_malloc (in /lib64/libglib-2.0.so.0.2600.0) ==1077== by 0x36886024FE: ??? (in /lib64/libgthread-2.0.so.0.2600.0) ==1077== by 0x3688266B23: g_static_mutex_get_mutex_impl (in /lib64/libglib-2.0.so.0.2600.0) ==1077== by 0x523D83C: affile_dw_queue (affile-dest.c:263) ==1077== by 0x523D05B: affile_dd_queue (logpipe.h:320) ==1077== by 0x3688E32150: log_multiplexer_queue (logpipe.h:320) patch Fix-the-memory-leak-problem-for-mutex.patch could resolve them. 2). memory leak when HAVE_ENVIRON is defined: ==1077== 1,258 (208 direct, 1,050 indirect) bytes in 1 blocks are definitely lost in loss record 617 of 636 ==1077== at 0x4A05F58: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==1077== by 0x36882478F4: g_malloc (in /lib64/libglib-2.0.so.0.2600.0) ==1077== by 0x3688E2F5FF: g_process_set_argv_space (gprocess.c:502) ==1077== by 0x40164C: main (main.c:196) patch Fix-the-memory-leak-problem-when-HAVE_ENVIRON-defined.patch could resolve them. 3). memory leak problem for unconsumed item during configuration reloading: ==1077== 36,864 bytes in 45 blocks are definitely lost in loss record 636 of 636 ==1077== at 0x4A05F58: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==1077== by 0x36882478F4: g_malloc (in /lib64/libglib-2.0.so.0.2600.0) ==1077== by 0x3688E3D178: log_writer_flush (logwriter.c:973) ==1077== by 0x3688E3D2D2: log_writer_deinit (logwriter.c:1169) ==1077== by 0x523C09A: affile_dw_deinit (logpipe.h:268) ==1077== by 0x523C93B: affile_dd_deinit (logpipe.h:268) ==1077== by 0x3688E2A4FE: cfg_tree_stop (logpipe.h:268) ==1077== by 0x3688E3DB9F: main_loop_reload_config_apply (mainloop.c:540) ==1077== by 0x3688E644DA: iv_signal_event (iv_signal.c:170) ==1077== by 0x3688E62F28: iv_event_raw_got_event (iv_event_raw_posix.c:89) ==1077== by 0x3688E635F1: iv_fd_poll_and_run (iv_fd.c:163) ==1077== by 0x3688E63D73: iv_main (iv_main_posix.c:117) patch logwriter-still-free-the-unconsumed-item.patch 4). The remaining memory leak problems need to be resolved, and they are not in 3.4.2 version: ==1077== 24 bytes in 1 blocks are definitely lost in loss record 483 of 636 ==1077== at 0x4A04961: calloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==1077== by 0x3688247819: g_malloc0 (in /lib64/libglib-2.0.so.0.2600.0) ==1077== by 0x5023BCA: socket_options_new (socket-options.c:90) ==1077== by 0x502568A: afunix_sd_new_instance (afunix-source.c:241) ==1077== by 0x502691A: afsocket_parse (afsocket-grammar.y:537) ==1077== by 0x3688E440D1: plugin_parse_config (cfg-parser.h:83) ==1077== by 0x3688E4F0BC: main_parse (cfg-grammar.y:549) ==1077== by 0x3688E26DC2: cfg_run_parser (cfg-parser.h:83) ==1077== by 0x3688E26ED5: cfg_read_config (cfg.c:382) ==1077== by 0x3688E3DAE5: main_loop_reload_config_initiate (mainloop.c:611) ==1077== by 0x3688E644DA: iv_signal_event (iv_signal.c:170) ==1077== by 0x3688E62F28: iv_event_raw_got_event (iv_event_raw_posix.c:89) ==1077== ==1077== 24 bytes in 1 blocks are definitely lost in loss record 484 of 636 ==1077== at 0x4A04961: calloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==1077== by 0x3688247819: g_malloc0 (in /lib64/libglib-2.0.so.0.2600.0) ==1077== by 0x5023BCA: socket_options_new (socket-options.c:90) ==1077== by 0x502568A: afunix_sd_new_instance (afunix-source.c:241) ==1077== by 0x502691A: afsocket_parse (afsocket-grammar.y:537) ==1077== by 0x3688E440D1: plugin_parse_config (cfg-parser.h:83) ==1077== by 0x3688E4F0BC: main_parse (cfg-grammar.y:549) ==1077== by 0x3688E26DC2: cfg_run_parser (cfg-parser.h:83) ==1077== by 0x3688E26ED5: cfg_read_config (cfg.c:382) ==1077== by 0x3688E3E42E: main_loop_init (mainloop.c:729) ==1077== by 0x401774: main (main.c:246) ==1077== 64 bytes in 1 blocks are definitely lost in loss record 531 of 636 ==1077== at 0x4A04961: calloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==1077== by 0x3688247819: g_malloc0 (in /lib64/libglib-2.0.so.0.2600.0) ==1077== by 0x5025A81: transport_mapper_unix_new_instance (transport-mapper-unix.c:40) ==1077== by 0x5025B04: transport_mapper_unix_dgram_new (transport-mapper-unix.c:51) ==1077== by 0x50257C8: afunix_sd_new_dgram (afunix-source.c:262) ==1077== by 0x502691A: afsocket_parse (afsocket-grammar.y:537) ==1077== by 0x3688E440D1: plugin_parse_config (cfg-parser.h:83) ==1077== by 0x3688E4F0BC: main_parse (cfg-grammar.y:549) ==1077== by 0x3688E26DC2: cfg_run_parser (cfg-parser.h:83) ==1077== by 0x3688E26ED5: cfg_read_config (cfg.c:382) ==1077== by 0x3688E3DAE5: main_loop_reload_config_initiate (mainloop.c:611) ==1077== by 0x3688E644DA: iv_signal_event (iv_signal.c:170) ==1077== ==1077== 64 bytes in 1 blocks are definitely lost in loss record 532 of 636 ==1077== at 0x4A04961: calloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==1077== by 0x3688247819: g_malloc0 (in /lib64/libglib-2.0.so.0.2600.0) ==1077== by 0x5025A81: transport_mapper_unix_new_instance (transport-mapper-unix.c:40) ==1077== by 0x5025B04: transport_mapper_unix_dgram_new (transport-mapper-unix.c:51) ==1077== by 0x50257C8: afunix_sd_new_dgram (afunix-source.c:262) ==1077== by 0x502691A: afsocket_parse (afsocket-grammar.y:537) ==1077== by 0x3688E440D1: plugin_parse_config (cfg-parser.h:83) ==1077== by 0x3688E4F0BC: main_parse (cfg-grammar.y:549) ==1077== by 0x3688E26DC2: cfg_run_parser (cfg-parser.h:83) ==1077== by 0x3688E26ED5: cfg_read_config (cfg.c:382) ==1077== by 0x3688E3E42E: main_loop_init (mainloop.c:729) ==1077== by 0x401774: main (main.c:246) ==1077== 96 (24 direct, 72 indirect) bytes in 1 blocks are definitely lost in loss record 541 of 636 ==1077== at 0x4A05F58: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==1077== by 0x36882478F4: g_malloc (in /lib64/libglib-2.0.so.0.2600.0) ==1077== by 0x368825CC99: g_slice_alloc (in /lib64/libglib-2.0.so.0.2600.0) ==1077== by 0x368823D14D: g_list_prepend (in /lib64/libglib-2.0.so.0.2600.0) ==1077== by 0x3688E5DE0A: log_template_add_macro_elem (templates.c:778) ==1077== by 0x3688E5EC74: log_template_compile (templates.c:1232) ==1077== by 0x3688E507AF: main_parse (cfg-grammar.y:741) ==1077== by 0x3688E26DC2: cfg_run_parser (cfg-parser.h:83) ==1077== by 0x3688E26ED5: cfg_read_config (cfg.c:382) ==1077== by 0x3688E3DAE5: main_loop_reload_config_initiate (mainloop.c:611) ==1077== by 0x3688E644DA: iv_signal_event (iv_signal.c:170) ==1077== by 0x3688E62F28: iv_event_raw_got_event (iv_event_raw_posix.c:89) ==1077== 144 (48 direct, 96 indirect) bytes in 2 blocks are definitely lost in loss record 556 of 636 ==1077== at 0x4A05F58: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==1077== by 0x36882478F4: g_malloc (in /lib64/libglib-2.0.so.0.2600.0) ==1077== by 0x368825CC99: g_slice_alloc (in /lib64/libglib-2.0.so.0.2600.0) ==1077== by 0x368823D14D: g_list_prepend (in /lib64/libglib-2.0.so.0.2600.0) ==1077== by 0x3688E5DE0A: log_template_add_macro_elem (templates.c:778) ==1077== by 0x3688E5E661: log_template_compiler_process_token (templates.c:1129) ==1077== by 0x3688E5EBD8: log_template_compile (templates.c:1222) ==1077== by 0x3688E507AF: main_parse (cfg-grammar.y:741) ==1077== by 0x3688E26DC2: cfg_run_parser (cfg-parser.h:83) ==1077== by 0x3688E26ED5: cfg_read_config (cfg.c:382) ==1077== by 0x3688E3E42E: main_loop_init (mainloop.c:729) ==1077== by 0x401774: main (main.c:246) ==1077== 1,008 bytes in 42 blocks are definitely lost in loss record 606 of 636 ==1077== at 0x4A05F58: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==1077== by 0x36882478F4: g_malloc (in /lib64/libglib-2.0.so.0.2600.0) ==1077== by 0x368825CC99: g_slice_alloc (in /lib64/libglib-2.0.so.0.2600.0) ==1077== by 0x368823D14D: g_list_prepend (in /lib64/libglib-2.0.so.0.2600.0) ==1077== by 0x3688E5DE0A: log_template_add_macro_elem (templates.c:778) ==1077== by 0x3688E5EC74: log_template_compile (templates.c:1232) ==1077== by 0x523BC1E: affile_dd_new_instance (affile-dest.c:706) ==1077== by 0x523BCA8: affile_dd_new (affile-dest.c:722) ==1077== by 0x523EEED: affile_parse (affile-grammar.y:490) ==1077== by 0x3688E440D1: plugin_parse_config (cfg-parser.h:83) ==1077== by 0x3688E4FFCA: main_parse (cfg-grammar.y:641) ==1077== by 0x3688E26DC2: cfg_run_parser (cfg-parser.h:83) ==1077== 176 bytes in 1 blocks are definitely lost in loss record 563 of 636 ==1077== at 0x4A04961: calloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==1077== by 0x3688247819: g_malloc0 (in /lib64/libglib-2.0.so.0.2600.0) ==1077== by 0x3688E44299: poll_fd_events_new (poll-fd-events.c:77) ==1077== by 0x5022582: afsocket_sc_init (afsocket-source.c:91) ==1077== by 0x5021FC7: afsocket_sd_process_connection (logpipe.h:253) ==1077== by 0x5022B6D: afsocket_sd_init_method (afsocket-source.c:540) ==1077== by 0x5025855: afunix_sd_init (afunix-source.c:223) ==1077== by 0x3688E2BAF2: cfg_tree_start (logpipe.h:253) ==1077== by 0x3688E2716A: cfg_init (cfg.c:218) ==1077== by 0x3688E3E48C: main_loop_init (mainloop.c:527) ==1077== by 0x401774: main (main.c:246) ==1077== ==1077== 176 bytes in 1 blocks are definitely lost in loss record 564 of 636 ==1077== at 0x4A04961: calloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==1077== by 0x3688247819: g_malloc0 (in /lib64/libglib-2.0.so.0.2600.0) ==1077== by 0x3688E44299: poll_fd_events_new (poll-fd-events.c:77) ==1077== by 0x523AC56: affile_sd_construct_poll_events (affile-source.c:195) ==1077== by 0x523B696: affile_sd_init (affile-source.c:375) ==1077== by 0x3688E2BAF2: cfg_tree_start (logpipe.h:253) ==1077== by 0x3688E2716A: cfg_init (cfg.c:218) ==1077== by 0x3688E3E48C: main_loop_init (mainloop.c:527) ==1077== by 0x401774: main (main.c:246) ==1077== 352 bytes in 2 blocks are definitely lost in loss record 576 of 636 ==1077== at 0x4A04961: calloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==1077== by 0x3688247819: g_malloc0 (in /lib64/libglib-2.0.so.0.2600.0) ==1077== by 0x3688E44299: poll_fd_events_new (poll-fd-events.c:77) ==1077== by 0x523AC56: affile_sd_construct_poll_events (affile-source.c:195) ==1077== by 0x523B696: affile_sd_init (affile-source.c:375) ==1077== by 0x3688E2BAF2: cfg_tree_start (logpipe.h:253) ==1077== by 0x3688E2716A: cfg_init (cfg.c:218) ==1077== by 0x3688E3DBBE: main_loop_reload_config_apply (mainloop.c:543) ==1077== by 0x3688E644DA: iv_signal_event (iv_signal.c:170) ==1077== by 0x3688E62F28: iv_event_raw_got_event (iv_event_raw_posix.c:89) ==1077== by 0x3688E635F1: iv_fd_poll_and_run (iv_fd.c:163) ==1077== by 0x3688E63D73: iv_main (iv_main_posix.c:117) When udp connection is used, I found three new additional memory leaks(I have applied the attached three patches): ==25796== 3,072 bytes in 3 blocks are definitely lost in loss record 616 of 622 ==25796== at 0x4A05F58: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==25796== by 0x3B742478F4: g_malloc (in /lib64/libglib-2.0.so.0.2600.0) ==25796== by 0x3B74E3D1D8: log_writer_flush (logwriter.c:974) ==25796== by 0x3B74E3D2BC: log_writer_work_perform (logwriter.c:163) ==25796== by 0x3B74E3D327: log_writer_io_flush_output (logwriter.c:243) ==25796== by 0x3B74E63666: iv_fd_poll_and_run (iv_fd.c:167) ==25796== by 0x3B74E63E13: iv_main (iv_main_posix.c:117) ==25796== by 0x3B74E3D7E6: main_loop_run (mainloop.c:788) ==25796== by 0x4017A0: main (main.c:267) ==25796== 20 bytes in 2 blocks are definitely lost in loss record 17 of 622 ==25796== at 0x4A05F58: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==25796== by 0x3B6FA7C6B1: strdup (strdup.c:43) ==25796== by 0x3B74E4D082: _cfg_lexer_lex (cfg-lex.l:205) ==25796== by 0x3B74E293A1: cfg_lexer_lex (cfg-lexer.c:759) ==25796== by 0x3B74E5C7A6: rewrite_expr_parse (rewrite-expr-grammar.c:2824) ==25796== by 0x3B74E4EDD2: T.102 (cfg-parser.h:83) ==25796== by 0x3B74E5016D: main_parse (cfg-grammar.y:606) ==25796== by 0x3B74E26DC2: cfg_run_parser (cfg-parser.h:83) ==25796== by 0x3B74E26ED5: cfg_read_config (cfg.c:382) ==25796== by 0x3B74E3E4CE: main_loop_init (mainloop.c:729) ==25796== by 0x401774: main (main.c:246) ==25796== 40 bytes in 4 blocks are definitely lost in loss record 514 of 622 ==25796== at 0x4A05F58: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==25796== by 0x3B6FA7C6B1: strdup (strdup.c:43) ==25796== by 0x3B74E4D082: _cfg_lexer_lex (cfg-lex.l:205) ==25796== by 0x3B74E293A1: cfg_lexer_lex (cfg-lexer.c:759) ==25796== by 0x3B74E5C7A6: rewrite_expr_parse (rewrite-expr-grammar.c:2824) ==25796== by 0x3B74E4EDD2: T.102 (cfg-parser.h:83) ==25796== by 0x3B74E5016D: main_parse (cfg-grammar.y:606) ==25796== by 0x3B74E26DC2: cfg_run_parser (cfg-parser.h:83) ==25796== by 0x3B74E26ED5: cfg_read_config (cfg.c:382) ==25796== by 0x3B74E3DB85: main_loop_reload_config_initiate (mainloop.c:611) ==25796== by 0x3B74E6457A: iv_signal_event (iv_signal.c:170) ==25796== by 0x3B74E62FC8: iv_event_raw_got_event (iv_event_raw_posix.c:89) -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.
https://bugzilla.balabit.com/show_bug.cgi?id=278 --- Comment #1 from Xufeng Zhang <xufeng.zhang@windriver.com> 2014-04-15 08:03:01 --- Created an attachment (id=96) --> (https://bugzilla.balabit.com/attachment.cgi?id=96) Fix mutex memory leak -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.
https://bugzilla.balabit.com/show_bug.cgi?id=278 --- Comment #2 from Xufeng Zhang <xufeng.zhang@windriver.com> 2014-04-15 08:03:41 --- Created an attachment (id=97) --> (https://bugzilla.balabit.com/attachment.cgi?id=97) Fix the memory leak problem when HAVE_ENVIRON is defined -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.
https://bugzilla.balabit.com/show_bug.cgi?id=278 --- Comment #3 from Xufeng Zhang <xufeng.zhang@windriver.com> 2014-04-15 08:04:14 --- Created an attachment (id=98) --> (https://bugzilla.balabit.com/attachment.cgi?id=98) logwritter: still free the unconsumed item during reloading configuration -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.
https://bugzilla.balabit.com/show_bug.cgi?id=278 --- Comment #4 from Xufeng Zhang <xufeng.zhang@windriver.com> 2014-04-15 08:06:07 --- Steps to reproduce: 1). Decompress the attached testcase.tar.bz2, and copy the configuration files and test scripts to the test machine. 2). Run the test scripts: # ./fifo_loggen_full.sh # ./syslog_mem.sh -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.
https://bugzilla.balabit.com/show_bug.cgi?id=278 --- Comment #5 from Xufeng Zhang <xufeng.zhang@windriver.com> 2014-04-16 05:31:54 --- Created an attachment (id=99) --> (https://bugzilla.balabit.com/attachment.cgi?id=99) Don't allocate a new buffer if fails to consume current item -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.
https://bugzilla.balabit.com/show_bug.cgi?id=278 --- Comment #6 from Xufeng Zhang <xufeng.zhang@windriver.com> 2014-04-16 05:33:42 --- (In reply to comment #5)
Created an attachment (id=99) --> (https://bugzilla.balabit.com/attachment.cgi?id=99) [details] Don't allocate a new buffer if fails to consume current item
After apply this patch, below valgrind memory leak disappears: ==25796== 3,072 bytes in 3 blocks are definitely lost in loss record 616 of 622 ==25796== at 0x4A05F58: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==25796== by 0x3B742478F4: g_malloc (in /lib64/libglib-2.0.so.0.2600.0) ==25796== by 0x3B74E3D1D8: log_writer_flush (logwriter.c:974) ==25796== by 0x3B74E3D2BC: log_writer_work_perform (logwriter.c:163) ==25796== by 0x3B74E3D327: log_writer_io_flush_output (logwriter.c:243) ==25796== by 0x3B74E63666: iv_fd_poll_and_run (iv_fd.c:167) ==25796== by 0x3B74E63E13: iv_main (iv_main_posix.c:117) ==25796== by 0x3B74E3D7E6: main_loop_run (mainloop.c:788) ==25796== by 0x4017A0: main (main.c:267) But I'm not sure whether or not it would introduce any side effect. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.
https://bugzilla.balabit.com/show_bug.cgi?id=278 Xufeng Zhang <xufeng.zhang@windriver.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Severity|major |critical --- Comment #7 from Xufeng Zhang <xufeng.zhang@windriver.com> 2014-04-23 09:19:23 --- Hello, Does anybody know how to fix below kind of memory leak? it could be triggered by append below config line: ======================================================================= rewrite r_host_1 { set("10.15.15.2", value("HOST")); }; rewrite r_program_1 { set("xxx", value("PROGRAM")); }; destination d_1 { udp("10.15.15.4" port(514) localip("10.15.15.2") template("$MSG\n") flags(syslog-protocol) frac_digits(3) log_fifo_size(1000)); }; log { source(s_to_remote); rewrite(r_host_1); rewrite(r_program_1); filter(f_level_7); destination(d_1); }; ======================================================================= ==25796== 20 bytes in 2 blocks are definitely lost in loss record 17 of 622 ==25796== at 0x4A05F58: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==25796== by 0x3B6FA7C6B1: strdup (strdup.c:43) ==25796== by 0x3B74E4D082: _cfg_lexer_lex (cfg-lex.l:205) ==25796== by 0x3B74E293A1: cfg_lexer_lex (cfg-lexer.c:759) ==25796== by 0x3B74E5C7A6: rewrite_expr_parse (rewrite-expr-grammar.c:2824) ==25796== by 0x3B74E4EDD2: T.102 (cfg-parser.h:83) ==25796== by 0x3B74E5016D: main_parse (cfg-grammar.y:606) ==25796== by 0x3B74E26DC2: cfg_run_parser (cfg-parser.h:83) ==25796== by 0x3B74E26ED5: cfg_read_config (cfg.c:382) ==25796== by 0x3B74E3E4CE: main_loop_init (mainloop.c:729) ==25796== by 0x401774: main (main.c:246) ==25796== 40 bytes in 4 blocks are definitely lost in loss record 514 of 622 ==25796== at 0x4A05F58: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==25796== by 0x3B6FA7C6B1: strdup (strdup.c:43) ==25796== by 0x3B74E4D082: _cfg_lexer_lex (cfg-lex.l:205) ==25796== by 0x3B74E293A1: cfg_lexer_lex (cfg-lexer.c:759) ==25796== by 0x3B74E5C7A6: rewrite_expr_parse (rewrite-expr-grammar.c:2824) ==25796== by 0x3B74E4EDD2: T.102 (cfg-parser.h:83) ==25796== by 0x3B74E5016D: main_parse (cfg-grammar.y:606) ==25796== by 0x3B74E26DC2: cfg_run_parser (cfg-parser.h:83) ==25796== by 0x3B74E26ED5: cfg_read_config (cfg.c:382) ==25796== by 0x3B74E3DB85: main_loop_reload_config_initiate (mainloop.c:611) ==25796== by 0x3B74E6457A: iv_signal_event (iv_signal.c:170) ==25796== by 0x3B74E62FC8: iv_event_raw_got_event (iv_event_raw_posix.c:89) -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.
https://bugzilla.balabit.com/show_bug.cgi?id=278 Tusa Viktor <tusavik@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |tusavik@gmail.com --- Comment #8 from Tusa Viktor <tusavik@gmail.com> 2014-04-23 19:32:28 --- Hi Xufeng! Sorry for the late answer, we didn't have any time to investigate the issues. Thank you for your patches. I am trying to prepare your patches for integration. The first two are valid and fine, but for the third, casting the LogProtoClient back to a LogProtoTextClient is a layering violation. We should unref the message if during the flush the proto cannot consume it, but it is the duty of the proto to track it's internal thingies state. Oh, the situation is worse, partial comes from LogWriter's line_buffer, and we free it in LogProtoTextClient. Well, that's kind of a mess, but I don't have energy to think about the correct solution. Next days I'm on a conference, but I'll try to figure out something by next week and I'll definitely take a look on your other patches. The current patches are on github, talien/syslog-ng repo, f/memory-leak-fixes branch. The last patch is not working (at least it does not really fix the leak). Thank you for your effort! -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.
https://bugzilla.balabit.com/show_bug.cgi?id=278 --- Comment #9 from Xufeng Zhang <xufeng.zhang@windriver.com> 2014-04-24 04:08:43 --- Thanks a lot for your reply, Tusa! Please let me know if you require any further information, and it would be great if you could take a look at the "_cfg_lexer_lex" leak firstly when you are not busy. Please let me know ASAP if you have any update, I really appreciate your help. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.
https://bugzilla.balabit.com/show_bug.cgi?id=278 --- Comment #10 from Xufeng Zhang <xufeng.zhang@windriver.com> 2014-05-05 10:29:03 --- Ping? -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.
https://bugzilla.balabit.com/show_bug.cgi?id=278 --- Comment #11 from Tusa Viktor <tusavik@gmail.com> 2014-05-12 19:15:21 --- Hi Xufeng! Sorry for the late answer. I think I managed to fix all these leaks, the only doubt is about the logwriter ones, could you please verify them? The fixes are in my git repository at github, https://github.com/talien/syslog-ng on git branch f/memory-leak-fixes. I created a pull request, I hope it will be merged soon. Best Regards, Viktor -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.
https://bugzilla.balabit.com/show_bug.cgi?id=278 --- Comment #12 from Xufeng Zhang <xufeng.zhang@windriver.com> 2014-05-13 04:55:56 --- Thank you very much for all the fixes, Tusa! I'll test the fixes on my platform and let you know the result. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.
https://bugzilla.balabit.com/show_bug.cgi?id=278 --- Comment #13 from Xufeng Zhang <xufeng.zhang@windriver.com> 2014-05-14 04:15:17 --- Hi Tusa! After running a long-time testing, it is much better than previous, but there are still some memory leaks: 1). ==888== 371 bytes in 1 blocks are definitely lost in loss record 553 of 607 ==888== at 0x4A05F58: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==888== by 0x3AE26478F4: g_malloc (in /lib64/libglib-2.0.so.0.2600.0) ==888== by 0x52397D3: log_proto_file_writer_flush (logproto-file-writer.c:98) ==888== by 0x3AE1E41780: log_writer_flush (logproto-client.h:91) ==888== by 0x3AE1E418EC: log_writer_work_perform (logwriter.c:129) ==888== by 0x3AE1E41957: log_writer_io_flush_output (logwriter.c:209) ==888== by 0x3AE1E5EC29: iv_run_tasks (iv_task.c:48) ==888== by 0x3AE1E60E7B: iv_main (iv_main_posix.c:106) ==888== by 0x3AE1E41E16: main_loop_run (mainloop.c:737) ==888== by 0x4017A0: main (main.c:267) logproto-file-writer.c:98 self->partial = (guchar *)g_malloc(self->partial_len); 2). ==888== 3,105 (1,484 direct, 1,621 indirect) bytes in 1 blocks are definitely lost in loss record 600 of 607 ==888== at 0x4A05F58: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==888== by 0x3AE26478F4: g_malloc (in /lib64/libglib-2.0.so.0.2600.0) ==888== by 0x3AE1E3565E: log_msg_new (logmsg.c:1041) ==888== by 0x3AE1E3B61C: log_reader_work_perform (logreader.c:534) ==888== by 0x3AE1E3B867: log_reader_io_process_input (logreader.c:225) ==888== by 0x3AE1E606F1: iv_fd_poll_and_run (iv_fd.c:163) ==888== by 0x3AE1E60E73: iv_main (iv_main_posix.c:117) ==888== by 0x3AE1E41E16: main_loop_run (mainloop.c:737) ==888== by 0x4017A0: main (main.c:267) logreader.c:534 /* use the current time to get the time zone offset */ m = log_msg_new((gchar *) line, length, saddr, &self->options->parse_options); 3). ==888== 77,312 bytes in 151 blocks are definitely lost in loss record 607 of 607 ==888== at 0x4A05F58: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==888== by 0x3AE26478F4: g_malloc (in /lib64/libglib-2.0.so.0.2600.0) ==888== by 0x3AE1E41838: log_writer_flush (logwriter.c:949) ==888== by 0x3AE1E418EC: log_writer_work_perform (logwriter.c:129) ==888== by 0x3AE1E41957: log_writer_io_flush_output (logwriter.c:209) ==888== by 0x3AE1E5EC29: iv_run_tasks (iv_task.c:48) ==888== by 0x3AE1E60E7B: iv_main (iv_main_posix.c:106) ==888== by 0x3AE1E41E16: main_loop_run (mainloop.c:737) ==888== by 0x4017A0: main (main.c:267) logwriter.c:949 self->line_buffer->str = g_malloc(self->line_buffer->allocated_len); This is test for syslog-ng-3.4.2, if you need, I can run the test again for syslog-ng_3.5.4.1, but I don't think it will make much difference. I think all these memory leaks are new ones because this time my test is a little tighter(stop and restart the syslog-ng service on remote host continuously), so if you need me to close this bug and open a new one, please let me know! Thanks a lot for all your help! -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.
https://bugzilla.balabit.com/show_bug.cgi?id=278 --- Comment #14 from Xufeng Zhang <xufeng.zhang@windriver.com> 2014-05-15 07:22:11 --- For the above 3) case, I think there is a memory leak problem in modules/affile/logproto-file-writer.c: Take the following scenario, we call log_proto_file_writer_post() to post a message, if we have consumed the previous data and after register the new message, consumed variable will be TRUE, this indicates we will allocate a new self->line_buffer->str buffer in log_writer_flush() after we return from log_proto_file_writer_post(), but there are cases that the new message has not been copied to the partial buffer and freed in log_proto_file_writer_flush(), one case is log_proto_file_writer_flush() return LPS_ERROR when I/O error happened, I think this is the reason why my previous patch "Don't allocate a new buffer if fails to consume current item" partially works. How would we handle this situation? -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.
https://bugzilla.balabit.com/show_bug.cgi?id=278 --- Comment #15 from Xufeng Zhang <xufeng.zhang@windriver.com> 2014-06-03 05:24:12 --- (In reply to comment #14)
For the above 3) case, I think there is a memory leak problem in modules/affile/logproto-file-writer.c: Take the following scenario, we call log_proto_file_writer_post() to post a message, if we have consumed the previous data and after register the new message, consumed variable will be TRUE, this indicates we will allocate a new self->line_buffer->str buffer in log_writer_flush() after we return from log_proto_file_writer_post(), but there are cases that the new message has not been copied to the partial buffer and freed in log_proto_file_writer_flush(), one case is log_proto_file_writer_flush() return LPS_ERROR when I/O error happened, I think this is the reason why my previous patch "Don't allocate a new buffer if fails to consume current item" partially works. How would we handle this situation?
My above statement is not correct, from my test result, the below memory leak: 3). ==888== 77,312 bytes in 151 blocks are definitely lost in loss record 607 of 607 ==888== at 0x4A05F58: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==888== by 0x3AE26478F4: g_malloc (in /lib64/libglib-2.0.so.0.2600.0) ==888== by 0x3AE1E41838: log_writer_flush (logwriter.c:949) ==888== by 0x3AE1E418EC: log_writer_work_perform (logwriter.c:129) ==888== by 0x3AE1E41957: log_writer_io_flush_output (logwriter.c:209) ==888== by 0x3AE1E5EC29: iv_run_tasks (iv_task.c:48) ==888== by 0x3AE1E60E7B: iv_main (iv_main_posix.c:106) ==888== by 0x3AE1E41E16: main_loop_run (mainloop.c:737) ==888== by 0x4017A0: main (main.c:267) logwriter.c:949 self->line_buffer->str = g_malloc(self->line_buffer->allocated_len); is still come from the text-client, not from file-writer, so the fix ("logwriter: still free the unconsumed item during reloading configuration") on git branch f/memory-leak-fixes only partly works. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.
participants (1)
-
bugzilla＠bugzilla.balabit.com