<div dir="auto">Hi,<div dir="auto"><br></div><div dir="auto">Unfortunately I am not sure this is a leak from what you have posted. The issues valgrind reported were one-off allocations caused by plugins being loaded into syslog-ng.</div><div dir="auto"><br></div><div dir="auto">The numbers (26, 35 and even 45mb) is not conclusive either. Syslog-ng uses internal data structures that are dynamically growing up to a point and then stabilize. Also, syslog-ng uses threads and per thread instances of these data structures, not to mention the libc's heap allocator that also tends to use per thread areas. Similarly stacks used by threads also get an allocation.</div><div dir="auto"><br></div><div dir="auto">You can get a glimpse of chunks being allocated to the process by checking /proc/PID/maps, and syslog-ng also has some stats counters that reflect its internal memory usage. (Scratch-buffers, etc).</div><div dir="auto"><br></div><div dir="auto">You can try to disable multi-threaded operation of syslog-ng and see if that decreases usage.</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Jun 9, 2020, 09:31 Vidya Balakrishnan <<a href="mailto:vidyabalakrishnan96@gmail.com">vidyabalakrishnan96@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_quote"><div dir="ltr" class="gmail_attr">Greetings,<br></div><div dir="ltr"><div><br></div><div>This mail is regarding Memory leak in Syslog-ng v3.19.1. The process gets launched at 26MB and then grows to 37MB or even upto 45MB.</div><div><br></div><div>Pasting the configurations used below: </div><div>@version: 3.19<br>@include "scl.conf"<br>source s_local {<br> #system();<br> #internal();<br> syslog(port(514) transport("udp"));<br>};<br>rewrite tag_SN {<br> set("ABC-XXXXXXX", value("HOST"));<br>};<br><br>destination d_Log_Server {<br> network("<a href="http://abcd.com/" target="_blank" rel="noreferrer">abcd.com</a>"<br> port(514)<br> transport(tls)<br> tls( ca-dir("/etc/ssl") peer-verify("required-trusted") )<br> time-zone("+00:00")<br> template("${R_YEAR}-${R_MONTH}-${R_DAY}T${R_HOUR}:${R_MIN}:${R_SEC}.${R_MSEC}${TZOFFSET} ${HOST} ${MSGHDR}${MESSAGE}\n")<br> );<br>};<br>log {<br> source(s_local);<br> rewrite(tag_SN);<br> destination(d_Log_Server);<br>};<br></div><div><br></div><div>Also, tried debugging using valgrind but did not resolve the issue.</div><div><br></div><div><p style="margin:0in;font-family:Calibri;font-size:11pt">==19671== Memcheck, a memory error detector</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19671== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19671== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19671== Command: /sbin/syslog-ng --cfgfile=/var/syslog-ng.conf</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19671== Parent PID: 2190</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19671==</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19671== Conditional jump or move depends on uninitialised value(s)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19671== at 0x401A8D4: index (in /lib/ld-linux.so.3)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19671== by 0x4007F33: expand_dynamic_string_token (in /lib/ld-linux.so.3)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19671==</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19671== Conditional jump or move depends on uninitialised value(s)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19671== at 0x401A8D8: index (in /lib/ld-linux.so.3)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19671== by 0x4007F33: expand_dynamic_string_token (in /lib/ld-linux.so.3)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19671==</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19671== Conditional jump or move depends on uninitialised value(s)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19671== at 0x4007F38: expand_dynamic_string_token (in /lib/ld-linux.so.3)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19671==</p><p style="margin:0in;font-family:Calibri;font-size:11pt">disInstr(arm): unhandled instruction: 0xEC510F1E</p><p style="margin:0in;font-family:Calibri;font-size:11pt"> cond=14(0xE) 27:20=197(0xC5) 4:4=1 3:0=14(0xE)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19671== valgrind: Unrecognised instruction at address 0x4cd6e68.</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19671== at 0x4CD6E68: _armv7_tick (in /lib/libcrypto.so.1.1)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19671== Your program just tried to execute an instruction that Valgrind</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19671== did not recognise. There are two possible reasons for this.</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19671== 1. Your program has a bug and erroneously jumped to a non-code</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19671== location. If you are running Memcheck and you just saw a</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19671== warning about a bad jump, it's probably your program's fault.</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19671== 2. The instruction is legitimate but Valgrind doesn't handle it,</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19671== i.e. it's Valgrind's fault. If you think this is the case or</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19671== you are not sure, please let us know and we'll try to fix it.</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19671== Either way, Valgrind will now raise a SIGILL signal which will</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19671== probably kill your program.</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== Invalid read of size 4</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== at 0x401A8D4: index (in /lib/ld-linux.so.3)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== by 0x40145BB: dl_open_worker (in /lib/ld-linux.so.3)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== Address 0x4f86bb4 is 3 bytes after a block of size 41 alloc'd</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== at 0x4847654: malloc (vg_replace_malloc.c:299)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== by 0x49B3BC3: g_malloc (gmem.c:94)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== by 0x49CEA17: g_strdup (gstrfuncs.c:363)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== by 0x49417BF: g_module_open (gmodule.c:543)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== by 0x489AFDB: plugin_dlopen_module_as_filename (plugin.c:252)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== by 0x489B8B3: plugin_dlopen_module_as_dir_and_filename (plugin.c:271)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== by 0x489B8B3: plugin_load_candidate_modules (plugin.c:457)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== by 0x4887017: cfg_lexer_preprocess (cfg-lexer.c:1068)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== by 0x4887017: cfg_lexer_lex (cfg-lexer.c:1110)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== by 0x48A876F: main_parse (cfg-grammar.c:3143)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== by 0x4887D8F: cfg_parser_parse (cfg-parser.c:369)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== by 0x48845BF: cfg_run_parser (cfg.c:504)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== by 0x48846B3: cfg_read_config (cfg.c:556)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== by 0x489766B: <span style="background:yellow">main_loop_read_and_init_config (mainloop.c:552)</span></p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700==</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== Conditional jump or move depends on uninitialised value(s)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== at 0x401A8D4: index (in /lib/ld-linux.so.3)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== by 0x40145BB: dl_open_worker (in /lib/ld-linux.so.3)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700==</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== Conditional jump or move depends on uninitialised value(s)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== at 0x401A8D8: index (in /lib/ld-linux.so.3)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== by 0x40145BB: dl_open_worker (in /lib/ld-linux.so.3)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700==</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== Conditional jump or move depends on uninitialised value(s)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== at 0x40145C0: dl_open_worker (in /lib/ld-linux.so.3)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700==</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== Conditional jump or move depends on uninitialised value(s)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== at 0x401AD9C: strcpy (in /lib/ld-linux.so.3)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== by 0x400A8D3: _dl_lookup_symbol_x (in /lib/ld-linux.so.3)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700==</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== Invalid read of size 4</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== at 0x401AC60: strcpy (in /lib/ld-linux.so.3)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== by 0x400A8D3: _dl_lookup_symbol_x (in /lib/ld-linux.so.3)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== Address 0x5185254 is 0 bytes after a block of size 20 alloc'd</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== at 0x4849EB4: realloc (vg_replace_malloc.c:785)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== by 0x49B3C9B: g_realloc (gmem.c:159)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== by 0x49FCAF3: _g_gnulib_vasnprintf (vasnprintf.c:5533)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== by 0x49FD263: _g_gnulib_vasprintf (printf.c:142)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== by 0x49F6B0B: g_vasprintf (gprintf.c:310)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== by 0x49CEB7F: g_strdup_vprintf (gstrfuncs.c:514)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== by 0x49CEBAB: g_strdup_printf (gstrfuncs.c:540)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== by 0x489AF7F: plugin_get_module_init_name (plugin.c:234)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== by 0x489B363: plugin_load_module (plugin.c:364)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== by 0x48ABCAB: pragma_parse (pragma-grammar.y:443)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== by 0x4887D8F: cfg_parser_parse (cfg-parser.c:369)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== by 0x4886E9B: cfg_lexer_parse_pragma (cfg-lexer.c:996)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== by 0x4886E9B: cfg_lexer_preprocess (cfg-lexer.c:1040)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== by 0x4886E9B: <span style="background:yellow">cfg_lexer_lex (cfg-lexer.c:1110)</span></p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700==</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== Conditional jump or move depends on uninitialised value(s)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== at 0x401AD80: strcpy (in /lib/ld-linux.so.3)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== by 0x400A8D3: _dl_lookup_symbol_x (in /lib/ld-linux.so.3)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700==</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== Use of uninitialised value of size 4</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== at 0x401ACB8: strcpy (in /lib/ld-linux.so.3)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== by 0x400A8D3: _dl_lookup_symbol_x (in /lib/ld-linux.so.3)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700==</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== Invalid read of size 4</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== at 0x401ADAC: strcpy (in /lib/ld-linux.so.3)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== by 0x400A8D3: _dl_lookup_symbol_x (in /lib/ld-linux.so.3)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== Address 0x533bdac is 3 bytes after a block of size 25 alloc'd</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== at 0x4849EB4: realloc (vg_replace_malloc.c:785)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== by 0x49B3C9B: g_realloc (gmem.c:159)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== by 0x49FCAF3: _g_gnulib_vasnprintf (vasnprintf.c:5533)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== by 0x49FD263: _g_gnulib_vasprintf (printf.c:142)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== by 0x49F6B0B: g_vasprintf (gprintf.c:310)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== by 0x49CEB7F: g_strdup_vprintf (gstrfuncs.c:514)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== by 0x49CEBAB: g_strdup_printf (gstrfuncs.c:540)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== by 0x489AF7F: plugin_get_module_init_name (plugin.c:234)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== by 0x489B363: plugin_load_module (plugin.c:364)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== by 0x489B6BB: plugin_find (plugin.c:199)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== by 0x489921B: msg_format_options_init (msg-format.c:81)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== by 0x4891A9B:<span style="background:yellow"> log_reader_options_init (logreader.c:728)</span></p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700==</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== Conditional jump or move depends on uninitialised value(s)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== at 0x401ADC4: strcpy (in /lib/ld-linux.so.3)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== by 0x400A8D3: _dl_lookup_symbol_x (in /lib/ld-linux.so.3)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700==</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== Conditional jump or move depends on uninitialised value(s)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== at 0x401A8D4: index (in /lib/ld-linux.so.3)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== by 0x4008BFB: _dl_map_object (in /lib/ld-linux.so.3)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700==</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== Conditional jump or move depends on uninitialised value(s)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== at 0x401A8D8: index (in /lib/ld-linux.so.3)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== by 0x4008BFB: _dl_map_object (in /lib/ld-linux.so.3)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700==</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== Conditional jump or move depends on uninitialised value(s)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== at 0x4008C00: _dl_map_object (in /lib/ld-linux.so.3)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700==</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== Syscall param epoll_pwait(sigmask) points to unaddressable byte(s)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== at 0x4BCC73C: <span style="background:yellow">epoll_pwait (in /lib/libc.so.6) </span></p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700== Address 0x0 is not stack'd, malloc'd or (recently) free'd</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19700==</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19671==</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19671== HEAP SUMMARY:</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19671== in use at exit: 11,980 bytes in 55 blocks</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19671== total heap usage: 179 allocs, 124 frees, 49,157 bytes allocated</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19671==</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19671== 419 (44 direct, 375 indirect) bytes in 1 blocks are definitely lost in loss record 44 of 46</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19671== at 0x4847654: malloc (vg_replace_malloc.c:299)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19671== by 0x49B3BC3: g_malloc (gmem.c:94)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19671== by 0x488BD07: g_process_set_argv_space (gprocess.c:569)</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19671== by 0x1127B: <span style="background:yellow">main (main.c:216)</span></p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19671==</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19671== LEAK SUMMARY:</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19671== definitely lost: 44 bytes in 1 blocks</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19671== indirectly lost: 375 bytes in 10 blocks</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19671== possibly lost: 0 bytes in 0 blocks</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19671== still reachable: 11,561 bytes in 44 blocks</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19671== suppressed: 0 bytes in 0 blocks</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19671== Reachable blocks (those to which a pointer was found) are not shown.</p><p style="margin:0in;font-family:Calibri;font-size:11pt">==19671== To see them, rerun with: --leak-check=full --show-leak-kinds=all</p><p style="margin:0in;font-family:Calibri;font-size:11pt"><br></p><p style="margin:0in;font-family:Calibri;font-size:11pt"><br></p><p style="margin:0in;font-family:Calibri;font-size:11pt">The memory usage keeps increasing. There are no local disks used to store the logs messages. It is directly sent to the Graylog server.</p><p style="margin:0in;font-family:Calibri;font-size:11pt">Could you please take a look at this issue and help resolving the same. <br></p><p style="margin:0in;font-family:Calibri;font-size:11pt"><br></p><p style="margin:0in;font-family:Calibri;font-size:11pt">Regards,</p><p style="margin:0in;font-family:Calibri;font-size:11pt">Vidya Balakrishnan</p></div></div>
</div></div>
______________________________________________________________________________<br>
Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" rel="noreferrer noreferrer" target="_blank">https://lists.balabit.hu/mailman/listinfo/syslog-ng</a><br>
Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" rel="noreferrer noreferrer" target="_blank">http://www.balabit.com/support/documentation/?product=syslog-ng</a><br>
FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" rel="noreferrer noreferrer" target="_blank">http://www.balabit.com/wiki/syslog-ng-faq</a><br>
<br>
</blockquote></div>