<div dir="ltr"><div dir="ltr">Hi,<br><br>I was going through syslog-ng administrators guide and came across the following point from the given link:<br><br><b>"NOTE:Message contexts are persistent and are not lost when syslog-ng OSE is reloaded (SIGHUP), but are lost when syslog-ng OSE is restarted."</b><br><br><a href="https://www.syslog-ng.com/technical-documents/doc/syslog-ng-open-source-edition/3.18/administration-guide/72#TOPIC-1044162">https://www.syslog-ng.com/technical-documents/doc/syslog-ng-open-source-edition/3.18/administration-guide/72#TOPIC-1044162</a><br><br>1) Is this because of the message contexts that are persisted on reload, the memory is not freeing up at the time of reload of patterndb ?<div><br></div><div>2) Is there a way we can free-up these message contexts without a restart ?<br><div><br>Please correct me if I am wrong or missing something here.<br></div></div><div><br></div><div>Thanks,</div><div>Nitish</div><div><br></div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Jan 7, 2020 at 5:37 PM Nitish Saboo <<a href="mailto:nitish.saboo55@gmail.com">nitish.saboo55@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>Hi Peter,</div><div><br></div><div>Thanks for your response :)</div><div>I am seeing the memory usage getting increased ONLY at the time of reload of patterndb.<br>All other operation are having constant memory consumption.</div><div>Hence, my question is specific to reload operation if it is by default that patterndb component is not freeing up the memory on reload operation ? If it is by default, why is it so ?</div><div><b><i><br>The init and deinit can be repeated multiple times (init, deinit, init, deinit, ...) also usually the repeation of single init is fine (init, init, ...).<br></i></b></div><div>>> With this you meant initialising syslog-ng every-time for every reload operation?</div><div><b><i><br></i></b></div><div>Thanks<b style="font-style:italic">,</b></div><div>Nitish</div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Jan 7, 2020 at 5:05 PM Peter Kokai (pkokai) <<a href="mailto:Peter.Kokai@oneidentity.com" target="_blank">Peter.Kokai@oneidentity.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hello,<br>
<br>
Yes, we like to keep memory for ourselfs ;)[1]<br>
<br>
This is in fact not syslog-ng reload, and mostly about patterndb component. You could verify leak with valgrind or sanitizer. Please if you find a leak check it on upstream and report if you have any.<br>
<br>
Regarding the question that your application does leak or not is hard to tell withouth knowing it.<br>
Please share your application and we might be able to help you.<br>
<br>
Just a hint: things in syslog-ng usually like to be called like this:<br>
<br>
new<br>
init<br>
deinit<br>
free<br>
<br>
In case of reference counted object also use unref instead of free, and ref well to reference stuff.<br>
<br>
The init and deinit can be repeated multiple times (init, deinit, init, deinit, ...) also usually the repeation of single init is fine (init, init, ...).<br>
(This is of course not a contract by any means.)<br>
<br>
Have fun.<br>
<br>
[1] Not really true.<br>
<br>
--<br>
Kokan<br>
<br>
On Tue, Jan 07, 2020 at 02:35:43PM +0530, Nitish Saboo wrote:<br>
> CAUTION: This email originated from outside of the organization. Do not follow guidance, click links, or open attachments unless you recognize the sender and know the content is safe.<br>
> <br>
> Hi,<br>
> <br>
> I am using syslog-ng version 3.6.2:<br>
> <br>
> syslog-ng --version<br>
> syslog-ng 3.6.2<br>
> Installer-Version: 3.6.2<br>
> Revision:<br>
> Compile-Date: May 2 2019 01:20:32<br>
> Available-Modules: afmongodb,afstomp,afuser,csvparser,afprog,afamqp,cryptofuncs,afsocket,graphite,dbparser,affile,syslogformat,afsocket-notls,system-source,confgen,pseudofile,linux-kmsg-format,basicfuncs<br>
> Enable-Debug: off<br>
> Enable-GProf: off<br>
> Enable-Memtrace: off<br>
> Enable-IPv6: on<br>
> Enable-Spoof-Source: off<br>
> Enable-TCP-Wrapper: off<br>
> Enable-Linux-Caps: off<br>
> <br>
> <br>
> I initialised syslog-ng with the following configuration:<br>
> <br>
> int initialize_engine(const gchar* filename, const gchar* _module_path)<br>
> {<br>
> module_path = _module_path;<br>
> app_startup();<br>
> msg_init(TRUE);<br>
> <br>
> configuration = cfg_new(0x0302);<br>
> plugin_load_module("basicfuncs", configuration, NULL);<br>
> plugin_load_module("syslogformat", configuration, NULL);<br>
> <br>
> pattern_db_global_init();<br>
> return 0;<br>
> }<br>
> <br>
> int reload_pattern_db(const gchar* filename, key_value_cb cb)<br>
> {<br>
> if(patterndb!= NULL){<br>
> pattern_db_free(patterndb);<br>
> }<br>
> patterndb = pattern_db_new();<br>
> pattern_db_reload_ruleset(patterndb, configuration, filename);<br>
> pattern_db_set_emit_func(patterndb, pdbtool_pdb_emit_accumulate, cb);<br>
> return 0;<br>
> }<br>
> <br>
> When I reload the syslog-ng, the memory keeps on increasing.Looks like syslog-ng is not freeing up the memory.It continues to occupy the memory unless the process is restarted.<br>
> Just want to understand if it is a sign of memory leak or default behavior of syslog-ng to not free-up memory during reload ?<br>
> <br>
> Thanks,<br>
> Nitish<br>
> <br>
<br>
> ______________________________________________________________________________<br>
> Member info: <a href="https://nam05.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.balabit.hu%2Fmailman%2Flistinfo%2Fsyslog-ng&data=02%7C01%7Cpeter.kokai%40oneidentity.com%7C787bcb55bf024c043c6308d79350d004%7C91c369b51c9e439c989c1867ec606603%7C0%7C0%7C637139847625536021&sdata=siZ9%2BKokl%2FvxLmhWsuoL4PZcmzGFKk7PNW3vhfEQMRg%3D&reserved=0" rel="noreferrer" target="_blank">https://nam05.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.balabit.hu%2Fmailman%2Flistinfo%2Fsyslog-ng&data=02%7C01%7Cpeter.kokai%40oneidentity.com%7C787bcb55bf024c043c6308d79350d004%7C91c369b51c9e439c989c1867ec606603%7C0%7C0%7C637139847625536021&sdata=siZ9%2BKokl%2FvxLmhWsuoL4PZcmzGFKk7PNW3vhfEQMRg%3D&reserved=0</a><br>
> Documentation: <a href="https://nam05.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fsupport%2Fdocumentation%2F%3Fproduct%3Dsyslog-ng&data=02%7C01%7Cpeter.kokai%40oneidentity.com%7C787bcb55bf024c043c6308d79350d004%7C91c369b51c9e439c989c1867ec606603%7C0%7C0%7C637139847625536021&sdata=%2FdBMM4ZmjzNQI2NYt8T3s3WXx1XTd7QtGmVjEvLDBA0%3D&reserved=0" rel="noreferrer" target="_blank">https://nam05.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fsupport%2Fdocumentation%2F%3Fproduct%3Dsyslog-ng&data=02%7C01%7Cpeter.kokai%40oneidentity.com%7C787bcb55bf024c043c6308d79350d004%7C91c369b51c9e439c989c1867ec606603%7C0%7C0%7C637139847625536021&sdata=%2FdBMM4ZmjzNQI2NYt8T3s3WXx1XTd7QtGmVjEvLDBA0%3D&reserved=0</a><br>
> FAQ: <a href="https://nam05.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fwiki%2Fsyslog-ng-faq&data=02%7C01%7Cpeter.kokai%40oneidentity.com%7C787bcb55bf024c043c6308d79350d004%7C91c369b51c9e439c989c1867ec606603%7C0%7C0%7C637139847625536021&sdata=JcZw66v9GCGWFly1RKTeCdgVaHzOaL1oXenDCQIpLqs%3D&reserved=0" rel="noreferrer" target="_blank">https://nam05.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fwiki%2Fsyslog-ng-faq&data=02%7C01%7Cpeter.kokai%40oneidentity.com%7C787bcb55bf024c043c6308d79350d004%7C91c369b51c9e439c989c1867ec606603%7C0%7C0%7C637139847625536021&sdata=JcZw66v9GCGWFly1RKTeCdgVaHzOaL1oXenDCQIpLqs%3D&reserved=0</a><br>
> <br>
______________________________________________________________________________<br>
Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" rel="noreferrer" target="_blank">https://lists.balabit.hu/mailman/listinfo/syslog-ng</a><br>
Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" rel="noreferrer" target="_blank">http://www.balabit.com/support/documentation/?product=syslog-ng</a><br>
FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" rel="noreferrer" target="_blank">http://www.balabit.com/wiki/syslog-ng-faq</a><br>
<br>
</blockquote></div></div>
</blockquote></div></div>