[syslog-ng] Memory not freeing up in syslog-ng-3.6.2 during reload
Peter Kokai (pkokai)
Peter.Kokai at oneidentity.com
Mon Jan 13 10:00:34 UTC 2020
Hello,
> Can you please share the commit_id/link for this fix so that I can review the fix or try to backport the fix ?
I cannot, I do not have commit hash nor link for those fixes.
I would check out the git history for those fixes via:
```
$ git log syslog-ng-3.6.2...master -- modules/dbparser/ | grep -c commit
399
```
There are a few, but possibly commit message and diff could help to filter it down to something fewer.
Idea1: you could check if the upstream solved, as maybe it is not solved and your check out 400 commit for nothing
Idea2: you could use git bisect to automate the process of finding the commit fixing your issue (https://lwn.net/Articles/317154/)
--
Kokan
On Mon, Jan 13, 2020 at 03:14:26PM +0530, Nitish Saboo wrote:
> CAUTION: This email originated from outside of the organization. Do not follow guidance, click links, or open attachments unless you recognize the sender and know the content is safe.
>
> Hi Kokan,
>
> >And/or possibly you could check possible memory leak fixes done since 3.6.2 for dbparser and backport those fixes.
>
> I got this bug fixes description 'Fixed memleak in db-parser which could happen at every reload' in one the fixes from the following link 'https://nam05.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fsyslog-ng%2Fsyslog-ng%2Freleases%3Fafter%3Dsyslog-ng-3.8.1&data=02%7C01%7CPeter.Kokai%40oneidentity.com%7Ceeabf2c6608444a1ecd608d7980d369a%7C91c369b51c9e439c989c1867ec606603%7C0%7C0%7C637145054868725961&sdata=MKPubvcrqPCcHibTFhMBeA4EciEkEUvvhfnObO0I%2FlI%3D&reserved=0'.
> Looks like there is a fix related to memleak in dbparser in version 'syslog-ng-3.7.1'. Am I going in the right direction ? Can this be the root cause of the issue ?
>
> Can you please share the commit_id/link for this fix so that I can review the fix or try to backport the fix ?
>
> Thanks,
> Nitish
>
>
> On Fri, Jan 10, 2020 at 1:13 PM Peter Kokai (pkokai) <Peter.Kokai at oneidentity.com<mailto:Peter.Kokai at oneidentity.com>> wrote:
> Hello,
>
> > Can we use valgrind for Go and C combination (CGO) ?
>
> I never used valgrind on executable that was created from golang.
> But valgrind should work with any executable.
>
> If you still have consern regarding using valgrind with golang+c integration, why not create some test for your C wrapper and do the memory check on that test ?
> You can write that test in C.
>
> And/or possibly you could check possible memory leak fixes done since 3.6.2 for dbparser and backport those fixes.
>
> --
> Kokan
>
> On Thu, Jan 09, 2020 at 03:12:03PM +0530, Nitish Saboo wrote:
> > CAUTION: This email originated from outside of the organization. Do not follow guidance, click links, or open attachments unless you recognize the sender and know the content is safe.
> >
> > Hi Kokan,
> >
> > >You should destroy, free patterndb via this function. This destroy context, and unreference LogMessage it holds a reference to.
> >
> > From the following link, I am using the same 'pattern_db_free' function.
> > https://nam05.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fsyslog-ng%2Fsyslog-ng%2Fblob%2Fc220d1364ce338725097dabd171819660530018f%2Fmodules%2Fdbparser%2Fpatterndb.c%23L806-L815&data=02%7C01%7CPeter.Kokai%40oneidentity.com%7Ceeabf2c6608444a1ecd608d7980d369a%7C91c369b51c9e439c989c1867ec606603%7C0%7C0%7C637145054868735953&sdata=1WdNnksUeRonBAnwQs3wuuwkdDzt7R7TQyAZ6dUVX9o%3D&reserved=0
> >
> > As you said 'pattern_db_free' function first destroys and then free patterndb, it also destroys context... make sense.
> >
> > >As LogMessages are reference counted ref/unref, it is possible that somebody still holds a reference to your LogMessages.
> >
> > void match(const gchar *pattern, size_t len, const gchar *program, size_t p_len)
> > {
> > LogMessage *msg = log_msg_new_empty();
> > PDBInput in;
> >
> > log_msg_set_value(msg, LM_V_MESSAGE, pattern, len);
> > log_msg_set_value(msg, LM_V_PROGRAM, program, p_len);
> > pattern_db_process(patterndb, PDB_INPUT_WRAP_MESSAGE(&in, msg));
> > log_msg_unref(msg);
> > }
> >
> > I am removing the reference to the the LogMessage via this function after parsing each one of them.It doesn't look like I am keeping any reference to the LogMessage atleast from the C code.
> > I am calling these C methods from Go.
> >
> > Any other thought that comes to your mind that points to increase in memory usage ONLY at the time of reload of patterndb ?
> >
> > >Valgrind is your friend, use it :)
> >
> > Can we use valgrind for Go and C combination (CGO) ?
> >
> > Thanks,
> > Nitish
> >
> >
> > On Thu, Jan 9, 2020 at 12:17 PM Peter Kokai (pkokai) <Peter.Kokai at oneidentity.com<mailto:Peter.Kokai at oneidentity.com><mailto:Peter.Kokai at oneidentity.com<mailto:Peter.Kokai at oneidentity.com>>> wrote:
> > Hello,
> >
> > > Is this the correct way to free up the existing patterndb instance ?
> > > pattern_db_free(patterndb);
> > You should destroy, free patterndb via this function. This destroy context, and unreference LogMessage it holds a reference to.
> > As LogMessages are reference counted ref/unref, it is possible that somebody still holds a reference to your LogMessages.
> >
> > > Is this because the message and context are not getting deleted?
> > I do not think so.
> >
> > Valgrind is your friend, use it :)
> >
> > --
> > Kokan
> >
> > On Wed, Jan 08, 2020 at 05:36:10PM +0530, Nitish Saboo wrote:
> > > CAUTION: This email originated from outside of the organization. Do not follow guidance, click links, or open attachments unless you recognize the sender and know the content is safe.
> > >
> > > Hi Kokan,
> > >
> > > >Please first of all correct me if I am wrong. But my assumption is that you build an application using one of the library in syslog-ng that provides the patterndb functionality.
> > >
> > > Yes, I have built a service with some of the libraries in syslog-ng that provides the patterndb functionality.I am using the following method to initialise syslog-ng and patterndb:
> > >
> > > PatternDB *patterndb;
> > >
> > > int initialize_engine(const gchar* filename, const gchar* _module_path)
> > > {
> > > module_path = _module_path;
> > > app_startup();
> > > msg_init(TRUE);
> > >
> > > configuration = cfg_new(0x0302);
> > > plugin_load_module("basicfuncs", configuration, NULL);
> > > plugin_load_module("syslogformat", configuration, NULL);
> > >
> > > pattern_db_global_init();
> > > return 0;
> > > }
> > >
> > > >In case of syslog-ng the LogDBParser own's a PatternDB. In case of reload the LogDBParser are destroyed (deinit, free) and the PatternDB is stored in a kinda map.
> > > When the new LogDBParser is created, it looks up that map if there is a PatternDB in it and if there is it uses the PatternDB (which holds the context and messages).
> > >
> > > I am reloading the patterndb using the following method, where I am initially freeing up the existing patterndb instance using 'pattern_db_free' method.Will this free-up the context and messages ?
> > > Is this the correct way to free up the existing patterndb instance ?
> > >
> > > int reload_pattern_db(const gchar* filename, key_value_cb cb)
> > > {
> > > if(patterndb!= NULL){
> > > pattern_db_free(patterndb);
> > > }
> > > patterndb = pattern_db_new();
> > > pattern_db_reload_ruleset(patterndb, configuration, filename);
> > > pattern_db_set_emit_func(patterndb, pdbtool_pdb_emit_accumulate, cb);
> > > return 0;
> > > }
> > >
> > > Let me clarify my issue to you and hope then you will be able to guide me correctly.
> > >
> > > ISSUE:
> > >
> > > 1) Suppose the patterndb.xml file size is 10MB. When I start the service the memory usage shows 2% for 10MB file size.
> > >
> > > 2)With every reload of patterndb(I am increasing the file size of patterndb.xml for example from 10MB to 11MB), the memory usage keeps on increasing like 2% to 3%.The increase in memory usage is seen ONLY during reload and not while parsing.This process is repeated and now the file size is 20 MB and memory usage has shot up to say 8%.
> > >
> > > 3)But now when I restart the service, the memory usage for the same patterndb.xml file of size 20MB shows only 4%.
> > >
> > > So my concern is why the memory usage is increasing with each reload of patterndb.xml file but when I restart the service the same size of patterndb.xml shows less memory usage.
> > > Is this because the message and context are not getting deleted?
> > >
> > > Thanks,
> > > Nitish
> > >
> > >
> > >
> > >
> > > On Wed, Jan 8, 2020 at 4:28 PM Peter Kokai (pkokai) <Peter.Kokai at oneidentity.com<mailto:Peter.Kokai at oneidentity.com><mailto:Peter.Kokai at oneidentity.com<mailto:Peter.Kokai at oneidentity.com>><mailto:Peter.Kokai at oneidentity.com<mailto:Peter.Kokai at oneidentity.com><mailto:Peter.Kokai at oneidentity.com<mailto:Peter.Kokai at oneidentity.com>>>> wrote:
> > > Hello,
> > >
> > > Please first of all correct me if I am wrong. But my assumption is that you build an application using one of the library in syslog-ng that provides the patterndb functionality.
> > >
> > > In that case reading the syslog-ng administrators guide is not the most useful thing to read, well there is no real development documentation focusing on patterndb.
> > >
> > >
> > > > 1) Is this because of the message contexts that are persisted on reload, the memory is not freeing up at the time of reload of patterndb ?
> > >
> > > In case of syslog-ng the LogDBParser own's a PatternDB. In case of reload the LogDBParser are destroyed (deinit, free) and the PatternDB is stored in a kinda map.
> > > When the new LogDBParser is created, it looks up that map if there is a PatternDB in it and if there is it uses the PatternDB (which holds the context and messages).
> > >
> > > > 2) Is there a way we can free-up these message contexts without a restart ?
> > >
> > > Probably there is. A proper free of PatternDB object should suffiece. Try that and if not, check out what is leaked.
> > > Please not that we do not update old releases like your 3.6.2 version. Even if you found a valid leak, it won't be fixed by us.
> > >
> > > On the other hand if that same issue applies to upstream, that is going to be solved sooner or later on upstream so reporting that might be beneficial.
> > >
> > >
> > > --
> > > Kokan
> > >
> > > On Wed, Jan 08, 2020 at 11:11:20AM +0530, Nitish Saboo wrote:
> > > > CAUTION: This email originated from outside of the organization. Do not follow guidance, click links, or open attachments unless you recognize the sender and know the content is safe.
> > > >
> > > > Hi,
> > > >
> > > > I was going through syslog-ng administrators guide and came across the following point from the given link:
> > > >
> > > > "NOTE:Message contexts are persistent and are not lost when syslog-ng OSE is reloaded (SIGHUP), but are lost when syslog-ng OSE is restarted."
> > > >
> > > > https://nam05.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.syslog-ng.com%2Ftechnical-documents%2Fdoc%2Fsyslog-ng-open-source-edition%2F3.18%2Fadministration-guide%2F72%23TOPIC-1044162&data=02%7C01%7CPeter.Kokai%40oneidentity.com%7Ceeabf2c6608444a1ecd608d7980d369a%7C91c369b51c9e439c989c1867ec606603%7C0%7C0%7C637145054868735953&sdata=uIFz5H3Ru4SSxschq88wKSLiyt1Pen2%2FcAbM95BUnHc%3D&reserved=0
> > > >
> > > > 1) Is this because of the message contexts that are persisted on reload, the memory is not freeing up at the time of reload of patterndb ?
> > > >
> > > > 2) Is there a way we can free-up these message contexts without a restart ?
> > > >
> > > > Please correct me if I am wrong or missing something here.
> > > >
> > > > Thanks,
> > > > Nitish
> > > >
> > > >
> > > >
> > > > On Tue, Jan 7, 2020 at 5:37 PM Nitish Saboo <nitish.saboo55 at gmail.com<mailto:nitish.saboo55 at gmail.com><mailto:nitish.saboo55 at gmail.com<mailto:nitish.saboo55 at gmail.com>><mailto:nitish.saboo55 at gmail.com<mailto:nitish.saboo55 at gmail.com><mailto:nitish.saboo55 at gmail.com<mailto:nitish.saboo55 at gmail.com>>><mailto:nitish.saboo55 at gmail.com<mailto:nitish.saboo55 at gmail.com><mailto:nitish.saboo55 at gmail.com<mailto:nitish.saboo55 at gmail.com>><mailto:nitish.saboo55 at gmail.com<mailto:nitish.saboo55 at gmail.com><mailto:nitish.saboo55 at gmail.com<mailto:nitish.saboo55 at gmail.com>>>>> wrote:
> > > > Hi Peter,
> > > >
> > > > Thanks for your response :)
> > > > I am seeing the memory usage getting increased ONLY at the time of reload of patterndb.
> > > > All other operation are having constant memory consumption.
> > > > Hence, my question is specific to reload operation if it is by default that patterndb component is not freeing up the memory on reload operation ? If it is by default, why is it so ?
> > > >
> > > > The init and deinit can be repeated multiple times (init, deinit, init, deinit, ...) also usually the repeation of single init is fine (init, init, ...).
> > > > >> With this you meant initialising syslog-ng every-time for every reload operation?
> > > >
> > > > Thanks,
> > > > Nitish
> > > >
> > > > On Tue, Jan 7, 2020 at 5:05 PM Peter Kokai (pkokai) <Peter.Kokai at oneidentity.com<mailto:Peter.Kokai at oneidentity.com><mailto:Peter.Kokai at oneidentity.com<mailto:Peter.Kokai at oneidentity.com>><mailto:Peter.Kokai at oneidentity.com<mailto:Peter.Kokai at oneidentity.com><mailto:Peter.Kokai at oneidentity.com<mailto:Peter.Kokai at oneidentity.com>>><mailto:Peter.Kokai at oneidentity.com<mailto:Peter.Kokai at oneidentity.com><mailto:Peter.Kokai at oneidentity.com<mailto:Peter.Kokai at oneidentity.com>><mailto:Peter.Kokai at oneidentity.com<mailto:Peter.Kokai at oneidentity.com><mailto:Peter.Kokai at oneidentity.com<mailto:Peter.Kokai at oneidentity.com>>>>> wrote:
> > > > Hello,
> > > >
> > > > Yes, we like to keep memory for ourselfs ;)[1]
> > > >
> > > > This is in fact not syslog-ng reload, and mostly about patterndb component. You could verify leak with valgrind or sanitizer. Please if you find a leak check it on upstream and report if you have any.
> > > >
> > > > Regarding the question that your application does leak or not is hard to tell withouth knowing it.
> > > > Please share your application and we might be able to help you.
> > > >
> > > > Just a hint: things in syslog-ng usually like to be called like this:
> > > >
> > > > new
> > > > init
> > > > deinit
> > > > free
> > > >
> > > > In case of reference counted object also use unref instead of free, and ref well to reference stuff.
> > > >
> > > > The init and deinit can be repeated multiple times (init, deinit, init, deinit, ...) also usually the repeation of single init is fine (init, init, ...).
> > > > (This is of course not a contract by any means.)
> > > >
> > > > Have fun.
> > > >
> > > > [1] Not really true.
> > > >
> > > > --
> > > > Kokan
> > > >
> > > > On Tue, Jan 07, 2020 at 02:35:43PM +0530, Nitish Saboo wrote:
> > > > > CAUTION: This email originated from outside of the organization. Do not follow guidance, click links, or open attachments unless you recognize the sender and know the content is safe.
> > > > >
> > > > > Hi,
> > > > >
> > > > > I am using syslog-ng version 3.6.2:
> > > > >
> > > > > syslog-ng --version
> > > > > syslog-ng 3.6.2
> > > > > Installer-Version: 3.6.2
> > > > > Revision:
> > > > > Compile-Date: May 2 2019 01:20:32
> > > > > Available-Modules: afmongodb,afstomp,afuser,csvparser,afprog,afamqp,cryptofuncs,afsocket,graphite,dbparser,affile,syslogformat,afsocket-notls,system-source,confgen,pseudofile,linux-kmsg-format,basicfuncs
> > > > > Enable-Debug: off
> > > > > Enable-GProf: off
> > > > > Enable-Memtrace: off
> > > > > Enable-IPv6: on
> > > > > Enable-Spoof-Source: off
> > > > > Enable-TCP-Wrapper: off
> > > > > Enable-Linux-Caps: off
> > > > >
> > > > >
> > > > > I initialised syslog-ng with the following configuration:
> > > > >
> > > > > int initialize_engine(const gchar* filename, const gchar* _module_path)
> > > > > {
> > > > > module_path = _module_path;
> > > > > app_startup();
> > > > > msg_init(TRUE);
> > > > >
> > > > > configuration = cfg_new(0x0302);
> > > > > plugin_load_module("basicfuncs", configuration, NULL);
> > > > > plugin_load_module("syslogformat", configuration, NULL);
> > > > >
> > > > > pattern_db_global_init();
> > > > > return 0;
> > > > > }
> > > > >
> > > > > int reload_pattern_db(const gchar* filename, key_value_cb cb)
> > > > > {
> > > > > if(patterndb!= NULL){
> > > > > pattern_db_free(patterndb);
> > > > > }
> > > > > patterndb = pattern_db_new();
> > > > > pattern_db_reload_ruleset(patterndb, configuration, filename);
> > > > > pattern_db_set_emit_func(patterndb, pdbtool_pdb_emit_accumulate, cb);
> > > > > return 0;
> > > > > }
> > > > >
> > > > > When I reload the syslog-ng, the memory keeps on increasing.Looks like syslog-ng is not freeing up the memory.It continues to occupy the memory unless the process is restarted.
> > > > > Just want to understand if it is a sign of memory leak or default behavior of syslog-ng to not free-up memory during reload ?
> > > > >
> > > > > Thanks,
> > > > > Nitish
> > > > >
> > > >
> > > > > ______________________________________________________________________________
> > > > > Member info: https://nam05.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.balabit.hu%2Fmailman%2Flistinfo%2Fsyslog-ng&data=02%7C01%7CPeter.Kokai%40oneidentity.com%7Ceeabf2c6608444a1ecd608d7980d369a%7C91c369b51c9e439c989c1867ec606603%7C0%7C0%7C637145054868735953&sdata=VpgivcnuQpzFJ5IwT5ELnqPEMXOeDmUBRfAxc%2FeBvXM%3D&reserved=0
> > > > > Documentation: https://nam05.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fsupport%2Fdocumentation%2F%3Fproduct%3Dsyslog-ng&data=02%7C01%7CPeter.Kokai%40oneidentity.com%7Ceeabf2c6608444a1ecd608d7980d369a%7C91c369b51c9e439c989c1867ec606603%7C0%7C0%7C637145054868735953&sdata=srxlQTClrZTb2L6wQglgjv7gEY54paPg7U2Fhw79Bmw%3D&reserved=0
> > > > > FAQ: https://nam05.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fwiki%2Fsyslog-ng-faq&data=02%7C01%7CPeter.Kokai%40oneidentity.com%7Ceeabf2c6608444a1ecd608d7980d369a%7C91c369b51c9e439c989c1867ec606603%7C0%7C0%7C637145054868735953&sdata=HLoVlxR5cDYuRne5IQR5tQqPD2LWm9V9shun3ct6t84%3D&reserved=0
> > > > >
> > > > ______________________________________________________________________________
> > > > Member info: https://nam05.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.balabit.hu%2Fmailman%2Flistinfo%2Fsyslog-ng&data=02%7C01%7CPeter.Kokai%40oneidentity.com%7Ceeabf2c6608444a1ecd608d7980d369a%7C91c369b51c9e439c989c1867ec606603%7C0%7C0%7C637145054868735953&sdata=VpgivcnuQpzFJ5IwT5ELnqPEMXOeDmUBRfAxc%2FeBvXM%3D&reserved=0
> > > > Documentation: https://nam05.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fsupport%2Fdocumentation%2F%3Fproduct%3Dsyslog-ng&data=02%7C01%7CPeter.Kokai%40oneidentity.com%7Ceeabf2c6608444a1ecd608d7980d369a%7C91c369b51c9e439c989c1867ec606603%7C0%7C0%7C637145054868735953&sdata=srxlQTClrZTb2L6wQglgjv7gEY54paPg7U2Fhw79Bmw%3D&reserved=0
> > > > FAQ: https://nam05.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fwiki%2Fsyslog-ng-faq&data=02%7C01%7CPeter.Kokai%40oneidentity.com%7Ceeabf2c6608444a1ecd608d7980d369a%7C91c369b51c9e439c989c1867ec606603%7C0%7C0%7C637145054868745942&sdata=GnbcivlnQJ753qZyi53xTUD5DHoUcBiGR1pIrSBX2%2BI%3D&reserved=0
> > > >
> > >
> > > > ______________________________________________________________________________
> > > > Member info: https://nam05.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.balabit.hu%2Fmailman%2Flistinfo%2Fsyslog-ng&data=02%7C01%7CPeter.Kokai%40oneidentity.com%7Ceeabf2c6608444a1ecd608d7980d369a%7C91c369b51c9e439c989c1867ec606603%7C0%7C0%7C637145054868745942&sdata=n%2FHLK9gyQW3VJMbx83%2F%2FbvDgKoLo4vklR%2FgKharJcEI%3D&reserved=0
> > > > Documentation: https://nam05.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fsupport%2Fdocumentation%2F%3Fproduct%3Dsyslog-ng&data=02%7C01%7CPeter.Kokai%40oneidentity.com%7Ceeabf2c6608444a1ecd608d7980d369a%7C91c369b51c9e439c989c1867ec606603%7C0%7C0%7C637145054868745942&sdata=3f02TLfIQ%2Bcs7XftcFB9PzheWSytMSzcyMGu3s5SkYo%3D&reserved=0
> > > > FAQ: https://nam05.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fwiki%2Fsyslog-ng-faq&data=02%7C01%7CPeter.Kokai%40oneidentity.com%7Ceeabf2c6608444a1ecd608d7980d369a%7C91c369b51c9e439c989c1867ec606603%7C0%7C0%7C637145054868745942&sdata=GnbcivlnQJ753qZyi53xTUD5DHoUcBiGR1pIrSBX2%2BI%3D&reserved=0
> > > >
> > > ______________________________________________________________________________
> > > Member info: https://nam05.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.balabit.hu%2Fmailman%2Flistinfo%2Fsyslog-ng&data=02%7C01%7CPeter.Kokai%40oneidentity.com%7Ceeabf2c6608444a1ecd608d7980d369a%7C91c369b51c9e439c989c1867ec606603%7C0%7C0%7C637145054868745942&sdata=n%2FHLK9gyQW3VJMbx83%2F%2FbvDgKoLo4vklR%2FgKharJcEI%3D&reserved=0
> > > Documentation: https://nam05.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fsupport%2Fdocumentation%2F%3Fproduct%3Dsyslog-ng&data=02%7C01%7CPeter.Kokai%40oneidentity.com%7Ceeabf2c6608444a1ecd608d7980d369a%7C91c369b51c9e439c989c1867ec606603%7C0%7C0%7C637145054868745942&sdata=3f02TLfIQ%2Bcs7XftcFB9PzheWSytMSzcyMGu3s5SkYo%3D&reserved=0
> > > FAQ: https://nam05.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fwiki%2Fsyslog-ng-faq&data=02%7C01%7CPeter.Kokai%40oneidentity.com%7Ceeabf2c6608444a1ecd608d7980d369a%7C91c369b51c9e439c989c1867ec606603%7C0%7C0%7C637145054868745942&sdata=GnbcivlnQJ753qZyi53xTUD5DHoUcBiGR1pIrSBX2%2BI%3D&reserved=0
> > >
> >
> > > ______________________________________________________________________________
> > > Member info: https://nam05.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.balabit.hu%2Fmailman%2Flistinfo%2Fsyslog-ng&data=02%7C01%7CPeter.Kokai%40oneidentity.com%7Ceeabf2c6608444a1ecd608d7980d369a%7C91c369b51c9e439c989c1867ec606603%7C0%7C0%7C637145054868745942&sdata=n%2FHLK9gyQW3VJMbx83%2F%2FbvDgKoLo4vklR%2FgKharJcEI%3D&reserved=0
> > > Documentation: https://nam05.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fsupport%2Fdocumentation%2F%3Fproduct%3Dsyslog-ng&data=02%7C01%7CPeter.Kokai%40oneidentity.com%7Ceeabf2c6608444a1ecd608d7980d369a%7C91c369b51c9e439c989c1867ec606603%7C0%7C0%7C637145054868755938&sdata=m2vQ127GvfiX7Sd93g7gpayYdUanOXibTCffNF2%2FJ3s%3D&reserved=0
> > > FAQ: https://nam05.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fwiki%2Fsyslog-ng-faq&data=02%7C01%7CPeter.Kokai%40oneidentity.com%7Ceeabf2c6608444a1ecd608d7980d369a%7C91c369b51c9e439c989c1867ec606603%7C0%7C0%7C637145054868755938&sdata=Xb5F8gdAmaFJXCG3nfNwpOh7K626lyOKzTiLC%2Brq13k%3D&reserved=0
> > >
> > ______________________________________________________________________________
> > Member info: https://nam05.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.balabit.hu%2Fmailman%2Flistinfo%2Fsyslog-ng&data=02%7C01%7CPeter.Kokai%40oneidentity.com%7Ceeabf2c6608444a1ecd608d7980d369a%7C91c369b51c9e439c989c1867ec606603%7C0%7C0%7C637145054868755938&sdata=EH4rxSwgZrkPVBqMn01uxbaCVvh1yaEHvNYhWFqhIDM%3D&reserved=0
> > Documentation: https://nam05.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fsupport%2Fdocumentation%2F%3Fproduct%3Dsyslog-ng&data=02%7C01%7CPeter.Kokai%40oneidentity.com%7Ceeabf2c6608444a1ecd608d7980d369a%7C91c369b51c9e439c989c1867ec606603%7C0%7C0%7C637145054868755938&sdata=m2vQ127GvfiX7Sd93g7gpayYdUanOXibTCffNF2%2FJ3s%3D&reserved=0
> > FAQ: https://nam05.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fwiki%2Fsyslog-ng-faq&data=02%7C01%7CPeter.Kokai%40oneidentity.com%7Ceeabf2c6608444a1ecd608d7980d369a%7C91c369b51c9e439c989c1867ec606603%7C0%7C0%7C637145054868755938&sdata=Xb5F8gdAmaFJXCG3nfNwpOh7K626lyOKzTiLC%2Brq13k%3D&reserved=0
> >
>
> > ______________________________________________________________________________
> > Member info: https://nam05.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.balabit.hu%2Fmailman%2Flistinfo%2Fsyslog-ng&data=02%7C01%7CPeter.Kokai%40oneidentity.com%7Ceeabf2c6608444a1ecd608d7980d369a%7C91c369b51c9e439c989c1867ec606603%7C0%7C0%7C637145054868755938&sdata=EH4rxSwgZrkPVBqMn01uxbaCVvh1yaEHvNYhWFqhIDM%3D&reserved=0
> > Documentation: https://nam05.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fsupport%2Fdocumentation%2F%3Fproduct%3Dsyslog-ng&data=02%7C01%7CPeter.Kokai%40oneidentity.com%7Ceeabf2c6608444a1ecd608d7980d369a%7C91c369b51c9e439c989c1867ec606603%7C0%7C0%7C637145054868755938&sdata=m2vQ127GvfiX7Sd93g7gpayYdUanOXibTCffNF2%2FJ3s%3D&reserved=0
> > FAQ: https://nam05.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fwiki%2Fsyslog-ng-faq&data=02%7C01%7CPeter.Kokai%40oneidentity.com%7Ceeabf2c6608444a1ecd608d7980d369a%7C91c369b51c9e439c989c1867ec606603%7C0%7C0%7C637145054868755938&sdata=Xb5F8gdAmaFJXCG3nfNwpOh7K626lyOKzTiLC%2Brq13k%3D&reserved=0
> >
> ______________________________________________________________________________
> Member info: https://nam05.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.balabit.hu%2Fmailman%2Flistinfo%2Fsyslog-ng&data=02%7C01%7CPeter.Kokai%40oneidentity.com%7Ceeabf2c6608444a1ecd608d7980d369a%7C91c369b51c9e439c989c1867ec606603%7C0%7C0%7C637145054868765934&sdata=6eAIpR9ieh4M4as3RSbXYYLX%2BtwGHz%2BoZyECRuJncPw%3D&reserved=0
> Documentation: https://nam05.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fsupport%2Fdocumentation%2F%3Fproduct%3Dsyslog-ng&data=02%7C01%7CPeter.Kokai%40oneidentity.com%7Ceeabf2c6608444a1ecd608d7980d369a%7C91c369b51c9e439c989c1867ec606603%7C0%7C0%7C637145054868765934&sdata=KpTCMcME6APsKL5OVTgYECC2plNn9mPKteFoG4aT2YA%3D&reserved=0
> FAQ: https://nam05.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fwiki%2Fsyslog-ng-faq&data=02%7C01%7CPeter.Kokai%40oneidentity.com%7Ceeabf2c6608444a1ecd608d7980d369a%7C91c369b51c9e439c989c1867ec606603%7C0%7C0%7C637145054868765934&sdata=zmyJKI1u6cpS%2BG1J42GqRgkp%2BFgUIem%2FfFGC33upfJI%3D&reserved=0
>
> ______________________________________________________________________________
> Member info: https://nam05.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.balabit.hu%2Fmailman%2Flistinfo%2Fsyslog-ng&data=02%7C01%7CPeter.Kokai%40oneidentity.com%7Ceeabf2c6608444a1ecd608d7980d369a%7C91c369b51c9e439c989c1867ec606603%7C0%7C0%7C637145054869135772&sdata=KMMmEAWm%2FaE791i6WASS0RnFPj09BY7LgeMdHqsEIiU%3D&reserved=0
> Documentation: https://nam05.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fsupport%2Fdocumentation%2F%3Fproduct%3Dsyslog-ng&data=02%7C01%7CPeter.Kokai%40oneidentity.com%7Ceeabf2c6608444a1ecd608d7980d369a%7C91c369b51c9e439c989c1867ec606603%7C0%7C0%7C637145054869135772&sdata=HnSlPV5ZePA2K6%2BH0NtV5uOlePqhD5rY2ta0kvyTSoM%3D&reserved=0
> FAQ: https://nam05.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fwiki%2Fsyslog-ng-faq&data=02%7C01%7CPeter.Kokai%40oneidentity.com%7Ceeabf2c6608444a1ecd608d7980d369a%7C91c369b51c9e439c989c1867ec606603%7C0%7C0%7C637145054869135772&sdata=WhztD8%2FabmuE7%2FEo5%2Fe93verR85tFdK5ZBYmckNA%2Fds%3D&reserved=0
>
More information about the syslog-ng
mailing list