[syslog-ng] syslog-ng Potential Memory Leak

[Tamayo, Andres]

Hello,

No, my log setup is not including the internal source logs, it only contains the logs from a text file and they are sent out using a TCP syslog destination.

In any case, I do not think this problem is related to the logs themselves.  The memory size of the process does not increase when syslog-ng client is sending messages to the syslog-ng server.  The memory only increases if the syslog-ng server is offline and the syslog-ng client tries to reconnect over and over.

Thanks.


-----Original Message-----
From: Juhász Viktor [mailto:jviktor at balabit.hu] 
Sent: Tuesday, March 25, 2014 1:16 AM
To: Syslog-ng users' and developers' mailing list
Subject: Re: [syslog-ng] syslog-ng Potential Memory Leak

Hi,

What about the internal source? Does your configuration contain the internal source?
And if it contains, do you send the internal logs to the server?

BR,
Viktor Juhasz
 
2014 Március 19, Szerda 23:13 CET dátummal, "Tamayo, Andres" <Andres.Tamayo at viasat.com> ezt írta: 
 
> Hello,
> 
> I have been testing syslog-ng under different scenarios and I noticed that when syslog-ng is trying to connect to an external syslog-ng server, it exhibits what could be called a memory leak.
> 
> Let me explain how I am setting up my test.  I have a syslog-ng client running collecting logs from a file (source) and sending them to an external syslog-ng server over TCP (destination).  If the syslog-ng server is online, the syslog-ng client daemon process virtual memory does not increase. Now, when I kill the syslog-ng server, the syslog-ng client daemon process starts to request more and more virtual memory.
> 
> Here are the details:
> 
> I started both the syslog-ng client and server and started to send logs from the client to the server.  At this point, the syslog-ng client daemon maintained its VmSize equal to 7144 Kbytes.  This condition was stable for 4 hours while the server was online.
> 
> Then, after 4 hours, I killed the syslog-ng server.  As soon as I did that, the syslog-ng client daemon virtual memory size started to increate every few seconds.  Started at 7144 Kbytest and after 4 hours VmSize was equal to 7752 Kbytes.  So it grew 608 Kbytes over the next 4 hours.
> 
> Finally, after the 4 hours I just mentioned, I restarted the syslog-ng server.  All the logs that had not been sent for 4 hours were transmitted to the syslog-ng server in a few seconds.  At this point, the syslog-ng client daemon virtual memory stopped growing and stayed stable for 4 hours.
> 
> I think that this behavior may be related to the logs generated every time the client tries to reconnect with the server.  By the way, the client time_reopen configuration was set to 15 seconds.
> 
> I am worried that over a long long period of time, if the syslog-ng server is not online, the syslog-ng client daemon may be killed by the OS for using too much memory.
> 
> Please let me know if you have any ideas how to fix this problem, thank you.
>