Hi,

I'm using syslog-ng 3.16.1 with wildcard-file as source (let's call it "syslog-agent), which sends log messages to another syslog-ng acting as a relay.

I've noticed that syslog-agent instances RAM consumption keeps increasing until they leave no free memory in the cluster (each server has 64G of RAM). In my use case, new folders & files are created constantly under base-dir folder, but every 2 days obsolete folders & files are deleted. I assumed that syslog-ng would free some RAM every time those folders & files are deleted, but it doesn't, not even if I run a syslog-ng-ctl reload operation.

log-fifo-size value is 345000, so I assume it can't be a buffer situation since 345000 messages can't occupy 40-50 GB of memory.

I've performed the following test to reproduce the situation in small scale:

- Launch a syslog-agent with a wildcard-file source reading from "/tmp/test/" base-dir. syslog-agent RAM usage is about 125M.

- Run a simple script to create complex folder hierarchy under /tmp/test and some files with 5000 log messages to read from.

- Wait until syslog-agent RAM usage gets 1GB

- Stop script execution and wait until syslog-agent has send all logs to relay.

- Delete everything under /tmp/test and execute syslog-ng-ctl reload operation

- 24 hours after, syslog-agent RAM usage still is 1GB

I've used heaptrack tool as a try to find a memory leak in syslog-ng, you can see in the attached image that iv_list_empty function in iv_list.h file is where most of the RAM usage is.

How do I get syslog-ng to free RAM? Or is it a memory leak?

Thanks in advance.

--

| Jose Angel Santiago

Vía de las dos Castillas, 33, Ática 4, 3ª Planta

28224 Pozuelo de Alarcón, Madrid, Spain

+34 918 286 473 | www.stratio.com

______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq