[syslog-ng] allowed concurrent connections - bug?

[Laszlo Varady (lvarady)]

Hi,

Do you have flags(flow-control) specified in your log paths?
If so, a dead destination in such log paths might cause the mentioned issue.

When flow-control is activated, the corresponding sources will be suspended. This suspended state does not even allow syslog-ng to truly release connections that have been closed by the clients.
This is actually more of expected behavior as we don't want to allow new connections in situations where logs could not be delivered anyway.

Please check the queued statistic counters of "syslog-ng-ctl stats" to see whether this is the case.

In case of anything else, I would suspect a bug.

--
László Várady
________________________________
From: syslog-ng <syslog-ng-bounces at lists.balabit.hu> on behalf of Evan Rempel <erempel at uvic.ca>
Sent: Thursday, February 17, 2022 19:01
To: syslog-ng at lists.balabit.hu <syslog-ng at lists.balabit.hu>
Subject: [syslog-ng] allowed concurrent connections - bug?

CAUTION: This email originated from outside of the organization. Do not follow guidance, click links, or open attachments unless you recognize the sender and know the content is safe.


I am having an issue that is a little difficult to reproduce so I wanted
some input from others.

I have a syslog-ng 3.35.1 that has a TLS source defined with
max-connections(10000)

After some time the server starts logging a lot of messages

syslog-ng[12802]: Number of allowed concurrent connections reached,
rejecting connection; client='AF_INET(XXXX:61062)',
local='AF_INET(YYYY:6514)', group_name='client_network_tcp',
location='/etc/syslog-ng/syslog-ng.server.conf:61:9', max='10000'

To the best of my ability I can only find about 2500 actual connections.

Both lsof and netstat report around the 2500 connections.

I had to restart syslog-ng to stop this situation.

Has anyone seen this behavior before?

I get a lot of TLS connections without a certificate.

Error reading RFC6587 style framed data

Pperhaps the counters are not decremented for those timed out connections?

--
Evan Rempel

______________________________________________________________________________
Member info: https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.balabit.hu%2Fmailman%2Flistinfo%2Fsyslog-ng&data=04%7C01%7Claszlo.varady%40oneidentity.com%7C4f9e5678c2bb4140645c08d9f23f840c%7C91c369b51c9e439c989c1867ec606603%7C0%7C1%7C637807176916131984%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=zbW9OAsb8C1JcXGfhEOlpb1Iq8OpeMQB9BPEBPTgGHg%3D&reserved=0
Documentation: https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fsupport%2Fdocumentation%2F%3Fproduct%3Dsyslog-ng&data=04%7C01%7Claszlo.varady%40oneidentity.com%7C4f9e5678c2bb4140645c08d9f23f840c%7C91c369b51c9e439c989c1867ec606603%7C0%7C1%7C637807176916131984%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=D9Kx3H70ocShwCDySAitI0Yzai5%2F3WrpFipQgGZbZeQ%3D&reserved=0
FAQ: https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fwiki%2Fsyslog-ng-faq&data=04%7C01%7Claszlo.varady%40oneidentity.com%7C4f9e5678c2bb4140645c08d9f23f840c%7C91c369b51c9e439c989c1867ec606603%7C0%7C1%7C637807176916131984%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=GjZ6x6YgEXmZYXNRuMKqf60Vg3t5oG5n5P%2FFUyCV4Dk%3D&reserved=0

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20220217/6173b6d6/attachment-0001.htm>