If you are using UDP, understand that there is no “guarantee of delivery”. You can see if you are having any UDP packet loss by running ‘netstat -su’. Look for the number associated with ‘receive buffer errors'.

If ‘receive buffer errors' Is not zero, then you are losing messages. Understand that the ‘receive buffer errors' Is cumulative and only goes up and is never reset until a reboot. So if you want to “watch” the number, you need to query it on a daily basis to understand how many packets you are losing per day.

Bryan

From: syslog-ng <syslog-ng-bounces@lists.balabit.hu> On Behalf Of Daniel Ehrlich
Sent: Tuesday, June 15, 2021 12:49 AM
To: Syslog-ng users' and developers' mailing list <syslog-ng@lists.balabit.hu>
Subject: [EXTERNAL] Re: [syslog-ng] Tips to diagnose missing syslog messages?

Thanks Laci,

Yes this is syslog-ng as a receiver, from a network source of UDP 514.

The destination is files on the local drive.

The network path is allowed by firewall as we receive several messages, though some go missing.

For example, some hosts send every minute and only 5 of the 15 might make it to the dest file.

I may be overloading the listener with 3-500 events per second?

I think I am running syslog-ng 3.5 from RedHat repos.

I was sure if the multi-threaded option would help?

Thanks

Daniel

From: syslog-ng <syslog-ng-bounces@lists.balabit.hu> on behalf of Laszlo Szemere (lszemere) <Laszlo.Szemere@oneidentity.com>
Sent: Tuesday, 15 June 2021 12:34 AM
To: Syslog-ng users' and developers' mailing list <syslog-ng@lists.balabit.hu>
Subject: Re: [syslog-ng] Tips to diagnose missing syslog messages?

Hello Daniel,

If I understand you correctly: you have a problem that one of your Syslog-ng server is not receiving a log message. (Not that you do not receive a message FROM one of your Syslog-ng server.)

From my experience: In those cases when someone tries to diagnose an issue where Syslog-ng do not receive a message, it is always a good first step to determine if the message actually reaches Syslog-ng or not.

i.e.: in case of a network source, try to receive the message with a simple netcat command. (Maybe the message was dropped by a firewall before reaching your machine, and Syslog-ng has nothing to do with it.) At this point we do not really care about the format of the message.

If you made sure that those messages are reaching the application, we should try to diagnose Syslog-ng itself. For that we will need some information about your setup.

i.e.:

- your platform

- version of Syslog-ng (Where it is obtained from? i.e. local build)

- Related config parts. (including the source driver which is expected to receive the logs)

- How do you start Syslog-ng? (i.e.: as a service)

Best regards,

Laci

From: syslog-ng <syslog-ng-bounces@lists.balabit.hu> on behalf of Daniel Ehrlich <Daniel.Ehrlich@usq.edu.au>
Sent: Wednesday, June 9, 2021 07:24
To: Syslog-ng users' and developers' mailing list <syslog-ng@lists.balabit.hu>
Subject: [syslog-ng] Tips to diagnose missing syslog messages?

CAUTION: This email originated from outside of the organization. Do not follow guidance, click links, or open attachments unless you recognize the sender and know the content is safe.

Hi Everyone,

Does anyone have some diagnostic tips to offer to diagnose why syslog messages are not being received?

I have syslog-ng on a few servers but one is losing messages (others syslog-ng servers might be dropping that I am unaware off).

Not sure if there is some options I should add to the conf or diag commands or load specifications I should check?

Options:
options {

chain_hostnames(no);

create_dirs (yes);

dir_perm(0755);

dns_cache(yes);

keep_hostname(yes);

log_fifo_size(2048);

log_msg_size(8192);

perm(0644);

time_reopen (10);

use_dns(yes);

use_fqdn(yes);

flush_lines(100);

};

Also adding the flag-control flag to the log stanza.

Thanks

Daniel

__________________________________________________________________

This email (including any attached files) is confidential and is

for the intended recipient(s) only. If you received this email by

mistake, please, as a courtesy, tell the sender, then delete this

email.

The views and opinions are the originator's and do not necessarily

reflect those of the University of Southern Queensland. Although

all reasonable precautions were taken to ensure that this email

contained no viruses at the time it was sent we accept no

liability for any losses arising from its receipt.

The University of Southern Queensland is a registered provider

of education with the Australian Government.

(CRICOS Institution Code QLD 00244B / NSW 02225M, TEQSA PRV12081)