Syslog-ng don't Listening UDP packets PORT(514) - Problem History
Good Morning, Yesterday, we performed the first test of syslog-ng 3.24 OSE on a RHEL 7.6 VM in our customer's production environment. And we identified a very strange behavior. We saw through tcpdump that packets were being received by the network card but were not captured by syslog-ng. We have a UDP packet send simulator, and when I was pointing my simulator to another IP on the same machine(the users' connection IP), syslog-ng does the capping and consequently the treatment we put into the syslog-ng.conf file. What kind of TROUBLESHOOTING we can do in order to identify the reason for syslog-ng to capture packets on the users' connection IP, but for the application connection IP, it doesn't identify anything. It is informed by the client at the beginning of our project that syslog should always capture all UDP packets received by the application IP. This is the configuration of the "source" we have on this machine in our client's productive environment. source s_model { #udp(port(514)); udp(ip(10.46.105.34) port(514)); # udp(); #network(transport("udp")); #network( # ip("10.46.105.34") # transport("udp") #); }; Tks, Esta mensagem, incluindo seus anexos, pode conter informações privilegiadas e/ou de caráter confidencial, não podendo ser retransmitida sem autorização do remetente. Se você não é o destinatário ou pessoa autorizada para recebê-la, informamos que o seu uso, divulgação, cópia ou arquivamento são proibidos. Portanto, se você recebeu esta mensagem por engano, por favor nos informe respondendo imediatamente a este e-mail e delete o seu conteúdo. This message, including its attachments, may contain privileged or confidential information, and it must not be fowarded without the express authorization of the sender. If you are not the intended recipient, we hereby inform you that the use, disclosure, copy or filing are forbidden. So, if you received this message as a mistake, please inform us by answering this e-mail and deleting its contents Questo messaggio, inclusi gli allegati, potrebbe contenere informazioni privilegiate e/o riservate, e non deve essere ritrasmesse senza l'autorizzazione del mittente. Se non siete il destinatario o la persona autorizzata a riceverlo, informiamo che il suo utilizzo, diffusione, copia o archiviazione sono proibite. Quindi, se avete ricevuto questo messaggio per errore, per cortesia ci informi rispondendo immediatamente a questa email e cancelli il suo contenuto
Hi, The most likely reason that another application also listens on this IP:port pair (it is possible in case of UDP and in that case the packet will be captured by the app started later) Check the output of netstat -anu |grep 514 when syslog-ng does not capture the udp packets. You should see only one line here. In other case, you should check the debug log of syslog-ng (syslog-ng -Fevd) perhaps there is some useful information. ________________________________ Feladó: syslog-ng <syslog-ng-bounces@lists.balabit.hu>, meghatalmazó: William Luiz Ribeiro Vasconcelos Da Silva <wsilva_ericsson@timbrasil.com.br> Elküldve: 2019. november 8., péntek 16:28 Címzett: syslog-ng@lists.balabit.hu <syslog-ng@lists.balabit.hu> Tárgy: [syslog-ng] Syslog-ng don't Listening UDP packets PORT(514) - Problem History CAUTION: This email originated from outside of the organization. Do not follow guidance, click links, or open attachments unless you recognize the sender and know the content is safe. Good Morning, Yesterday, we performed the first test of syslog-ng 3.24 OSE on a RHEL 7.6 VM in our customer's production environment. And we identified a very strange behavior. We saw through tcpdump that packets were being received by the network card but were not captured by syslog-ng. We have a UDP packet send simulator, and when I was pointing my simulator to another IP on the same machine(the users' connection IP), syslog-ng does the capping and consequently the treatment we put into the syslog-ng.conf file. What kind of TROUBLESHOOTING we can do in order to identify the reason for syslog-ng to capture packets on the users' connection IP, but for the application connection IP, it doesn't identify anything. It is informed by the client at the beginning of our project that syslog should always capture all UDP packets received by the application IP. This is the configuration of the "source" we have on this machine in our client's productive environment. source s_model { #udp(port(514)); udp(ip(10.46.105.34) port(514)); # udp(); #network(transport("udp")); #network( # ip("10.46.105.34") # transport("udp") #); }; Tks, Esta mensagem, incluindo seus anexos, pode conter informações privilegiadas e/ou de caráter confidencial, não podendo ser retransmitida sem autorização do remetente. Se você não é o destinatário ou pessoa autorizada para recebê-la, informamos que o seu uso, divulgação, cópia ou arquivamento são proibidos. Portanto, se você recebeu esta mensagem por engano, por favor nos informe respondendo imediatamente a este e-mail e delete o seu conteúdo. This message, including its attachments, may contain privileged or confidential information, and it must not be fowarded without the express authorization of the sender. If you are not the intended recipient, we hereby inform you that the use, disclosure, copy or filing are forbidden. So, if you received this message as a mistake, please inform us by answering this e-mail and deleting its contents Questo messaggio, inclusi gli allegati, potrebbe contenere informazioni privilegiate e/o riservate, e non deve essere ritrasmesse senza l'autorizzazione del mittente. Se non siete il destinatario o la persona autorizzata a riceverlo, informiamo che il suo utilizzo, diffusione, copia o archiviazione sono proibite. Quindi, se avete ricevuto questo messaggio per errore, per cortesia ci informi rispondendo immediatamente a questa email e cancelli il suo contenuto
Hello Again, The anwser of command is this: netstat -anu |grep 514 udp 0 0 10.46.105.34:514 0.0.0.0:* Only one line as expected. When I try to start the syslog-ng using the command, I see the following error: "[2019-11-08T13:37:32.326014] WARNING: Your configuration file uses an obsoleted keyword, please update your configuration; keyword='use_rcptid', change='This has been deprecated, try use_uniqid() instead', location='/etc/syslog-ng/syslog-ng.conf:24:4' [2019-11-08T13:37:32.326240] Module loaded and initialized successfully; module='system-source' [2019-11-08T13:37:32.326505] Module loaded and initialized successfully; module='sdjournal' [2019-11-08T13:37:32.327037] Module loaded and initialized successfully; module='kvformat' [2019-11-08T13:37:32.327073] Finishing include; content='block parser iptables-parser() at /usr/share/syslog-ng/include/scl/iptables/iptables.conf:23', depth='3' [2019-11-08T13:37:32.327479] Module loaded and initialized successfully; module='csvparser' [2019-11-08T13:37:32.327591] Finishing include; content='block parser sudo-parser() at /usr/share/syslog-ng/include/scl/sudo/sudo.conf:23', depth='3' [2019-11-08T13:37:32.327669] Finishing include; content='parser generator app-parser', depth='2' [2019-11-08T13:37:32.327709] Finishing include; content='source generator system', depth='1' [2019-11-08T13:37:32.328122] Module loaded and initialized successfully; module='afsocket' [2019-11-08T13:37:32.328504] Module loaded and initialized successfully; module='affile' [2019-11-08T13:37:32.329762] Processing @include statement; filename='/etc/syslog-ng/conf.d/*.conf', include-path='/etc/syslog-ng:/usr/share/syslog-ng/include' [2019-11-08T13:37:32.329836] Error creating persistent state file; filename='/var/lib/syslog-ng/syslog-ng.persist-', error='Permission denied (13)'" Could this be the reason for not capturing UDP packets? Tks, William Luiz ________________________________ De: syslog-ng <syslog-ng-bounces@lists.balabit.hu> em nome de Zoltan Pallagi (zpallagi) <Zoltan.Pallagi@oneidentity.com> Enviado: sexta-feira, 8 de novembro de 2019 13:31 Para: Syslog-ng users' and developers' mailing list <syslog-ng@lists.balabit.hu> Assunto: Re: [syslog-ng] Syslog-ng don't Listening UDP packets PORT(514) - Problem History Hi, The most likely reason that another application also listens on this IP:port pair (it is possible in case of UDP and in that case the packet will be captured by the app started later) Check the output of netstat -anu |grep 514 when syslog-ng does not capture the udp packets. You should see only one line here. In other case, you should check the debug log of syslog-ng (syslog-ng -Fevd) perhaps there is some useful information. ________________________________ Feladó: syslog-ng <syslog-ng-bounces@lists.balabit.hu>, meghatalmazó: William Luiz Ribeiro Vasconcelos Da Silva <wsilva_ericsson@timbrasil.com.br> Elküldve: 2019. november 8., péntek 16:28 Címzett: syslog-ng@lists.balabit.hu <syslog-ng@lists.balabit.hu> Tárgy: [syslog-ng] Syslog-ng don't Listening UDP packets PORT(514) - Problem History CAUTION: This email originated from outside of the organization. Do not follow guidance, click links, or open attachments unless you recognize the sender and know the content is safe. Good Morning, Yesterday, we performed the first test of syslog-ng 3.24 OSE on a RHEL 7.6 VM in our customer's production environment. And we identified a very strange behavior. We saw through tcpdump that packets were being received by the network card but were not captured by syslog-ng. We have a UDP packet send simulator, and when I was pointing my simulator to another IP on the same machine(the users' connection IP), syslog-ng does the capping and consequently the treatment we put into the syslog-ng.conf file. What kind of TROUBLESHOOTING we can do in order to identify the reason for syslog-ng to capture packets on the users' connection IP, but for the application connection IP, it doesn't identify anything. It is informed by the client at the beginning of our project that syslog should always capture all UDP packets received by the application IP. This is the configuration of the "source" we have on this machine in our client's productive environment. source s_model { #udp(port(514)); udp(ip(10.46.105.34) port(514)); # udp(); #network(transport("udp")); #network( # ip("10.46.105.34") # transport("udp") #); }; Tks, Esta mensagem, incluindo seus anexos, pode conter informações privilegiadas e/ou de caráter confidencial, não podendo ser retransmitida sem autorização do remetente. Se você não é o destinatário ou pessoa autorizada para recebê-la, informamos que o seu uso, divulgação, cópia ou arquivamento são proibidos. Portanto, se você recebeu esta mensagem por engano, por favor nos informe respondendo imediatamente a este e-mail e delete o seu conteúdo. This message, including its attachments, may contain privileged or confidential information, and it must not be fowarded without the express authorization of the sender. If you are not the intended recipient, we hereby inform you that the use, disclosure, copy or filing are forbidden. So, if you received this message as a mistake, please inform us by answering this e-mail and deleting its contents Questo messaggio, inclusi gli allegati, potrebbe contenere informazioni privilegiate e/o riservate, e non deve essere ritrasmesse senza l'autorizzazione del mittente. Se non siete il destinatario o la persona autorizzata a riceverlo, informiamo che il suo utilizzo, diffusione, copia o archiviazione sono proibite. Quindi, se avete ricevuto questo messaggio per errore, per cortesia ci informi rispondendo immediatamente a questa email e cancelli il suo contenuto
One must remember that packet loss using UDP is normal. To check to see if you are experiencing any lost packets issue this command: netstat -su Look at the value for 'receive buffer errors'. This is a cumulative count of lost UDP packets. Normally, this number should be zero. If the number is greater than zero, then you are losing UDP packets. I have found that adding this to your syslog-ng config file on the 'source' statement is helpful: so-reuseport(1) so_rcvbuf(80000000) Also, increasing these values in 'sysctl' is also helpful net.core.rmem_max net.core.rmem_default net.ipv4.udp_mem Bryan Klimek Mayo Clinic From: syslog-ng [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of William Luiz Ribeiro Vasconcelos Da Silva Sent: Friday, November 08, 2019 10:47 AM To: Syslog-ng users' and developers' mailing list Subject: [EXTERNAL] Re: [syslog-ng] Syslog-ng don't Listening UDP packets PORT(514) - Problem History Hello Again, The anwser of command is this: netstat -anu |grep 514 udp 0 0 10.46.105.34:514 0.0.0.0:* Only one line as expected. When I try to start the syslog-ng using the command, I see the following error: "[2019-11-08T13:37:32.326014] WARNING: Your configuration file uses an obsoleted keyword, please update your configuration; keyword='use_rcptid', change='This has been deprecated, try use_uniqid() instead', location='/etc/syslog-ng/syslog-ng.conf:24:4' [2019-11-08T13:37:32.326240] Module loaded and initialized successfully; module='system-source' [2019-11-08T13:37:32.326505] Module loaded and initialized successfully; module='sdjournal' [2019-11-08T13:37:32.327037] Module loaded and initialized successfully; module='kvformat' [2019-11-08T13:37:32.327073] Finishing include; content='block parser iptables-parser() at /usr/share/syslog-ng/include/scl/iptables/iptables.conf:23', depth='3' [2019-11-08T13:37:32.327479] Module loaded and initialized successfully; module='csvparser' [2019-11-08T13:37:32.327591] Finishing include; content='block parser sudo-parser() at /usr/share/syslog-ng/include/scl/sudo/sudo.conf:23', depth='3' [2019-11-08T13:37:32.327669] Finishing include; content='parser generator app-parser', depth='2' [2019-11-08T13:37:32.327709] Finishing include; content='source generator system', depth='1' [2019-11-08T13:37:32.328122] Module loaded and initialized successfully; module='afsocket' [2019-11-08T13:37:32.328504] Module loaded and initialized successfully; module='affile' [2019-11-08T13:37:32.329762] Processing @include statement; filename='/etc/syslog-ng/conf.d/*.conf', include-path='/etc/syslog-ng:/usr/share/syslog-ng/include' [2019-11-08T13:37:32.329836] Error creating persistent state file; filename='/var/lib/syslog-ng/syslog-ng.persist-', error='Permission denied (13)'" Could this be the reason for not capturing UDP packets? Tks, William Luiz ________________________________ De: syslog-ng <syslog-ng-bounces@lists.balabit.hu> em nome de Zoltan Pallagi (zpallagi) <Zoltan.Pallagi@oneidentity.com> Enviado: sexta-feira, 8 de novembro de 2019 13:31 Para: Syslog-ng users' and developers' mailing list <syslog-ng@lists.balabit.hu> Assunto: Re: [syslog-ng] Syslog-ng don't Listening UDP packets PORT(514) - Problem History Hi, The most likely reason that another application also listens on this IP:port pair (it is possible in case of UDP and in that case the packet will be captured by the app started later) Check the output of netstat -anu |grep 514 when syslog-ng does not capture the udp packets. You should see only one line here. In other case, you should check the debug log of syslog-ng (syslog-ng -Fevd) perhaps there is some useful information. ________________________________ Feladó: syslog-ng <syslog-ng-bounces@lists.balabit.hu>, meghatalmazó: William Luiz Ribeiro Vasconcelos Da Silva <wsilva_ericsson@timbrasil.com.br> Elküldve: 2019. november 8., péntek 16:28 Címzett: syslog-ng@lists.balabit.hu <syslog-ng@lists.balabit.hu> Tárgy: [syslog-ng] Syslog-ng don't Listening UDP packets PORT(514) - Problem History CAUTION: This email originated from outside of the organization. Do not follow guidance, click links, or open attachments unless you recognize the sender and know the content is safe. Good Morning, Yesterday, we performed the first test of syslog-ng 3.24 OSE on a RHEL 7.6 VM in our customer's production environment. And we identified a very strange behavior. We saw through tcpdump that packets were being received by the network card but were not captured by syslog-ng. We have a UDP packet send simulator, and when I was pointing my simulator to another IP on the same machine(the users' connection IP), syslog-ng does the capping and consequently the treatment we put into the syslog-ng.conf file. What kind of TROUBLESHOOTING we can do in order to identify the reason for syslog-ng to capture packets on the users' connection IP, but for the application connection IP, it doesn't identify anything. It is informed by the client at the beginning of our project that syslog should always capture all UDP packets received by the application IP. This is the configuration of the "source" we have on this machine in our client's productive environment. source s_model { #udp(port(514)); udp(ip(10.46.105.34) port(514)); # udp(); #network(transport("udp")); #network( # ip("10.46.105.34") # transport("udp") #); }; Tks, Esta mensagem, incluindo seus anexos, pode conter informações privilegiadas e/ou de caráter confidencial, não podendo ser retransmitida sem autorização do remetente. Se você não é o destinatário ou pessoa autorizada para recebê-la, informamos que o seu uso, divulgação, cópia ou arquivamento são proibidos. Portanto, se você recebeu esta mensagem por engano, por favor nos informe respondendo imediatamente a este e-mail e delete o seu conteúdo. This message, including its attachments, may contain privileged or confidential information, and it must not be fowarded without the express authorization of the sender. If you are not the intended recipient, we hereby inform you that the use, disclosure, copy or filing are forbidden. So, if you received this message as a mistake, please inform us by answering this e-mail and deleting its contents Questo messaggio, inclusi gli allegati, potrebbe contenere informazioni privilegiate e/o riservate, e non deve essere ritrasmesse senza l'autorizzazione del mittente. Se non siete il destinatario o la persona autorizzata a riceverlo, informiamo che il suo utilizzo, diffusione, copia o archiviazione sono proibite. Quindi, se avete ricevuto questo messaggio per errore, per cortesia ci informi rispondendo immediatamente a questa email e cancelli il suo contenuto
participants (3)
-
Klimek, Bryan J.
-
William Luiz Ribeiro Vasconcelos Da Silva
-
Zoltan Pallagi (zpallagi)