syslog-ng parsing Error
Hello, I have a syslog source node sending syslogs, and they are being generated via a python script, and is using Python Rfc5426SysLogHandler. So, these syslog messages should be RFC compliant. However, syslog-ng does prepend an error message before sending it on to be put into storage. Example error message from syslog-ng = <43>Jun 21 10:27:38 syslog-ng-Server syslog-ng[2559]: Error processing log message: xxxxx timestamp, source hostname and payload follows. I've done some googling, but haven't been able to find out what error 2559 means. Any thoughts of what I might do to determine what syslog-ng isn't liking about the syslog it is receiving? I need to relay this information to a developer so they can make adjustments to the python script. Best regards, David
On Jun 21, 2018 18:11, "David Campeau" <David.Campeau@tn.gov> wrote: Hello, I have a syslog source node sending syslogs, and they are being generated via a python script, and is using Python Rfc5426SysLogHandler. So, these syslog messages should be RFC compliant. However, syslog-ng does prepend an error message before sending it on to be put into storage. Example error message from syslog-ng = <43>Jun 21 10:27:38 *syslog-ng-Server syslog-ng[2559]: **Error processing log message:* xxxxx timestamp, source hostname and payload follows. I’ve done some googling, but haven’t been able to find out what error 2559 means. 2559 is the pid of the syslog-ng process. Any thoughts of what I might do to determine what syslog-ng isn’t liking about the syslog it is receiving? I need to relay this information to a developer so they can make adjustments to the python script. After the colon the original message is reproduced verbatim, but as far as I understand you changed that so judging why parsing failed is not possible. One usual suspect is that you are using legacy bsd style source, wheras your message is in the 5424 format. Using the syslog() source instead of tcp/udp can help. Hope this helps. Best regards, David ____________________________________________________________ __________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/? product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
Thank you for the response. This is how the source is set up and is listening. It is expecting UDP on port 528. You mentioned syslog(), but does my example need to be tweaked in some way? source s_network { syslog(transport("udp") port(528)); Best Regards, From: syslog-ng [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Scheidler, Balázs Sent: Friday, June 22, 2018 12:44 AM To: Syslog-ng users' and developers' mailing list Subject: Re: [syslog-ng] syslog-ng parsing Error On Jun 21, 2018 18:11, "David Campeau" <David.Campeau@tn.gov<mailto:David.Campeau@tn.gov>> wrote: Hello, I have a syslog source node sending syslogs, and they are being generated via a python script, and is using Python Rfc5426SysLogHandler. So, these syslog messages should be RFC compliant. However, syslog-ng does prepend an error message before sending it on to be put into storage. Example error message from syslog-ng = <43>Jun 21 10:27:38 syslog-ng-Server syslog-ng[2559]: Error processing log message: xxxxx timestamp, source hostname and payload follows. I’ve done some googling, but haven’t been able to find out what error 2559 means. 2559 is the pid of the syslog-ng process. Any thoughts of what I might do to determine what syslog-ng isn’t liking about the syslog it is receiving? I need to relay this information to a developer so they can make adjustments to the python script. After the colon the original message is reproduced verbatim, but as far as I understand you changed that so judging why parsing failed is not possible. One usual suspect is that you are using legacy bsd style source, wheras your message is in the 5424 format. Using the syslog() source instead of tcp/udp can help. Hope this helps. Best regards, David ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
Looks like messaged are being properly filtered now. I substituted “syslog” with “network”, and the parsing errors went away. However, I’m not sure of the implications of this change? Network() source options vs. syslog() source options. source s_network { ## syslog(transport("udp") port(528)); network(transport("udp") port(528)); From: syslog-ng [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of David Campeau Sent: Friday, June 22, 2018 3:04 PM To: Syslog-ng users' and developers' mailing list Subject: Re: [syslog-ng] syslog-ng parsing Error Thank you for the response. This is how the source is set up and is listening. It is expecting UDP on port 528. You mentioned syslog(), but does my example need to be tweaked in some way? source s_network { syslog(transport("udp") port(528)); Best Regards, From: syslog-ng [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Scheidler, Balázs Sent: Friday, June 22, 2018 12:44 AM To: Syslog-ng users' and developers' mailing list Subject: Re: [syslog-ng] syslog-ng parsing Error On Jun 21, 2018 18:11, "David Campeau" <David.Campeau@tn.gov<mailto:David.Campeau@tn.gov>> wrote: Hello, I have a syslog source node sending syslogs, and they are being generated via a python script, and is using Python Rfc5426SysLogHandler. So, these syslog messages should be RFC compliant. However, syslog-ng does prepend an error message before sending it on to be put into storage. Example error message from syslog-ng = <43>Jun 21 10:27:38 syslog-ng-Server syslog-ng[2559]: Error processing log message: xxxxx timestamp, source hostname and payload follows. I’ve done some googling, but haven’t been able to find out what error 2559 means. 2559 is the pid of the syslog-ng process. Any thoughts of what I might do to determine what syslog-ng isn’t liking about the syslog it is receiving? I need to relay this information to a developer so they can make adjustments to the python script. After the colon the original message is reproduced verbatim, but as far as I understand you changed that so judging why parsing failed is not possible. One usual suspect is that you are using legacy bsd style source, wheras your message is in the 5424 format. Using the syslog() source instead of tcp/udp can help. Hope this helps. Best regards, David ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
Hi, * syslog(transport(udp)) expects RFC5424 format, and in versions past 3.3, it accepts both RFC5424 and RFC3164. * network(transport(udp)) expects RFC3164, can be enabled to accept RFC5424 by adding flags(syslog-protocol) to the options Also the two differ in framing of TCP and TLS transports, but are the same in UDP. -- Bazsi On Fri, Jun 22, 2018 at 10:29 PM, David Campeau <David.Campeau@tn.gov> wrote:
Looks like messaged are being properly filtered now. I substituted “syslog” with “network”, and the parsing errors went away. However, I’m not sure of the implications of this change? Network() source options vs. syslog() source options.
source s_network {
## syslog(transport("udp") port(528));
network(transport("udp") port(528));
*From:* syslog-ng [mailto:syslog-ng-bounces@lists.balabit.hu] *On Behalf Of *David Campeau *Sent:* Friday, June 22, 2018 3:04 PM
*To:* Syslog-ng users' and developers' mailing list *Subject:* Re: [syslog-ng] syslog-ng parsing Error
Thank you for the response.
This is how the source is set up and is listening. It is expecting UDP on port 528. You mentioned syslog(), but does my example need to be tweaked in some way?
source s_network {
syslog(transport("udp") port(528));
Best Regards,
*From:* syslog-ng [mailto:syslog-ng-bounces@lists.balabit.hu] *On Behalf Of *Scheidler, Balázs *Sent:* Friday, June 22, 2018 12:44 AM *To:* Syslog-ng users' and developers' mailing list *Subject:* Re: [syslog-ng] syslog-ng parsing Error
On Jun 21, 2018 18:11, "David Campeau" <David.Campeau@tn.gov> wrote:
Hello,
I have a syslog source node sending syslogs, and they are being generated via a python script, and is using Python Rfc5426SysLogHandler. So, these syslog messages should be RFC compliant. However, syslog-ng does prepend an error message before sending it on to be put into storage.
Example error message from syslog-ng = <43>Jun 21 10:27:38 *syslog-ng-Server syslog-ng[2559]: **Error processing log message:* xxxxx timestamp, source hostname and payload follows.
I’ve done some googling, but haven’t been able to find out what error 2559 means.
2559 is the pid of the syslog-ng process.
Any thoughts of what I might do to determine what syslog-ng isn’t liking about the syslog it is receiving? I need to relay this information to a developer so they can make adjustments to the python script.
After the colon the original message is reproduced verbatim, but as far as I understand you changed that so judging why parsing failed is not possible.
One usual suspect is that you are using legacy bsd style source, wheras your message is in the 5424 format.
Using the syslog() source instead of tcp/udp can help.
Hope this helps.
Best regards,
David
____________________________________________________________ __________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/? product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
____________________________________________________________ __________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/? product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
participants (2)
-
David Campeau
-
Scheidler, Balázs