Potential issue with line breaking not working correctly when writing logs to disk
Hi, We have a number of network devices sending syslog traffic to syslog-ng OSE 3.25 installed on RHEL 7.9. We are having an issue where multiple events are being written as the same line. The config file is template-function adm1 "${HOST} message-length=$(length \"${MSG}\") ${MESSAGE}\n"; source s_adm1 { udp(ip(0.0.0.0) port(5527) ); tcp(ip(0.0.0.0) port(5527) ); }; destination d_adm1 { file( "/Data/syslog_data/$LOGHOST/$R_YEAR-$R_MONTH-$R_DAY/adm1/$HOST_FROM/$HOST/$FACILITY.local" template("$(adm1-function)\n") ); }; log { source(s_adm1); destination(d_adm1); flags(final); }; A small extract of one of the files is, naturally I have changed the IPs 192.1.1.1/31181->192.1.3.1/135 0x0 source rule r12 N/A N/A 6 ACME-CNB010 ACME CNB 1727847 N/A(N/A) reth3.860 UNKNOWN UNKNOWN UNKNOWN275 <14>Aug 19 11:38:50 sunny-fwl29 RT_FLOW: RT_FLOW_SESSION_CREATE: session created 192.6.20.1/47250->192.104.20.1/443 0x0 junos-https 192.6.20.1/47250->192.104.20.1/443 0x0 N/A N/A N/A N/A 6 ACME-BWC002 ACME BWC_EDESIX_AWS 917879 N/A(N/A) reth3.860 UNKNOWN UNKNOWN UNKNOWN275 <14>Aug 19 11:38:50 sunny-fwl29 RT_FLOW: RT_FLOW_SESSION_CREATE: session created 192.6.20.1/47252->192.104.20.1/443 0x0 junos-https 192.6.20.1/47252->192.104.20.1/443 0x0 N/A N/A N/A N/A 6 ACME-BWC002 ACME BWC_EDESIX_AWS 595069 N/A(N/A) reth3.860 UNKNOWN UNKNOWN UNKNOWN285 <14>Aug 19 11:38:50 sunny-fwl29 RT_FLOW: RT_FLOW_SESSION_CREATE: session created 192.1.57.1/54205->192.1.7.1/137 0x0 junos-nbname 192.1.57.1/54205->192.1.7.1/137 0x0 N/A N/A N/A N/A 17 ACME-COMP007 ACME CCA 1735324 N/A(N/A) reth3.860 UNKNOWN UNKNOWN UNKNOWN288 <14>Aug 19 11:38:50 sunny-fwl29 RT_FLOW: RT_FLOW_SESSION_CREATE: session created 192.1.57.1/62486->192.1.7.1/135 0x0 junos-ms-rpc-tcp 192.1.57.1/62486->192.1.7.1/135 0x0 N/A N/A N/A N/A 6 ACME-COMP007 ACME CCA 1027474 N/A(N/A) reth3.860 UNKNOWN UNKNOWN UNKNOWN249 I suspect it might be something to do with the amount of logs that are received in very quick succession, but I have no idea how to address this. One thing I have noticed is that it always appears to be the same device that is causing the issue. When I look at other devices coming in on the same port, they appear to write out correctly. Any suggestion would be really welcome. Thanks John This message has been scanned for malware by Forcepoint. www.forcepoint.com
Hi, Sorry I have trouble matching up the config with the log output. Please find some comments inline. Balázs On Fri, Aug 19, 2022, 16:47 John Law <JohnLaw@tfl.gov.uk> wrote:
Hi,
We have a number of network devices sending syslog traffic to syslog-ng OSE 3.25 installed on RHEL 7.9.
We are having an issue where multiple events are being written as the same line.
The config file is
template-function adm1 "${HOST} message-length=$(length \"${MSG}\") ${MESSAGE}\n"; source s_adm1 { udp(ip(0.0.0.0) port(5527) ); tcp(ip(0.0.0.0) port(5527) ); }; destination d_adm1 { file(
"/Data/syslog_data/$LOGHOST/$R_YEAR-$R_MONTH-$R_DAY/adm1/$HOST_FROM/$HOST/$FACILITY.local" template("$(adm1-function)\n")
You only mention $(adm1) as a template function but this one refers to $(adm1-function) );
}; log { source(s_adm1); destination(d_adm1); flags(final); };
A small extract of one of the files is, naturally I have changed the IPs
192.1.1.1/31181->192.1.3.1/135 0x0 source rule r12 N/A N/A 6 ACME-CNB010 ACME CNB 1727847 N/A(N/A) reth3.860 UNKNOWN UNKNOWN UNKNOWN275 <14>Aug 19 11:38:50 sunny-fwl29 RT_FLOW: RT_FLOW_SESSION_CREATE: session created 192.6.20.1/47250->192.104.20.1/443 0x0 junos-https 192.6.20.1/47250-> 192.104.20.1/443 0x0 N/A N/A N/A N/A 6 ACME-BWC002 ACME BWC_EDESIX_AWS 917879 N/A(N/A) reth3.860 UNKNOWN UNKNOWN UNKNOWN275 <14>Aug 19 11:38:50 sunny-fwl29 RT_FLOW: RT_FLOW_SESSION_CREATE: session created
Can you show me the lines intermixed here? Again this does not seem to have been formatted with $(adm1) at least as defined above. 192.6.20.1/47252->192.104.20.1/443 0x0 junos-https 192.6.20.1/47252->
192.104.20.1/443 0x0 N/A N/A N/A N/A 6 ACME-BWC002 ACME BWC_EDESIX_AWS 595069 N/A(N/A) reth3.860 UNKNOWN UNKNOWN UNKNOWN285 <14>Aug 19 11:38:50 sunny-fwl29 RT_FLOW: RT_FLOW_SESSION_CREATE: session created 192.1.57.1/54205->192.1.7.1/137 0x0 junos-nbname 192.1.57.1/54205-> 192.1.7.1/137 0x0 N/A N/A N/A N/A 17 ACME-COMP007 ACME CCA 1735324 N/A(N/A) reth3.860 UNKNOWN UNKNOWN UNKNOWN288 <14>Aug 19 11:38:50 sunny-fwl29 RT_FLOW: RT_FLOW_SESSION_CREATE: session created 192.1.57.1/62486->192.1.7.1/135 0x0 junos-ms-rpc-tcp 192.1.57.1/62486-> 192.1.7.1/135 0x0 N/A N/A N/A N/A 6 ACME-COMP007 ACME CCA 1027474 N/A(N/A) reth3.860 UNKNOWN UNKNOWN UNKNOWN249
I suspect it might be something to do with the amount of logs that are received in very quick succession, but I have no idea how to address this. One thing I have noticed is that it always appears to be the same device that is causing the issue. When I look at other devices coming in on the same port, they appear to write out correctly.
Any suggestion would be really welcome.
Thanks
John
This message has been scanned for malware by Forcepoint. www.forcepoint.com
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
Thank you Balázs, I have done some more digging and I don't believe it is syslog that is the issue, I think the traffic is being forwarded to me from another syslog platform and that is where the issue lies. A packet capture is showing individual events with the correct line breaks, but then it also shows a number of events being sent as one large packet. Unless I can get syslog-ng to linebreak on regex when the data comes in, I need to go back to the source and get them to address this. PS the template mismatch below was a typo. Kind Regards John ________________________________ From: syslog-ng <syslog-ng-bounces@lists.balabit.hu> on behalf of Balazs Scheidler <bazsi77@gmail.com> Sent: 20 August 2022 06:33 To: Syslog-ng users' and developers' mailing list <syslog-ng@lists.balabit.hu> Subject: Re: [syslog-ng] Potential issue with line breaking not working correctly when writing logs to disk Hi, Sorry I have trouble matching up the config with the log output. Please find some comments inline. Balázs On Fri, Aug 19, 2022, 16:47 John Law <JohnLaw@tfl.gov.uk<mailto:JohnLaw@tfl.gov.uk>> wrote: Hi, We have a number of network devices sending syslog traffic to syslog-ng OSE 3.25 installed on RHEL 7.9. We are having an issue where multiple events are being written as the same line. The config file is template-function adm1 "${HOST} message-length=$(length \"${MSG}\") ${MESSAGE}\n"; source s_adm1 { udp(ip(0.0.0.0) port(5527) ); tcp(ip(0.0.0.0) port(5527) ); }; destination d_adm1 { file( "/Data/syslog_data/$LOGHOST/$R_YEAR-$R_MONTH-$R_DAY/adm1/$HOST_FROM/$HOST/$FACILITY.local" template("$(adm1)\n") You only mention $(adm1) as a template function but this one refers to $(adm1-function) ); }; log { source(s_adm1); destination(d_adm1); flags(final); }; A small extract of one of the files is, naturally I have changed the IPs 192.1.1.1/31181-<http://webdefence.global.blackspider.com/urlwrap/?q=AXicY3BnUGFmYAgUZWQoyqk0NMjQKy4q08tNzMxJzs8rKcrP0UvOz2UwMvc2jYhMNDU0MDO1MGfIys_Iy0ksdyhJy9FLzy_TK81myCgpKbDS1ze0NNIzBEF9Y0NDC0NdBgYGLR0GBgCskBrr&Z>>192.1.3.1/135<http://webdefence.global.blackspider.com/urlwrap/?q=AXicY3BnUGFmYAgUZWQoyqk0NMjQKy4q08tNzMxJzs8rKcrP0UvOz2UwMvc2jYhMNDU0MDO1MGfIys_Iy0ksdyhJy9FLzy_TK81myCgpKbDS1ze0NNIz1DPWM9Q3NDZlYGDQ0mFgAABZfBpb&Z> 0x0 source rule r12 N/A N/A 6 ACME-CNB010 ACME CNB 1727847 N/A(N/A) reth3.860 UNKNOWN UNKNOWN UNKNOWN275 <14>Aug 19 11:38:50 sunny-fwl29 RT_FLOW: RT_FLOW_SESSION_CREATE: session created 192.6.20.1/47250-<http://webdefence.global.blackspider.com/urlwrap/?q=AXicBcG9DoIwFAbQz_gMvIGTkduf2Bac3Byc3GBsSLTKhUtqxfj2noMLdlvgVm2Q-Wd0ondeaYpPHmQuWZgGmWDD1XV9dEZ71wS8JM0cv-dyZ3rISp8RqZTlpJRpLXmymow6But0DWB_AP7JSxsl&Z>>192.104.20.1/443<http://webdefence.global.blackspider.com/urlwrap/?q=AXicY3BnUGFmYAgUZWQoyqk0NMjQKy4q08tNzMxJzs8rKcrP0UvOz2UwMvc2jYhMNDU0MDO1MGfIys_Iy0ksdyhJy9FLzy_TK81myCgpKbDS1ze0NNIzNDDRMzLQM9Q3MTFmYGDQ0mFgAACs0Brw&Z> 0x0 junos-https 192.6.20.1/47250-<http://webdefence.global.blackspider.com/urlwrap/?q=AXicBcG9DoIwFAbQz_gMvIGTkduf2Bac3Byc3GBsSLTKhUtqxfj2noMLdlvgVm2Q-Wd0ondeaYpPHmQuWZgGmWDD1XV9dEZ71wS8JM0cv-dyZ3rISp8RqZTlpJRpLXmymow6But0DWB_AP7JSxsl&Z>>192.104.20.1/443<http://webdefence.global.blackspider.com/urlwrap/?q=AXicY3BnUGFmYAgUZWQoyqk0NMjQKy4q08tNzMxJzs8rKcrP0UvOz2UwMvc2jYhMNDU0MDO1MGfIys_Iy0ksdyhJy9FLzy_TK81myCgpKbDS1ze0NNIzNDDRMzLQM9Q3MTFmYGDQ0mFgAACs0Brw&Z> 0x0 N/A N/A N/A N/A 6 ACME-BWC002 ACME BWC_EDESIX_AWS 917879 N/A(N/A) reth3.860 UNKNOWN UNKNOWN UNKNOWN275 <14>Aug 19 11:38:50 sunny-fwl29 RT_FLOW: RT_FLOW_SESSION_CREATE: session created Can you show me the lines intermixed here? Again this does not seem to have been formatted with $(adm1) at least as defined above. 192.6.20.1/47252-<http://webdefence.global.blackspider.com/urlwrap/?q=AXicBcGxDoIwEAbg3_gMvoGT0Wt7sRSZ3ByY3HRsSLTqwZFaML4934cLtmvgulkhy9_ZRN88Ux9f0ulQsgp12oND62_36J2tfB3w1jRI_J3LQ-ipM00fpFLGxhh3YqqILTlzDOz5AGC3BxbJXRsn&Z>>192.104.20.1/443<http://webdefence.global.blackspider.com/urlwrap/?q=AXicY3BnUGFmYAgUZWQoyqk0NMjQKy4q08tNzMxJzs8rKcrP0UvOz2UwMvc2jYhMNDU0MDO1MGfIys_Iy0ksdyhJy9FLzy_TK81myCgpKbDS1ze0NNIzNDDRMzLQM9Q3MTFmYGDQ0mFgAACs0Brw&Z> 0x0 junos-https 192.6.20.1/47252-<http://webdefence.global.blackspider.com/urlwrap/?q=AXicBcGxDoIwEAbg3_gMvoGT0Wt7sRSZ3ByY3HRsSLTqwZFaML4934cLtmvgulkhy9_ZRN88Ux9f0ulQsgp12oND62_36J2tfB3w1jRI_J3LQ-ipM00fpFLGxhh3YqqILTlzDOz5AGC3BxbJXRsn&Z>>192.104.20.1/443<http://webdefence.global.blackspider.com/urlwrap/?q=AXicY3BnUGFmYAgUZWQoyqk0NMjQKy4q08tNzMxJzs8rKcrP0UvOz2UwMvc2jYhMNDU0MDO1MGfIys_Iy0ksdyhJy9FLzy_TK81myCgpKbDS1ze0NNIzNDDRMzLQM9Q3MTFmYGDQ0mFgAACs0Brw&Z> 0x0 N/A N/A N/A N/A 6 ACME-BWC002 ACME BWC_EDESIX_AWS 595069 N/A(N/A) reth3.860 UNKNOWN UNKNOWN UNKNOWN285 <14>Aug 19 11:38:50 sunny-fwl29 RT_FLOW: RT_FLOW_SESSION_CREATE: session created 192.1.57.1/54205-<http://webdefence.global.blackspider.com/urlwrap/?q=AXicBcHBDgExEAbgXzyDN3ASpp3GKHtyc3By49hsQpfZHanuirf3fThhOQcuixmK_thn-pSJ-tRpa0MtptRajxDPcr0lYb-TfcTT8qDpe6x3pYdNNL6Qa303zvEhEJNEYifb4GUDYLUG_sl4Gyg&Z>>192.1.7.1/137<http://webdefence.global.blackspider.com/urlwrap/?q=AXicY3BnUGFmYAgUZWQoyqk0NMjQKy4q08tNzMxJzs8rKcrP0UvOz2UwMvc2jYhMNDU0MDO1MGfIys_Iy0ksdyhJy9FLzy_TK81myCgpKbDS1ze0NNIz1DPXM9Q3NDZnYGDQ0mFgAABZxBph&Z> 0x0 junos-nbname 192.1.57.1/54205-<http://webdefence.global.blackspider.com/urlwrap/?q=AXicBcHBDgExEAbgXzyDN3ASpp3GKHtyc3By49hsQpfZHanuirf3fThhOQcuixmK_thn-pSJ-tRpa0MtptRajxDPcr0lYb-TfcTT8qDpe6x3pYdNNL6Qa303zvEhEJNEYifb4GUDYLUG_sl4Gyg&Z>>192.1.7.1/137<http://webdefence.global.blackspider.com/urlwrap/?q=AXicY3BnUGFmYAgUZWQoyqk0NMjQKy4q08tNzMxJzs8rKcrP0UvOz2UwMvc2jYhMNDU0MDO1MGfIys_Iy0ksdyhJy9FLzy_TK81myCgpKbDS1ze0NNIz1DPXM9Q3NDZnYGDQ0mFgAABZxBph&Z> 0x0 N/A N/A N/A N/A 17 ACME-COMP007 ACME CCA 1735324 N/A(N/A) reth3.860 UNKNOWN UNKNOWN UNKNOWN288 <14>Aug 19 11:38:50 sunny-fwl29 RT_FLOW: RT_FLOW_SESSION_CREATE: session created 192.1.57.1/62486-<http://webdefence.global.blackspider.com/urlwrap/?q=AXicBcGxDoIwEAbgn_gMvoGT0SvX2Bad3ByY2HRsSLTowZlaMb4934cLNiugW1fI8uc60SfPNMZBep1KVqFeR9jQuustOq69awKemiaJv3O5Cz10pu8LqZT3yRg-WmJygdh4e2j8HsB2ByzJ3Bsy&Z>>192.1.7.1/135<http://webdefence.global.blackspider.com/urlwrap/?q=AXicY3BnUGFmYAgUZWQoyqk0NMjQKy4q08tNzMxJzs8rKcrP0UvOz2UwMvc2jYhMNDU0MDO1MGfIys_Iy0ksdyhJy9FLzy_TK81myCgpKbDS1ze0NNIz1DPXM9Q3NDZlYGDQ0mFgAABZtBpf&Z> 0x0 junos-ms-rpc-tcp 192.1.57.1/62486-<http://webdefence.global.blackspider.com/urlwrap/?q=AXicBcGxDoIwEAbgn_gMvoGT0SvX2Bad3ByY2HRsSLTowZlaMb4934cLNiugW1fI8uc60SfPNMZBep1KVqFeR9jQuustOq69awKemiaJv3O5Cz10pu8LqZT3yRg-WmJygdh4e2j8HsB2ByzJ3Bsy&Z>>192.1.7.1/135<http://webdefence.global.blackspider.com/urlwrap/?q=AXicY3BnUGFmYAgUZWQoyqk0NMjQKy4q08tNzMxJzs8rKcrP0UvOz2UwMvc2jYhMNDU0MDO1MGfIys_Iy0ksdyhJy9FLzy_TK81myCgpKbDS1ze0NNIz1DPXM9Q3NDZlYGDQ0mFgAABZtBpf&Z> 0x0 N/A N/A N/A N/A 6 ACME-COMP007 ACME CCA 1027474 N/A(N/A) reth3.860 UNKNOWN UNKNOWN UNKNOWN249 I suspect it might be something to do with the amount of logs that are received in very quick succession, but I have no idea how to address this. One thing I have noticed is that it always appears to be the same device that is causing the issue. When I look at other devices coming in on the same port, they appear to write out correctly. Any suggestion would be really welcome. Thanks John This message has been scanned for malware by Forcepoint. www.forcepoint.com<http://www.forcepoint.com/> ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
Syslog-ng supports breaking up lines based on regényes, but only for files. I could possibly adapt that functionality to network sources of you are willing to try the functionality. But it's best if the source of fixed. On Mon, Aug 22, 2022, 14:58 John Law <JohnLaw@tfl.gov.uk> wrote:
Thank you Balázs,
I have done some more digging and I don't believe it is syslog that is the issue, I think the traffic is being forwarded to me from another syslog platform and that is where the issue lies.
A packet capture is showing individual events with the correct line breaks, but then it also shows a number of events being sent as one large packet. Unless I can get syslog-ng to linebreak on regex when the data comes in, I need to go back to the source and get them to address this.
PS the template mismatch below was a typo.
Kind Regards
John
------------------------------ *From:* syslog-ng <syslog-ng-bounces@lists.balabit.hu> on behalf of Balazs Scheidler <bazsi77@gmail.com> *Sent:* 20 August 2022 06:33 *To:* Syslog-ng users' and developers' mailing list < syslog-ng@lists.balabit.hu> *Subject:* Re: [syslog-ng] Potential issue with line breaking not working correctly when writing logs to disk
Hi,
Sorry I have trouble matching up the config with the log output. Please find some comments inline.
Balázs
On Fri, Aug 19, 2022, 16:47 John Law <JohnLaw@tfl.gov.uk> wrote:
Hi,
We have a number of network devices sending syslog traffic to syslog-ng OSE 3.25 installed on RHEL 7.9.
We are having an issue where multiple events are being written as the same line.
The config file is
template-function adm1 "${HOST} message-length=$(length \"${MSG}\") ${MESSAGE}\n"; source s_adm1 { udp(ip(0.0.0.0) port(5527) ); tcp(ip(0.0.0.0) port(5527) ); }; destination d_adm1 { file(
"/Data/syslog_data/$LOGHOST/$R_YEAR-$R_MONTH-$R_DAY/adm1/$HOST_FROM/$HOST/$FACILITY.local" template("$(adm1)\n")
You only mention $(adm1) as a template function but this one refers to $(adm1-function)
); }; log { source(s_adm1); destination(d_adm1); flags(final); };
A small extract of one of the files is, naturally I have changed the IPs
192.1.3.1/135 <http://webdefence.global.blackspider.com/urlwrap/?q=AXicY3BnUGFmYAgUZWQoyqk0NMjQKy4q08tNzMxJzs8rKcrP0UvOz2UwMvc2jYhMNDU0MDO1MGfIys_Iy0ksdyhJy9FLzy_TK81myCgpKbDS1ze0NNIz1DPWM9Q3NDZlYGDQ0mFgAABZfBpb&Z> 0x0 source rule r12 N/A N/A 6 ACME-CNB010 ACME CNB 1727847 N/A(N/A) reth3.860 UNKNOWN UNKNOWN UNKNOWN275 <14>Aug 19 11:38:50 sunny-fwl29 RT_FLOW: RT_FLOW_SESSION_CREATE: session created 192.6.20.1/47250- <http://webdefence.global.blackspider.com/urlwrap/?q=AXicBcG9DoIwFAbQz_gMvIGTkduf2Bac3Byc3GBsSLTKhUtqxfj2noMLdlvgVm2Q-Wd0ondeaYpPHmQuWZgGmWDD1XV9dEZ71wS8JM0cv-dyZ3rISp8RqZTlpJRpLXmymow6But0DWB_AP7JSxsl&Z> 192.104.20.1/443 <http://webdefence.global.blackspider.com/urlwrap/?q=AXicY3BnUGFmYAgUZWQoyqk0NMjQKy4q08tNzMxJzs8rKcrP0UvOz2UwMvc2jYhMNDU0MDO1MGfIys_Iy0ksdyhJy9FLzy_TK81myCgpKbDS1ze0NNIzNDDRMzLQM9Q3MTFmYGDQ0mFgAACs0Brw&Z> 0x0 junos-https 192.6.20.1/47250- <http://webdefence.global.blackspider.com/urlwrap/?q=AXicBcG9DoIwFAbQz_gMvIGTkduf2Bac3Byc3GBsSLTKhUtqxfj2noMLdlvgVm2Q-Wd0ondeaYpPHmQuWZgGmWDD1XV9dEZ71wS8JM0cv-dyZ3rISp8RqZTlpJRpLXmymow6But0DWB_AP7JSxsl&Z> 192.104.20.1/443 <http://webdefence.global.blackspider.com/urlwrap/?q=AXicY3BnUGFmYAgUZWQoyqk0NMjQKy4q08tNzMxJzs8rKcrP0UvOz2UwMvc2jYhMNDU0MDO1MGfIys_Iy0ksdyhJy9FLzy_TK81myCgpKbDS1ze0NNIzNDDRMzLQM9Q3MTFmYGDQ0mFgAACs0Brw&Z> 0x0 N/A N/A N/A N/A 6 ACME-BWC002 ACME BWC_EDESIX_AWS 917879 N/A(N/A) reth3.860 UNKNOWN UNKNOWN UNKNOWN275 <14>Aug 19 11:38:50 sunny-fwl29 RT_FLOW: RT_FLOW_SESSION_CREATE: session created
Can you show me the lines intermixed here? Again this does not seem to have been formatted with $(adm1) at least as defined above.
192.104.20.1/443 <http://webdefence.global.blackspider.com/urlwrap/?q=AXicY3BnUGFmYAgUZWQoyqk0NMjQKy4q08tNzMxJzs8rKcrP0UvOz2UwMvc2jYhMNDU0MDO1MGfIys_Iy0ksdyhJy9FLzy_TK81myCgpKbDS1ze0NNIzNDDRMzLQM9Q3MTFmYGDQ0mFgAACs0Brw&Z> 0x0 junos-https 192.6.20.1/47252- <http://webdefence.global.blackspider.com/urlwrap/?q=AXicBcGxDoIwEAbg3_gMvoGT0Wt7sRSZ3ByY3HRsSLTqwZFaML4934cLtmvgulkhy9_ZRN88Ux9f0ulQsgp12oND62_36J2tfB3w1jRI_J3LQ-ipM00fpFLGxhh3YqqILTlzDOz5AGC3BxbJXRsn&Z> 192.104.20.1/443 <http://webdefence.global.blackspider.com/urlwrap/?q=AXicY3BnUGFmYAgUZWQoyqk0NMjQKy4q08tNzMxJzs8rKcrP0UvOz2UwMvc2jYhMNDU0MDO1MGfIys_Iy0ksdyhJy9FLzy_TK81myCgpKbDS1ze0NNIzNDDRMzLQM9Q3MTFmYGDQ0mFgAACs0Brw&Z> 0x0 N/A N/A N/A N/A 6 ACME-BWC002 ACME BWC_EDESIX_AWS 595069 N/A(N/A) reth3.860 UNKNOWN UNKNOWN UNKNOWN285 <14>Aug 19 11:38:50 sunny-fwl29 RT_FLOW: RT_FLOW_SESSION_CREATE: session created 192.1.57.1/54205- <http://webdefence.global.blackspider.com/urlwrap/?q=AXicBcHBDgExEAbgXzyDN3ASpp3GKHtyc3By49hsQpfZHanuirf3fThhOQcuixmK_thn-pSJ-tRpa0MtptRajxDPcr0lYb-TfcTT8qDpe6x3pYdNNL6Qa303zvEhEJNEYifb4GUDYLUG_sl4Gyg&Z> 192.1.7.1/137 <http://webdefence.global.blackspider.com/urlwrap/?q=AXicY3BnUGFmYAgUZWQoyqk0NMjQKy4q08tNzMxJzs8rKcrP0UvOz2UwMvc2jYhMNDU0MDO1MGfIys_Iy0ksdyhJy9FLzy_TK81myCgpKbDS1ze0NNIz1DPXM9Q3NDZnYGDQ0mFgAABZxBph&Z> 0x0 junos-nbname 192.1.57.1/54205- <http://webdefence.global.blackspider.com/urlwrap/?q=AXicBcHBDgExEAbgXzyDN3ASpp3GKHtyc3By49hsQpfZHanuirf3fThhOQcuixmK_thn-pSJ-tRpa0MtptRajxDPcr0lYb-TfcTT8qDpe6x3pYdNNL6Qa303zvEhEJNEYifb4GUDYLUG_sl4Gyg&Z> 192.1.7.1/137 <http://webdefence.global.blackspider.com/urlwrap/?q=AXicY3BnUGFmYAgUZWQoyqk0NMjQKy4q08tNzMxJzs8rKcrP0UvOz2UwMvc2jYhMNDU0MDO1MGfIys_Iy0ksdyhJy9FLzy_TK81myCgpKbDS1ze0NNIz1DPXM9Q3NDZnYGDQ0mFgAABZxBph&Z> 0x0 N/A N/A N/A N/A 17 ACME-COMP007 ACME CCA 1735324 N/A(N/A) reth3.860 UNKNOWN UNKNOWN UNKNOWN288 <14>Aug 19 11:38:50 sunny-fwl29 RT_FLOW: RT_FLOW_SESSION_CREATE: session created 192.1.57.1/62486- <http://webdefence.global.blackspider.com/urlwrap/?q=AXicBcGxDoIwEAbgn_gMvoGT0SvX2Bad3ByY2HRsSLTowZlaMb4934cLNiugW1fI8uc60SfPNMZBep1KVqFeR9jQuustOq69awKemiaJv3O5Cz10pu8LqZT3yRg-WmJygdh4e2j8HsB2ByzJ3Bsy&Z> 192.1.7.1/135 <http://webdefence.global.blackspider.com/urlwrap/?q=AXicY3BnUGFmYAgUZWQoyqk0NMjQKy4q08tNzMxJzs8rKcrP0UvOz2UwMvc2jYhMNDU0MDO1MGfIys_Iy0ksdyhJy9FLzy_TK81myCgpKbDS1ze0NNIz1DPXM9Q3NDZlYGDQ0mFgAABZtBpf&Z> 0x0 junos-ms-rpc-tcp 192.1.57.1/62486- <http://webdefence.global.blackspider.com/urlwrap/?q=AXicBcGxDoIwEAbgn_gMvoGT0SvX2Bad3ByY2HRsSLTowZlaMb4934cLNiugW1fI8uc60SfPNMZBep1KVqFeR9jQuustOq69awKemiaJv3O5Cz10pu8LqZT3yRg-WmJygdh4e2j8HsB2ByzJ3Bsy&Z> 192.1.7.1/135 <http://webdefence.global.blackspider.com/urlwrap/?q=AXicY3BnUGFmYAgUZWQoyqk0NMjQKy4q08tNzMxJzs8rKcrP0UvOz2UwMvc2jYhMNDU0MDO1MGfIys_Iy0ksdyhJy9FLzy_TK81myCgpKbDS1ze0NNIz1DPXM9Q3NDZlYGDQ0mFgAABZtBpf&Z> 0x0 N/A N/A N/A N/A 6 ACME-COMP007 ACME CCA 1027474 N/A(N/A) reth3.860 UNKNOWN UNKNOWN UNKNOWN249
I suspect it might be something to do with the amount of logs that are received in very quick succession, but I have no idea how to address this. One thing I have noticed is that it always appears to be the same device that is causing the issue. When I look at other devices coming in on the same port, they appear to write out correctly.
Any suggestion would be really welcome.
Thanks
John
This message has been scanned for malware by Forcepoint. www.forcepoint.com
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
Thanks Balazs, I have gone back to the source to get them to address. Kind Regards John ________________________________ From: syslog-ng <syslog-ng-bounces@lists.balabit.hu> on behalf of Balazs Scheidler <bazsi77@gmail.com> Sent: 22 August 2022 21:19 To: Syslog-ng users' and developers' mailing list <syslog-ng@lists.balabit.hu> Subject: Re: [syslog-ng] Potential issue with line breaking not working correctly when writing logs to disk Syslog-ng supports breaking up lines based on regényes, but only for files. I could possibly adapt that functionality to network sources of you are willing to try the functionality. But it's best if the source of fixed. On Mon, Aug 22, 2022, 14:58 John Law <JohnLaw@tfl.gov.uk<mailto:JohnLaw@tfl.gov.uk>> wrote: Thank you Balázs, I have done some more digging and I don't believe it is syslog that is the issue, I think the traffic is being forwarded to me from another syslog platform and that is where the issue lies. A packet capture is showing individual events with the correct line breaks, but then it also shows a number of events being sent as one large packet. Unless I can get syslog-ng to linebreak on regex when the data comes in, I need to go back to the source and get them to address this. PS the template mismatch below was a typo. Kind Regards John ________________________________ From: syslog-ng <syslog-ng-bounces@lists.balabit.hu<mailto:syslog-ng-bounces@lists.balabit.hu>> on behalf of Balazs Scheidler <bazsi77@gmail.com<mailto:bazsi77@gmail.com>> Sent: 20 August 2022 06:33 To: Syslog-ng users' and developers' mailing list <syslog-ng@lists.balabit.hu<mailto:syslog-ng@lists.balabit.hu>> Subject: Re: [syslog-ng] Potential issue with line breaking not working correctly when writing logs to disk Hi, Sorry I have trouble matching up the config with the log output. Please find some comments inline. Balázs On Fri, Aug 19, 2022, 16:47 John Law <JohnLaw@tfl.gov.uk<mailto:JohnLaw@tfl.gov.uk>> wrote: Hi, We have a number of network devices sending syslog traffic to syslog-ng OSE 3.25 installed on RHEL 7.9. We are having an issue where multiple events are being written as the same line. The config file is template-function adm1 "${HOST} message-length=$(length \"${MSG}\") ${MESSAGE}\n"; source s_adm1 { udp(ip(0.0.0.0) port(5527) ); tcp(ip(0.0.0.0) port(5527) ); }; destination d_adm1 { file( "/Data/syslog_data/$LOGHOST/$R_YEAR-$R_MONTH-$R_DAY/adm1/$HOST_FROM/$HOST/$FACILITY.local" template("$(adm1)\n") You only mention $(adm1) as a template function but this one refers to $(adm1-function) ); }; log { source(s_adm1); destination(d_adm1); flags(final); }; A small extract of one of the files is, naturally I have changed the IPs 192.1.1.1/31181-<http://webdefence.global.blackspider.com/urlwrap/?q=AXicY3BnUGFmYAgUZWQoyqk0NMjQKy4q08tNzMxJzs8rKcrP0UvOz2UwMvc2jYhMNDU0MDO1MGfIys_Iy0ksdyhJy9FLzy_TK81myCgpKbDS1ze0NNIzBEF9Y0NDC0NdBgYGLR0GBgCskBrr&Z>>192.1.3.1/135<http://webdefence.global.blackspider.com/urlwrap/?q=AXicY3BnUGFmYAgUZWQoyqk0NMjQKy4q08tNzMxJzs8rKcrP0UvOz2UwMvc2jYhMNDU0MDO1MGfIys_Iy0ksdyhJy9FLzy_TK81myCgpKbDS1ze0NNIz1DPWM9Q3NDZlYGDQ0mFgAABZfBpb&Z> 0x0 source rule r12 N/A N/A 6 ACME-CNB010 ACME CNB 1727847 N/A(N/A) reth3.860 UNKNOWN UNKNOWN UNKNOWN275 <14>Aug 19 11:38:50 sunny-fwl29 RT_FLOW: RT_FLOW_SESSION_CREATE: session created 192.6.20.1/47250-<http://webdefence.global.blackspider.com/urlwrap/?q=AXicBcG9DoIwFAbQz_gMvIGTkduf2Bac3Byc3GBsSLTKhUtqxfj2noMLdlvgVm2Q-Wd0ondeaYpPHmQuWZgGmWDD1XV9dEZ71wS8JM0cv-dyZ3rISp8RqZTlpJRpLXmymow6But0DWB_AP7JSxsl&Z>>192.104.20.1/443<http://webdefence.global.blackspider.com/urlwrap/?q=AXicY3BnUGFmYAgUZWQoyqk0NMjQKy4q08tNzMxJzs8rKcrP0UvOz2UwMvc2jYhMNDU0MDO1MGfIys_Iy0ksdyhJy9FLzy_TK81myCgpKbDS1ze0NNIzNDDRMzLQM9Q3MTFmYGDQ0mFgAACs0Brw&Z> 0x0 junos-https 192.6.20.1/47250-<http://webdefence.global.blackspider.com/urlwrap/?q=AXicBcG9DoIwFAbQz_gMvIGTkduf2Bac3Byc3GBsSLTKhUtqxfj2noMLdlvgVm2Q-Wd0ondeaYpPHmQuWZgGmWDD1XV9dEZ71wS8JM0cv-dyZ3rISp8RqZTlpJRpLXmymow6But0DWB_AP7JSxsl&Z>>192.104.20.1/443<http://webdefence.global.blackspider.com/urlwrap/?q=AXicY3BnUGFmYAgUZWQoyqk0NMjQKy4q08tNzMxJzs8rKcrP0UvOz2UwMvc2jYhMNDU0MDO1MGfIys_Iy0ksdyhJy9FLzy_TK81myCgpKbDS1ze0NNIzNDDRMzLQM9Q3MTFmYGDQ0mFgAACs0Brw&Z> 0x0 N/A N/A N/A N/A 6 ACME-BWC002 ACME BWC_EDESIX_AWS 917879 N/A(N/A) reth3.860 UNKNOWN UNKNOWN UNKNOWN275 <14>Aug 19 11:38:50 sunny-fwl29 RT_FLOW: RT_FLOW_SESSION_CREATE: session created Can you show me the lines intermixed here? Again this does not seem to have been formatted with $(adm1) at least as defined above. 192.6.20.1/47252-<http://webdefence.global.blackspider.com/urlwrap/?q=AXicBcGxDoIwEAbg3_gMvoGT0Wt7sRSZ3ByY3HRsSLTqwZFaML4934cLtmvgulkhy9_ZRN88Ux9f0ulQsgp12oND62_36J2tfB3w1jRI_J3LQ-ipM00fpFLGxhh3YqqILTlzDOz5AGC3BxbJXRsn&Z>>192.104.20.1/443<http://webdefence.global.blackspider.com/urlwrap/?q=AXicY3BnUGFmYAgUZWQoyqk0NMjQKy4q08tNzMxJzs8rKcrP0UvOz2UwMvc2jYhMNDU0MDO1MGfIys_Iy0ksdyhJy9FLzy_TK81myCgpKbDS1ze0NNIzNDDRMzLQM9Q3MTFmYGDQ0mFgAACs0Brw&Z> 0x0 junos-https 192.6.20.1/47252-<http://webdefence.global.blackspider.com/urlwrap/?q=AXicBcGxDoIwEAbg3_gMvoGT0Wt7sRSZ3ByY3HRsSLTqwZFaML4934cLtmvgulkhy9_ZRN88Ux9f0ulQsgp12oND62_36J2tfB3w1jRI_J3LQ-ipM00fpFLGxhh3YqqILTlzDOz5AGC3BxbJXRsn&Z>>192.104.20.1/443<http://webdefence.global.blackspider.com/urlwrap/?q=AXicY3BnUGFmYAgUZWQoyqk0NMjQKy4q08tNzMxJzs8rKcrP0UvOz2UwMvc2jYhMNDU0MDO1MGfIys_Iy0ksdyhJy9FLzy_TK81myCgpKbDS1ze0NNIzNDDRMzLQM9Q3MTFmYGDQ0mFgAACs0Brw&Z> 0x0 N/A N/A N/A N/A 6 ACME-BWC002 ACME BWC_EDESIX_AWS 595069 N/A(N/A) reth3.860 UNKNOWN UNKNOWN UNKNOWN285 <14>Aug 19 11:38:50 sunny-fwl29 RT_FLOW: RT_FLOW_SESSION_CREATE: session created 192.1.57.1/54205-<http://webdefence.global.blackspider.com/urlwrap/?q=AXicBcHBDgExEAbgXzyDN3ASpp3GKHtyc3By49hsQpfZHanuirf3fThhOQcuixmK_thn-pSJ-tRpa0MtptRajxDPcr0lYb-TfcTT8qDpe6x3pYdNNL6Qa303zvEhEJNEYifb4GUDYLUG_sl4Gyg&Z>>192.1.7.1/137<http://webdefence.global.blackspider.com/urlwrap/?q=AXicY3BnUGFmYAgUZWQoyqk0NMjQKy4q08tNzMxJzs8rKcrP0UvOz2UwMvc2jYhMNDU0MDO1MGfIys_Iy0ksdyhJy9FLzy_TK81myCgpKbDS1ze0NNIz1DPXM9Q3NDZnYGDQ0mFgAABZxBph&Z> 0x0 junos-nbname 192.1.57.1/54205-<http://webdefence.global.blackspider.com/urlwrap/?q=AXicBcHBDgExEAbgXzyDN3ASpp3GKHtyc3By49hsQpfZHanuirf3fThhOQcuixmK_thn-pSJ-tRpa0MtptRajxDPcr0lYb-TfcTT8qDpe6x3pYdNNL6Qa303zvEhEJNEYifb4GUDYLUG_sl4Gyg&Z>>192.1.7.1/137<http://webdefence.global.blackspider.com/urlwrap/?q=AXicY3BnUGFmYAgUZWQoyqk0NMjQKy4q08tNzMxJzs8rKcrP0UvOz2UwMvc2jYhMNDU0MDO1MGfIys_Iy0ksdyhJy9FLzy_TK81myCgpKbDS1ze0NNIz1DPXM9Q3NDZnYGDQ0mFgAABZxBph&Z> 0x0 N/A N/A N/A N/A 17 ACME-COMP007 ACME CCA 1735324 N/A(N/A) reth3.860 UNKNOWN UNKNOWN UNKNOWN288 <14>Aug 19 11:38:50 sunny-fwl29 RT_FLOW: RT_FLOW_SESSION_CREATE: session created 192.1.57.1/62486-<http://webdefence.global.blackspider.com/urlwrap/?q=AXicBcGxDoIwEAbgn_gMvoGT0SvX2Bad3ByY2HRsSLTowZlaMb4934cLNiugW1fI8uc60SfPNMZBep1KVqFeR9jQuustOq69awKemiaJv3O5Cz10pu8LqZT3yRg-WmJygdh4e2j8HsB2ByzJ3Bsy&Z>>192.1.7.1/135<http://webdefence.global.blackspider.com/urlwrap/?q=AXicY3BnUGFmYAgUZWQoyqk0NMjQKy4q08tNzMxJzs8rKcrP0UvOz2UwMvc2jYhMNDU0MDO1MGfIys_Iy0ksdyhJy9FLzy_TK81myCgpKbDS1ze0NNIz1DPXM9Q3NDZlYGDQ0mFgAABZtBpf&Z> 0x0 junos-ms-rpc-tcp 192.1.57.1/62486-<http://webdefence.global.blackspider.com/urlwrap/?q=AXicBcGxDoIwEAbgn_gMvoGT0SvX2Bad3ByY2HRsSLTowZlaMb4934cLNiugW1fI8uc60SfPNMZBep1KVqFeR9jQuustOq69awKemiaJv3O5Cz10pu8LqZT3yRg-WmJygdh4e2j8HsB2ByzJ3Bsy&Z>>192.1.7.1/135<http://webdefence.global.blackspider.com/urlwrap/?q=AXicY3BnUGFmYAgUZWQoyqk0NMjQKy4q08tNzMxJzs8rKcrP0UvOz2UwMvc2jYhMNDU0MDO1MGfIys_Iy0ksdyhJy9FLzy_TK81myCgpKbDS1ze0NNIz1DPXM9Q3NDZlYGDQ0mFgAABZtBpf&Z> 0x0 N/A N/A N/A N/A 6 ACME-COMP007 ACME CCA 1027474 N/A(N/A) reth3.860 UNKNOWN UNKNOWN UNKNOWN249 I suspect it might be something to do with the amount of logs that are received in very quick succession, but I have no idea how to address this. One thing I have noticed is that it always appears to be the same device that is causing the issue. When I look at other devices coming in on the same port, they appear to write out correctly. Any suggestion would be really welcome. Thanks John This message has been scanned for malware by Forcepoint. www.forcepoint.com<http://www.forcepoint.com/> ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
participants (2)
-
Balazs Scheidler
-
John Law