[syslog-ng] Potential issue with line breaking not working correctly when writing logs to disk

[Balazs Scheidler]

Hi,

Sorry I have trouble matching up the config with the log output. Please
find some comments inline.

Balázs

On Fri, Aug 19, 2022, 16:47 John Law <JohnLaw at tfl.gov.uk> wrote:

> Hi,
>
> We have a number of network devices sending syslog traffic to syslog-ng
> OSE 3.25 installed on RHEL 7.9.
>
> We are having an issue where multiple events are being written as the same
> line.
>
> The config file is
>
> template-function adm1 "${HOST} message-length=$(length \"${MSG}\")
> ${MESSAGE}\n";
> source s_adm1 {
>     udp(ip(0.0.0.0) port(5527) );
>     tcp(ip(0.0.0.0) port(5527) );
> };
> destination d_adm1 {
>     file(
>
> "/Data/syslog_data/$LOGHOST/$R_YEAR-$R_MONTH-$R_DAY/adm1/$HOST_FROM/$HOST/$FACILITY.local"
>         template("$(adm1-function)\n")
>

You only mention $(adm1) as a template function but this one refers to
$(adm1-function)

    );
> };
> log { source(s_adm1); destination(d_adm1); flags(final); };
>
>
> A small extract of one of the files is, naturally I have changed the IPs
>
>
> 192.1.1.1/31181->192.1.3.1/135 0x0 source rule r12 N/A N/A 6 ACME-CNB010
> ACME CNB 1727847 N/A(N/A) reth3.860 UNKNOWN UNKNOWN UNKNOWN275 <14>Aug 19
> 11:38:50 sunny-fwl29 RT_FLOW: RT_FLOW_SESSION_CREATE: session created
> 192.6.20.1/47250->192.104.20.1/443 0x0 junos-https 192.6.20.1/47250->
> 192.104.20.1/443 0x0 N/A N/A N/A N/A 6 ACME-BWC002 ACME BWC_EDESIX_AWS
> 917879 N/A(N/A) reth3.860 UNKNOWN UNKNOWN UNKNOWN275 <14>Aug 19 11:38:50
> sunny-fwl29 RT_FLOW: RT_FLOW_SESSION_CREATE: session created
>

Can you show me the lines intermixed here? Again this does not seem to have
been formatted with $(adm1) at least as defined above.


 192.6.20.1/47252->192.104.20.1/443 0x0 junos-https 192.6.20.1/47252->
> 192.104.20.1/443 0x0 N/A N/A N/A N/A 6 ACME-BWC002 ACME BWC_EDESIX_AWS
> 595069 N/A(N/A) reth3.860 UNKNOWN UNKNOWN UNKNOWN285 <14>Aug 19 11:38:50
> sunny-fwl29 RT_FLOW: RT_FLOW_SESSION_CREATE: session created
> 192.1.57.1/54205->192.1.7.1/137 0x0 junos-nbname 192.1.57.1/54205->
> 192.1.7.1/137 0x0 N/A N/A N/A N/A 17 ACME-COMP007 ACME CCA 1735324
> N/A(N/A) reth3.860 UNKNOWN UNKNOWN UNKNOWN288 <14>Aug 19 11:38:50
> sunny-fwl29 RT_FLOW: RT_FLOW_SESSION_CREATE: session created
> 192.1.57.1/62486->192.1.7.1/135 0x0 junos-ms-rpc-tcp 192.1.57.1/62486->
> 192.1.7.1/135 0x0 N/A N/A N/A N/A 6 ACME-COMP007 ACME CCA 1027474
> N/A(N/A) reth3.860 UNKNOWN UNKNOWN UNKNOWN249
>
> I suspect it might be something to do with the amount of logs that are
> received in very quick succession, but I have no idea how to address this.
> One thing I have noticed is that it always appears to be the same device
> that is causing the issue. When I look at other devices coming in on the
> same port, they appear to write out correctly.
>
> Any suggestion would be really welcome.
>
> Thanks
>
> John
>
>
> This message has been scanned for malware by Forcepoint.
> www.forcepoint.com
>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation:
> http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20220820/1cbde755/attachment-0001.htm>