<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Hi Everyone,</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
I am having an issue when the Zulu timestamp is between 10 and 23:59. i.e. the logs format differently before 10AM and after 10AM.</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
I have captured in a tcpdump the syslogs coming in and they both seem the same. </div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<span style="color: rgb(0, 0, 0); font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;">We're at GMT+10 so this event was as </span><span style="color: rgb(0, 0, 0); font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;">11:14:14
on 19th May.</span></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<span style="color: rgb(0, 0, 0); font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;">Msg: 2021-05-19T1:14:14Z 10.18.0.14 E-MICRO 1621386854,23.85,31.48,5.91,n/a,n/a,n/a,n/a,n/a,n/a,O,O\0x0a\0x0d<br>
</span></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<div style="font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; font-size: 12pt; font-family: Calibri, Arial, Helvetica, sans-serif; color: rgb(0, 0, 0);">
<span style="font-size: 12pt; font-family: Calibri, Arial, Helvetica, sans-serif; color: rgb(0, 0, 0);"><br>
</span></div>
<div style="font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; font-size: 12pt; font-family: Calibri, Arial, Helvetica, sans-serif; color: rgb(0, 0, 0);">
<span style="font-size: 12pt; font-family: Calibri, Arial, Helvetica, sans-serif; color: rgb(0, 0, 0);">This event was 09:07:47 the next day the 20th May:</span></div>
<div style="font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; font-size: 12pt; font-family: Calibri, Arial, Helvetica, sans-serif; color: rgb(0, 0, 0);">
<span style="font-size: 12pt; font-family: Calibri, Arial, Helvetica, sans-serif; color: rgb(0, 0, 0);">Msg: 2021-05-19T23:07:46Z 10.18.0.14 E-MICRO 1621465666,24.96,32.54,7.36,n/a,n/a,n/a,n/a,n/a,n/a,O,O\0x0a\0x0d</span></div>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
In the output files, both events go to the 0519.log file, until 10AM or 00:00:00Z the next day.</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
First event logs as:</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
May 19 11:14:14 10.18.0.14 2021-05-19T1:14:14Z 10.18.0.14 E-MICRO 1621386854,23.85,31.48,5.91,n/a,n/a,n/a,n/a,n/a,n/a,O,O<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Second event logs as:</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
May 19 23:07:46 10.18.0.14 E-MICRO 1621465666,24.96,32.54,7.36,n/a,n/a,n/a,n/a,n/a,n/a,O,O<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
I assume some built-in filtering is changing the way these are parsed in syslog-ng?</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
</div>
<div>
<div dir="ltr">
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0); background-color:rgb(255,255,255)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0); background-color:rgb(255,255,255)">
I have tried to play with raw message filtering but it doesn't take the conf file:</div>
<div style="color:rgb(212,212,212); background-color:rgb(30,30,30); font-family:"Droid Sans Mono",monospace,monospace,"Droid Sans Fallback"; font-weight:normal; font-size:14px; line-height:19px">
<span style="color: rgb(106, 153, 85);">@version:3.5</span><br>
</div>
<div style="color:rgb(212,212,212); background-color:rgb(30,30,30); font-family:"Droid Sans Mono",monospace,monospace,"Droid Sans Fallback"; font-weight:normal; font-size:14px; line-height:19px">
<span><span style="color:rgb(106,153,85)">
<div>@include "scl.conf"</div>
<div><br>
</div>
<div># syslog-ng configuration file.</div>
<div>#</div>
<div><br>
</div>
<div>options {</div>
<div> chain_hostnames(no);</div>
<div> create_dirs (yes);</div>
<div> dir_perm(0755);</div>
<div> dns_cache(yes);</div>
<div> keep_hostname(yes);</div>
<div> log_fifo_size(2048);</div>
<div> log_msg_size(8192);</div>
<div> perm(0644);</div>
<div> time_reopen (10);</div>
<div> use_dns(yes);</div>
<div> use_fqdn(yes);</div>
<div>};</div>
<div><br>
</div>
<div>source s_network {</div>
<div> udp(port(514));</div>
<div>};</div>
<div><br>
</div>
<div>source attivo {</div>
<div> tcp(port(514));</div>
<div>};</div>
<div><br>
</div>
<div>### DESTINATIONS</div>
<div>destination d_files_splunk {</div>
<div> file("/opt/splunk/var/lib/splunk/syslog-ng/$HOST/$MONTH$DAY.log" create_dirs(yes));</div>
<div>};</div>
<div>destination d_files_nti {</div>
<div> file("/opt/splunk/var/lib/splunk/syslog-ng/$HOST/$MONTH$DAY.log" create_dirs(yes) template(t_nti));</div>
<div>};</div>
<div><br>
</div>
<div>### FILTERS</div>
<div>filter nti {</div>
<div> host("10.18.0.14" type(glob));</div>
<div>};</div>
<div>filter splunk {</div>
<div> not (filter(nti));</div>
<div>};</div>
<div><br>
</div>
<div>### LOG</div>
<div>log {</div>
<div> source(s_network);</div>
<div> #filter(splunk);</div>
<div> destination(d_files_splunk);</div>
<div>};</div>
<div>log {</div>
<div> source(s_network);</div>
<div> filter(nti);</div>
<div> destination(d_files_nti);</div>
<div>};</div>
<div>log {</div>
<div> source(attivo);</div>
<div># filter(splunk);</div>
<div> destination(d_files_splunk);</div>
<div>};</div>
<div><br>
</div>
<div>### TEMPLATES</div>
<div>template t_nti {</div>
<div> template("${RAWMSG}\n")</div>
<span>};</span><br>
</span></span></div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0); background-color:rgb(255,255,255)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0); background-color:rgb(255,255,255)">
<span style="background-color:rgb(255, 255, 255);display:inline !important">Any help is appreciated.</span><br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0); background-color:rgb(255,255,255)">
<span style="background-color:rgb(255, 255, 255);display:inline !important">Thanks</span></div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0); background-color:rgb(255,255,255)">
<span style="background-color:rgb(255, 255, 255);display:inline !important">Daniel Ehrlich</span></div>
</div>
</div>
<div>__________________________________________________________________<div>This email (including any attached files) is confidential and is <div>for the intended recipient(s) only. If you received this email by <div>mistake, please, as a courtesy, tell the sender, then delete this <div>email.<br><div>The views and opinions are the originator's and do not necessarily <div>reflect those of the University of Southern Queensland. Although <div>all reasonable precautions were taken to ensure that this email <div>contained no viruses at the time it was sent we accept no <div>liability for any losses arising from its receipt.<br><div>The University of Southern Queensland is a registered provider <div>of education with the Australian Government.<div>(CRICOS Institution Code QLD 00244B / NSW 02225M, TEQSA PRV12081)</body>
</html>