Re: [syslog-ng] using VARARGS correctly

20 May 2021

      Hi Everyone,

I am having an issue when the Zulu timestamp is between 10 and 23:59. i.e. the logs format differently before 10AM and after 10AM.

I have captured in a tcpdump the syslogs coming in and they both seem the same.
We're at GMT+10 so this event was as 11:14:14 on 19th May.
Msg: 2021-05-19T1:14:14Z 10.18.0.14 E-MICRO 1621386854,23.85,31.48,5.91,n/a,n/a,n/a,n/a,n/a,n/a,O,O\0x0a\0x0d

This event was 09:07:47 the next day the 20th May:
Msg: 2021-05-19T23:07:46Z 10.18.0.14 E-MICRO 1621465666,24.96,32.54,7.36,n/a,n/a,n/a,n/a,n/a,n/a,O,O\0x0a\0x0d

In the output files, both events go to the 0519.log file, until 10AM or 00:00:00Z the next day.
First event logs as:
May 19 11:14:14 10.18.0.14 2021-05-19T1:14:14Z 10.18.0.14 E-MICRO 1621386854,23.85,31.48,5.91,n/a,n/a,n/a,n/a,n/a,n/a,O,O

Second event logs as:
May 19 23:07:46 10.18.0.14 E-MICRO 1621465666,24.96,32.54,7.36,n/a,n/a,n/a,n/a,n/a,n/a,O,O

I assume some built-in filtering is changing the way these are parsed in syslog-ng?

I have tried to play with raw message filtering but it doesn't take the conf file:
@version:3.5
@include "scl.conf"

# syslog-ng configuration file.
#

options {
   chain_hostnames(no);
   create_dirs (yes);
   dir_perm(0755);
   dns_cache(yes);
   keep_hostname(yes);
   log_fifo_size(2048);
   log_msg_size(8192);
   perm(0644);
   time_reopen (10);
   use_dns(yes);
   use_fqdn(yes);
};

source s_network {
   udp(port(514));
};

source attivo {
   tcp(port(514));
};

### DESTINATIONS
destination d_files_splunk {
   file("/opt/splunk/var/lib/splunk/syslog-ng/$HOST/$MONTH$DAY.log" create_dirs(yes));
};
destination d_files_nti {
   file("/opt/splunk/var/lib/splunk/syslog-ng/$HOST/$MONTH$DAY.log" create_dirs(yes) template(t_nti));
};

### FILTERS
filter nti {
   host("10.18.0.14" type(glob));
};
filter splunk {
   not (filter(nti));
};

### LOG
log {
   source(s_network);
   #filter(splunk);
   destination(d_files_splunk);
};
log {
   source(s_network);
   filter(nti);
   destination(d_files_nti);
};
log {
   source(attivo);
#   filter(splunk);
   destination(d_files_splunk);
};

### TEMPLATES
template t_nti {
        template("${RAWMSG}\n")
};

Any help is appreciated.
Thanks
Daniel Ehrlich
__________________________________________________________________

This email (including any attached files) is confidential and is 

for the intended recipient(s) only. If you received this email by 

mistake, please, as a courtesy, tell the sender, then delete this 

email.

The views and opinions are the originator's and do not necessarily 

reflect those of the University of Southern Queensland. Although 

all reasonable precautions were taken to ensure that this email 

contained no viruses at the time it was sent we accept no 

liability for any losses arising from its receipt.

The University of Southern Queensland is a registered provider 

of education with the Australian Government.

(CRICOS Institution Code QLD 00244B / NSW 02225M, TEQSA PRV12081)