Some thoughts on my deployment
Logstash
I think I’m going to need to re-introduce logstash just to leverage the existing open source material of logstash filters and Kibana desktops.
VMware, ASA for example but wanted more real time data. I could probably do the realtime tags with pattendb.
syslog-ng counters
We use an IPAM API to create unique filters, log and destination conf files. The goal was to get unique syslog counters for every VLAN realtime directly from syslog-ng-ctl stats..
@include IPAM-filters
filter f_192_168_252_0 { netmask(192.168.252.0/24);};
filter f_192_168_253_0 { netmask(192.168.253.0/24);};
filter f_192_168_254_0 { netmask(192.168.254.0/30);};
@include IPAM-dest.conf
destination d_192_168_252_0 { file(/opt/syslog-ng/logs/192_168_252_0/$YEAR$MONTH$DAY-$HOUR-$HOST.log);};
destination d_192_168_253_0 { file(/opt/syslog-ng/logs/192_168_253_0/$YEAR$MONTH$DAY-$HOUR-$HOST.log);};
destination d_192_168_254_0 { file(/opt/syslog-ng/logs/192_168_254_0/$YEAR$MONTH$DAY-$HOUR-$HOST.log);};
@include IPAM-log.conf
log { source(s_net); filter(f_192_168_252_0); destination(d_192_168_252_0);};
log { source(s_net); filter(f_192_168_253_0); destination(d_192_168_253_0);};
log { source(s_net); filter(f_192_168_254_0); destination(d_192_168_254_0);};
log { source(s_net); filter(f_192_168_254_4); destination(d_192_168_254_4);};
Hi,
Does anyone have links or care to share notes on making a syslog-ng -> ELK scale for enterprise ?
I have some ideas and will gladly share my solution but also don’t want to spend days figuring these things out that have already been built.
There are many ELK specific references but I also want to make sure the model fits the syslog workload.
Thanks