Thanks for the summary, it was really interesting. On May 26, 2017 6:58 AM, "Scot" <scotrn@gmail.com> wrote: Hi All, Just wanted to report my progress on this subject, and say thanks for your input. I'm pretty happy with this setup so far I'll try to keep it short but wanted to share. *tl;dnr* HOWTO send Windows log data through syslog-ng and get it perfectly parsed in splunk and elasticsearch. *Purpose*: Funnel Windows events from enterprise or dmz zones through an remote aggregation relay into syslog-ng, allowing syslog-ng to replicate and broker log data based on business needs. *Components*: *Beats*- Winlogbeat is one of the beats agents by Elastic.co (Elasticsearch) that run on windows hosts to offload events to Logstash in this case. *Logstash*: Like beats it is able to take plugin based input and convert it to a plugin based output. I use it like a syslog relay but for compressed json streams. *syslog-ng:* Primary data aggregation and rule based distribution to SPLUNK,ES and Archive file. Beats support plugin based outputs, the primary two are es and logstash. This solution uses a logstash instance running on the same VM as my rsyslog relay. The output from winlogbeat, running on all windows servers, is a compressed json data stream to its designated logstash relay IP:PORT. This looks like this winlogbeat winlogbeat winlogbeat ---> TCP:5044 --> Logstash --> TCP:9150 --> syslog-ng winlogbeat winlogbeat Systems local to the syslog-ng server also go through a logstash hop. Logstash ads a single aggregator tag to each json message and can also take rule based actions like VictorOps, or massage data inline. syslog-ng then directs a copy of the json stream to a splunk forwarder, archive file, elasticsearch and any other output supported by Balabit. *Advantage*: I can write rules to tier my data to best of class or lower cost data analytic solutions or even new ones with a syslog-ng. *sylog-ng.conf * # This listens for data from logstash output. source s_BEATS {network(port(9150) log-msg-size(65536) flags(no-parse));}; # Define the output to splunkforwarder. destination d_splunk_BEAT {network("splunkforwarderhost" template("$(format-json --scope nv-pairs -p @timestamp=${ISODATE} -x MESSAGE)\n") port(5140));}; # Send a copy to SPLUNK or filter f_splunk_beat log { source(s_BEATS); parser {json-parser();}; destination(d_splunk_BEAT); }; *logstash beats-pipeline.conf output to syslog-ng* output { tcp { host => "syslog-ng host" port => "9150" mode => "client" codec => "json_lines" } } ____________________________________________________________ __________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/? product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq