You can parse json using the json-parser() and filter any json name value pair simply with a filter expression.



On Apr 3, 2018 17:39, "Scot" <scotrn@gmail.com> wrote:

Sorry if I'm missing this but whats the best way to implement a json filter like these in syslog-ng, Patterndb? 

This is my logstash filter that sends data to a specific syslog-ng PORT for each condition which may grow and become a management headache when there are 6 logstash hosts that will need to support a filter for each breakout needed. 

I'd like to just send everything to a single syslog-ng port and have syslog-ng do the logic. which would then become. 

input {
  beats {
    port => 5044
  }
}

output{
    tcp {
    host => "loghost"
    port => "5140"
    mode => "client"
    codec => "json_lines"
    }
}

Logstash bloated output filters.  

output{
  if [type]=="wineventlog" and "DC" in [tags] {
    tcp {
    host => "loghost"
    port => "5142"
    mode => "client"
    codec => "json_lines"
    }
  } else if [type]=="wineventlog" and "PCI" in [tags] {
    tcp {
    host => "loghost"
    port => "5141"
    mode => "client"
    codec => "json_lines"
    }
  } else if [type]=="wineventlog" {
    tcp {
    host => "loghost"
    port => "5140"
    mode => "client"
    codec => "json_lines"
    }
  } else if [type]=="filebeat" and "apache" in [tags] {
    tcp {
    host => "loghost"
    port => "5145"
    mode => "client"
    codec => "json_lines"
    }
  } else if [type]=="filebeat" and "PCI" in [tags] {
    tcp {
    host => "loghost"
    port => "5144"
    mode => "client"
    codec => "json_lines"
    }
  } else if [type]=="filebeat" {
    tcp {
    host => "loghost"
    port => "5143"
    mode => "client"
    codec => "json_lines"
    }
 } else {
    file {
    path => "/opt/syslog-ng/logs/logstash/%{host}-%{+YYYY-MM-dd}.json"
    codec => "json_lines"
    }
}
}

______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq