[syslog-ng] Syslog-ng input for beats ?

[Scot]

Using a RAW TCP seems to be loosing some of the beats header data and
messages are getting concatenated.
Trying different options but I'm fumbling.

  syslog-ng[4596]: Unparsable JSON stream encountered;
input='=net"},"message":"Synchronization of a replica of an Active
Directory naming context has begun.\n\nDestination DRA:\tCN=NTDS
Settings,CN=...blaaa"


source s_BEATS          {network(port(5140) flags(no-parse));}
parser p_json {
    json-parser (prefix(".json."));
};
log { source(s_BEATS);  parser(p_json); destination (d_file); };


Anyone have a howto or blog for using syslog-ng with json inputs ?
I'm looking at the syslog-ng-ose-latest-guides but it's hard to put all the
input output and parser requirements together.

Trying to get here
winlogbeat->syslog-ng->ES
winlogbeat->syslog-ng->SPLUNKForwader
winlogbeat->syslog-ng->/opt/syslog-ng/logs/$FROM_HOST.json

or
winlogbeat->logstash->syslog-ng->ES
...

On Tue, May 9, 2017 at 3:27 AM, Fabien Wernli <wernli at in2p3.fr> wrote:

> Hi,
>
> On Mon, May 08, 2017 at 11:30:14PM +0000, Scot wrote:
> > I'm trying to find a solution that will let me mirror my beats data like
> > syslog-ng lets me do with syslog traffic.
>
> As far as I know those tools simply send the data over TCP in JSON format.
> If you just need to do routing using syslog-ng, you can simply use network
> source with flags(no-parse). If you need to process the data using
> syslog-ng, you'll also need the json-parser().
>
> Cheers
>
> ____________________________________________________________
> __________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?product=
> syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20170510/722ae7e6/attachment.html>