<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<div class="moz-cite-prefix">Since you already have the no-parse
flag on the source, everything goes into $MESSAGE<br>
<br>
Make a file destination with the template of "$MESSAGE\n" and it
should contain the entire payload.<br>
<br>
Evan.<br>
<br>
On 05/10/2017 12:57 PM, Scot wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CAOxbc8EkyOmiaW03dDH0goHkayyy-xaAWUVdrJXi5CbaE6Gdhw@mail.gmail.com">
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<div dir="ltr">
<div>Thanks Evan, </div>
<div><br>
</div>
<div>Bumped it up to 32768 </div>
<div><br>
</div>
Error extracting JSON members into LogMessage as the top-level
JSON object is not an object; input='":"A <br>
<div>I think there may be something else I need to do with the
payload. </div>
<div><br>
</div>
<div>How would I dump everything to a file to look at it ? </div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Wed, May 10, 2017 at 2:10 PM, Evan
Rempel <span dir="ltr"><<a href="mailto:erempel@uvic.ca"
target="_blank" moz-do-not-send="true">erempel@uvic.ca</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF">
<div class="m_2935620687240762513moz-cite-prefix">looks
like you might be running into the maximum message size.<br>
Try setting the syslog-ng configuration item<br>
<br>
log_msg_size(64K);
<div>
<div class="h5"><br>
<br>
<br>
On 05/10/2017 10:50 AM, Scot wrote:<br>
</div>
</div>
</div>
<div>
<div class="h5">
<blockquote type="cite">
<div dir="ltr">Using a RAW TCP seems to be loosing
some of the beats header data and messages are
getting concatenated.
<div>
<div>Trying different options but I'm fumbling. <br>
</div>
<div><br>
</div>
<div>
<div> syslog-ng[4596]: Unparsable JSON stream
encountered; input='=net"},"message":"<wbr>Synchronization
of a replica of an Active Directory naming
context has begun.\n\nDestination
DRA:\tCN=NTDS Settings,CN=...blaaa"</div>
</div>
<div><br>
</div>
<div><br>
</div>
<div>source s_BEATS {network(port(5140)
flags(no-parse));}</div>
<div>
<div>parser p_json {</div>
<div> json-parser (prefix(".json."));</div>
<div>};</div>
</div>
<div>log { source(s_BEATS); parser(p_json);
destination (d_file); };<br>
</div>
<div><br>
</div>
<div><br>
</div>
<div>Anyone have a howto or blog for using
syslog-ng with json inputs ? </div>
<div>I'm looking at the syslog-ng-ose-latest-<wbr>guides
but it's hard to put all the input output and
parser requirements together. </div>
<div><br>
</div>
<div>Trying to get here </div>
<div>winlogbeat->syslog-ng->ES </div>
<div>winlogbeat->syslog-ng-><wbr>SPLUNKForwader</div>
<div>winlogbeat->syslog-ng->/opt/<wbr>syslog-ng/logs/$FROM_HOST.<wbr>json <br>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">or </div>
<div class="gmail_quote">
<div>winlogbeat->logstash->syslog-<wbr>ng->ES
</div>
<div>...</div>
<div><br>
</div>
</div>
<div class="gmail_quote">On Tue, May 9, 2017
at 3:27 AM, Fabien Wernli <span dir="ltr"><<a
href="mailto:wernli@in2p3.fr"
target="_blank" moz-do-not-send="true">wernli@in2p3.fr</a>></span>
wrote:<br>
<blockquote class="gmail_quote"
style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">Hi,<br>
<span><br>
On Mon, May 08, 2017 at 11:30:14PM
+0000, Scot wrote:<br>
> I'm trying to find a solution that
will let me mirror my beats data like<br>
> syslog-ng lets me do with syslog
traffic.<br>
<br>
</span>As far as I know those tools simply
send the data over TCP in JSON format.<br>
If you just need to do routing using
syslog-ng, you can simply use network<br>
source with flags(no-parse). If you need
to process the data using<br>
syslog-ng, you'll also need the
json-parser().<br>
<br>
Cheers<br>
<br>
</blockquote>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</blockquote>
<br>
</body>
</html>