Thanks Evan, Bumped it up to 32768 Error extracting JSON members into LogMessage as the top-level JSON object is not an object; input='":"A I think there may be something else I need to do with the payload. How would I dump everything to a file to look at it ? On Wed, May 10, 2017 at 2:10 PM, Evan Rempel <erempel@uvic.ca> wrote:
looks like you might be running into the maximum message size. Try setting the syslog-ng configuration item
log_msg_size(64K);
On 05/10/2017 10:50 AM, Scot wrote:
Using a RAW TCP seems to be loosing some of the beats header data and messages are getting concatenated. Trying different options but I'm fumbling.
syslog-ng[4596]: Unparsable JSON stream encountered; input='=net"},"message":"Synchronization of a replica of an Active Directory naming context has begun.\n\nDestination DRA:\tCN=NTDS Settings,CN=...blaaa"
source s_BEATS {network(port(5140) flags(no-parse));} parser p_json { json-parser (prefix(".json.")); }; log { source(s_BEATS); parser(p_json); destination (d_file); };
Anyone have a howto or blog for using syslog-ng with json inputs ? I'm looking at the syslog-ng-ose-latest-guides but it's hard to put all the input output and parser requirements together.
Trying to get here winlogbeat->syslog-ng->ES winlogbeat->syslog-ng->SPLUNKForwader winlogbeat->syslog-ng->/opt/syslog-ng/logs/$FROM_HOST.json
or winlogbeat->logstash->syslog-ng->ES ...
On Tue, May 9, 2017 at 3:27 AM, Fabien Wernli <wernli@in2p3.fr> wrote:
Hi,
On Mon, May 08, 2017 at 11:30:14PM +0000, Scot wrote:
I'm trying to find a solution that will let me mirror my beats data like syslog-ng lets me do with syslog traffic.
As far as I know those tools simply send the data over TCP in JSON format. If you just need to do routing using syslog-ng, you can simply use network source with flags(no-parse). If you need to process the data using syslog-ng, you'll also need the json-parser().
Cheers
____________________________________________________________ __________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support /documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
-- Evan Rempel erempel@uvic.ca Senior Systems Administrator 250.721.7691 <(250)%20721-7691> Data Centre Services, University Systems, University of Victoria
____________________________________________________________ __________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/? product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq