Hi, I meant the c source code for json-parser(), which is modules/json/json-parser.c, more specifically the json_parser_process() function: ``` static gboolean json_parser_process(LogParser *s, LogMessage **pmsg, const LogPathOptions *path_options, const gchar *input, gsize input_len) { JSONParser *self = (JSONParser *) s; struct json_object *jso; struct json_tokener *tok; if (self->marker) { if (strncmp(input, self->marker, self->marker_len) != 0) return FALSE; input += self->marker_len; while (isspace(*input)) input++; } tok = json_tokener_new(); jso = json_tokener_parse_ex(tok, input, input_len); if (tok->err != json_tokener_success || !jso) { msg_error("Unparsable JSON stream encountered", evt_tag_str ("input", input), tok->err != json_tokener_success ? evt_tag_str ("error", json_tokener_error_desc(tok->err)) : NULL, NULL); json_tokener_free (tok); return FALSE; } json_tokener_free(tok); log_msg_make_writable(pmsg, path_options); if (!json_parser_extract(self, jso, *pmsg)) { msg_error("Error extracting JSON members into LogMessage as the top-level JSON object is not an object", evt_tag_str ("input", input), NULL); json_object_put(jso); return FALSE; } json_object_put(jso); return TRUE; } ``` As it seems, there are three cases where json-parser() drops messages: 1) a marker() option is specified and the message doesn't start with that value 2) json parse error, in which case an error is printed 3) if the top-level json object in the input is not an Object (but a list or a single value), but again an error message is printed in this case too I didn't see the marker option in your configuration, so either of the error cases should be logged as an internal message, at the error level. Can you check that? Cheers, Bazsi On Tue, Jul 7, 2015 at 8:48 PM, Russell Fulton <r.fulton@auckland.ac.nz> wrote:
On 7 Jul 2015, at 08:57, Scheidler, Balázs <balazs.scheidler@balabit.com> wrote:
Seems that jsonparser drops messages for some reason. I don't have the code handy but perhaps you could check the error cases.
sorry, not sure what you mean by 'check the error cases’. There is nothing I can find in /var/log
BTW This is a corporate managed RHEL box that runs rsyslog for ‘normal’ logging. I am running syslog-ng listening on non standard ports as I have done on other systems.
It should report failures though.
Let me know if there is anything I can do to help diagnose this. I am keen to get this going.
Is there any docs apart from the section in the user manual? If anyone can point me to a non trivial example (including using pattern parsing) that would be really helpful.
Russell
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
-- Bazsi