<div dir="ltr"><div><div><div><div><div><div><div><div><div>Hi,<br><br></div>I meant the c source code for json-parser(), which is modules/json/json-parser.c, more specifically the json_parser_process() function:<br><br>```<br>static gboolean<br>json_parser_process(LogParser *s, LogMessage **pmsg, const LogPathOptions *path_options, const gchar *input, gsize input_len)<br>{<br> JSONParser *self = (JSONParser *) s;<br> struct json_object *jso;<br> struct json_tokener *tok;<br><br> if (self->marker)<br> {<br> if (strncmp(input, self->marker, self->marker_len) != 0)<br> return FALSE;<br> input += self->marker_len;<br><br> while (isspace(*input))<br> input++;<br> }<br><br> tok = json_tokener_new();<br> jso = json_tokener_parse_ex(tok, input, input_len);<br> if (tok->err != json_tokener_success || !jso)<br> {<br> msg_error("Unparsable JSON stream encountered",<br> evt_tag_str ("input", input),<br> tok->err != json_tokener_success ? evt_tag_str ("error", json_tokener_error_desc(tok->err)) : NULL,<br> NULL);<br> json_tokener_free (tok);<br> return FALSE;<br> }<br> json_tokener_free(tok);<br><br> log_msg_make_writable(pmsg, path_options);<br> if (!json_parser_extract(self, jso, *pmsg))<br> {<br> msg_error("Error extracting JSON members into LogMessage as the top-level JSON object is not an object",<br> evt_tag_str ("input", input),<br> NULL);<br> json_object_put(jso);<br> return FALSE;<br> }<br> json_object_put(jso);<br><br> return TRUE;<br>}<br><br>```<br><br></div>As it seems, there are three cases where json-parser() drops messages:<br></div>1) a marker() option is specified and the message doesn't start with that value<br></div>2) json parse error, in which case an error is printed<br></div>3) if the top-level json object in the input is not an Object (but a list or a single value), but again an error message is printed in this case too<br><br></div>I didn't see the marker option in your configuration, so either of the error cases should be logged as an internal message, at the error level.<br><br></div>Can you check that?<br><br></div>Cheers,<br></div>Bazsi<br><br><div><div><div><div><div><div><div><div><div><br></div></div></div></div></div></div></div></div></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Jul 7, 2015 at 8:48 PM, Russell Fulton <span dir="ltr"><<a href="mailto:r.fulton@auckland.ac.nz" target="_blank">r.fulton@auckland.ac.nz</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class=""><br>
> On 7 Jul 2015, at 08:57, Scheidler, Balázs <<a href="mailto:balazs.scheidler@balabit.com">balazs.scheidler@balabit.com</a>> wrote:<br>
><br>
> Seems that jsonparser drops messages for some reason.<br>
> I don't have the code handy but perhaps you could check the error cases.<br>
<br>
</span>sorry, not sure what you mean by 'check the error cases’. There is nothing I can find in /var/log<br>
<br>
BTW This is a corporate managed RHEL box that runs rsyslog for ‘normal’ logging. I am running syslog-ng listening on non standard ports as I have done on other systems.<br>
<span class=""><br>
><br>
> It should report failures though.<br>
<br>
</span>Let me know if there is anything I can do to help diagnose this. I am keen to get this going.<br>
<br>
Is there any docs apart from the section in the user manual? If anyone can point me to a non trivial example (including using pattern parsing) that would be really helpful.<br>
<span class="HOEnZb"><font color="#888888"><br>
Russell<br>
</font></span><div class="HOEnZb"><div class="h5">______________________________________________________________________________<br>
Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" rel="noreferrer" target="_blank">https://lists.balabit.hu/mailman/listinfo/syslog-ng</a><br>
Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" rel="noreferrer" target="_blank">http://www.balabit.com/support/documentation/?product=syslog-ng</a><br>
FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" rel="noreferrer" target="_blank">http://www.balabit.com/wiki/syslog-ng-faq</a><br>
<br>
</div></div></blockquote></div><br><br clear="all"><br>-- <br><div class="gmail_signature">Bazsi</div>
</div>