<div dir="ltr">you are setting template-escape(yes), which would turn apostrophes/quote characters into escaped sequences. turn that off.<div><br></div><div>also, it would be much better to use $(format-json) as Antal suggests.</div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Apr 21, 2020 at 11:57 AM Arthur Lambert <<a href="mailto:lambertarthur22@gmail.com">lambertarthur22@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Hello,<br><br>I tried to migrate from syslog to Syslog-ng. I am trying to deploy it on an embedded target<br>with buildroot. At the same time, I also tried to switch log format to JSON.<br><br>In my c code, I use Syslog function and init with the openlog function:<br>openlog("middleware", 0, LOG_LOCAL1);<br><br>I have some macro to generate the json logs:<br>#define _PRINT_DEBUG(M, ...) \<br>do { \<br>syslog(LOG_DEBUG, "\"category\":\"%s\",\"level\":\"DEBUG\",\"message\":\"" M "%s\"}\n", __DIR__, __VA_ARGS__); \<br>} while (0)<br><br>I define a syslog-ng conf file:<br><br>@version: 3.2X<br><br>filter f_middleware {<br>    facility("local1");<br>};<br><br>template t_json_filetemplate {<br>    template("{\"timestamp\":\"${ISODATE}\",${MESSAGE}\n");<br>};<br><br>source s_kernel {<br>    file("/proc/kmsg" program_override("kernel"));<br>};<br><br>source s_middleware {<br>    unix-stream("/dev/log");<br>};<br><br>destination d_kernel {<br>    file("/data/logs/kern.log");<br>};<br><br>destination d_middleware {<br>    file("/data/logs/middleware.log", template(t_json_filetemplate));<br>};<br><br>log {<br>    source(s_kernel);<br>    destination(d_kernel);<br>};<br><br>log {<br>    source(s_middleware);<br>    filter(f_middleware);<br>    destination(d_middleware);<br>};<br><br>So basically I want to redirect all my log from my code in /data/logs/middleware.log. I also want to redirect all kernel logs<br>in /data/logs/kern.log<br><br>I have two issues. The first one is escaping.<br>If I want to print a JSON file the double quote will be not escaped. My first solution was to use escape feature from Syslog-ng:<br>destination d_middleware {<br>        file("/data/logs/middleware.log", template("{\"timestamp\":\"${ISODATE}\",${MESSAGE}\n") template-escape(yes));                                          <br><br>};<br><br>Problem is that now everything is escaped and not only the message, some example:<br>{"timestamp":"2020-04-21T09:24:02+02:00",\"category\":\"BOOT\",\"level\":\"INFO\",\"message\":\"Check pstore partition..\"}<br>{"timestamp":"2020-04-21T09:24:02+02:00",\"category\":\"BOOT\",\"level\":\"INFO\",\"message\":\"Start hardware watchdog daemon...\"}<br>{"timestamp":"2020-04-21T09:24:02+02:00",\"category\":\"BOOT\",\"level\":\"INFO\",\"message\":\"Prepare asound configuration...\"}<br>{"timestamp":"2020-04-21T09:24:02+02:00",\"category\":\"BOOT\",\"level\":\"INFO\",\"message\":\"Middleware is about to start...\"}<br>{"timestamp":"2020-04-21T09:24:03+02:00",\"category\":\"BOOT\",\"level\":\"INFO\",\"message\":\"Load sdma firmware..\"}<br>{"timestamp":"2020-04-21T09:24:03+02:00",\"category\":\"BOOT\",\"level\":\"INFO\",\"message\":\"Watchdog is about to start...\"}<br>{"timestamp":"2020-04-21T09:24:03+02:00",\"category\":\"BOOT\",\"level\":\"INFO\",\"message\":\"Start mpu watchdog..\"}<br><br>As you can see, everything is escaped because for Syslog-ng, everything is the message and not only the final part. I feel like I am completely<br>using Syslog-ng in the wrong way right? How can I fix that? Do I have to handle escaping by myself manually in my middleware?<br>Do I have to pre-process all logs manually by myself? It seems quite awful.<br><br>I have a second issue which is probably related? When I tried to print the content of a file, the last '}' is truncated. The reason seems to be related<br>to the presence of \n. I can reproduce the issue by just calling my PRINT_DEBUG macro as follow:<br><br>PRINT_DEBUG ("YO YO YO KO\n");<br>PRINT_DEBUG ("YO YO YO OK");<br><br>The result:<br><br>{"timestamp":"2020-04-21T09:24:03+02:00",\"category\":\"MPU_CORE\",\"level\":\"DEBUG\",\"message\":\"YO YO YO KO<br>{"timestamp":"2020-04-21T09:24:03+02:00",\"category\":\"MPU_CORE\",\"level\":\"DEBUG\",\"message\":\"YO YO YO OK\"}<br><br>My syslog version/module information :<br><br># syslog-ng -V -s<br>syslog-ng 3 (3.26.1)<br>Config version: 3.22<br>Installer-Version: 3.26.1<br>Revision:<br>Compile-Date: Apr 10 2020 16:41:46<br>Module-Directory: /usr/lib/syslog-ng<br>Module-Path: /usr/lib/syslog-ng<br>Include-Path: /usr/share/syslog-ng/include<br>Available-Modules: add-contextual-data,affile,afprog,afsocket,afstomp,afuser,appmodel,azure-auth-header,basicfuncs,cef,confgen,cryptofuncs,csvparser,dbparser,disk-buffer,examples,graphite,hook-commands,ht<br>tp,kvformat,linux-kmsg-format,map-value-pairs,mod-python,pseudofile,snmptrapd-parser,stardate,syslogformat,system-source,tags-parser,tfgetent,timestamp,xml<br>Enable-Debug: off<br>Enable-GProf: off<br>Enable-Memtrace: off<br>Enable-IPv6: on<br>Enable-Spoof-Source: off<br>Enable-TCP-Wrapper: off<br>Enable-Linux-Caps: off<br>Enable-Systemd: off<br><br>--<br>- Arthur LAMBERT</div>
______________________________________________________________________________<br>
Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" rel="noreferrer" target="_blank">https://lists.balabit.hu/mailman/listinfo/syslog-ng</a><br>
Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" rel="noreferrer" target="_blank">http://www.balabit.com/support/documentation/?product=syslog-ng</a><br>
FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" rel="noreferrer" target="_blank">http://www.balabit.com/wiki/syslog-ng-faq</a><br>
<br>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature">Bazsi</div>