<div dir="ltr">And something else:<div>When I add to client configuration extra _SDATA fields, they appears on kibana parsed correctly. <a href="http://pastebin.com/6ZKVTwVS">http://pastebin.com/6ZKVTwVS</a></div><div><br></div><div>So I assume that syslog-ng passes fields correctly to elastic. </div></div><div class="gmail_extra"><br clear="all"><div><div class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><p><span lang="EN-US" style="font-family:Verdana,sans-serif;color:rgb(136,136,136)">-- <br></span><b><span lang="EN-US" style="font-size:13.5pt;font-family:Verdana,sans-serif;color:black">Jacek Drewniak</span></b><span lang="EN-US" style="font-family:Verdana,sans-serif;color:black"><br></span><font color="#000000" face="Verdana, sans-serif">R&D</font></p><p style="margin:0cm 0cm 0.0001pt;background-image:initial;background-repeat:initial"><b><span lang="PL" style="font-size:10pt;font-family:Verdana,sans-serif">email</span></b><span lang="PL" style="font-size:10pt;font-family:Verdana,sans-serif">: </span><span style="font-size:10pt;font-family:Verdana,sans-serif"><a href="mailto:jacek.drewniak@oort.in" target="_blank"><font color="#000000">jacek.drewniak@oort.in</font></a></span><span lang="PL" style="font-size:10pt;font-family:Arial,sans-serif"></span></p><p style="margin:0cm 0cm 0.0001pt;background-image:initial;background-repeat:initial"><b><span lang="PL" style="font-size:10pt;font-family:Verdana,sans-serif;color:black">mobile</span></b><span lang="PL" style="font-size:10pt;font-family:Verdana,sans-serif;color:black">: <u>+</u></span><span lang="PL" style="font-size:10pt;font-family:Verdana,sans-serif"><font color="#000000"><u>48 696 151 670</u></font></span><span lang="PL" style="font-size:10pt;font-family:Arial,sans-serif"></span></p><p style="margin:0cm 0cm 0.0001pt;background-image:initial;background-repeat:initial"></p><p style="margin:0cm 0cm 0.0001pt;background-image:initial;background-repeat:initial"><b><span style="font-size:10pt;font-family:Verdana,sans-serif;color:black">website</span></b><span style="font-size:10pt;font-family:Verdana,sans-serif"><font color="#000000">:</font><span style="color:black"> </span><a href="http://www.oort.in/" style="color:rgb(17,85,204)" target="_blank"><font color="#000000">www.oort.in</font></a></span><span style="font-size:10pt;font-family:Arial,sans-serif"></span></p><p style="margin:0cm 0cm 0.0001pt;background-image:initial;background-repeat:initial"><br></p><p><span lang="EN-US" style="font-family:Verdana,sans-serif;color:black"><img src="http://www.oort.in/oort-stuff/logo-mail2.png"><br></span></p><p><span style="color:rgb(153,153,153);font-family:verdana,sans-serif"><br></span></p><p><span style="color:rgb(153,153,153);font-family:verdana,sans-serif">AWARDS</span><br></p><p></p><p></p><p></p><p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><font face="verdana, sans-serif"><span style="color:rgb(153,153,153)">Bluetooth Breakthrough Award Finalist</span><br><span style="color:rgb(153,153,153)">CES 2015 </span><span style="color:rgb(153,153,153)">Envisioneering</span><i style="color:rgb(153,153,153)"> </i><span style="color:rgb(153,153,153)">Innovation & Design Award Winner</span><br><span style="color:rgb(153,153,153)">Tech Trailblazers Awards Winner</span><br><span style="color:rgb(153,153,153)">Most exciting company at Bluetooth Media Event in New York 2014</span><br><span style="color:rgb(153,153,153)">Polish Agency for Enterprise Development Award Winner</span></font><br></p></div></div></div></div></div>
<br><div class="gmail_quote">2015-08-14 15:44 GMT+02:00 Jacek Drewniak <span dir="ltr"><<a href="mailto:jacek.drewniak@oort.in" target="_blank">jacek.drewniak@oort.in</a>></span>:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Thanks for advises. <div><br></div><div>Now my configs:</div><div><a href="http://pastebin.com/G6S2YV6S" target="_blank">http://pastebin.com/G6S2YV6S</a></div><div><a href="http://pastebin.com/wCVc2hqH" target="_blank">http://pastebin.com/wCVc2hqH</a><br><div><br></div><div>Sending log: <a href="http://pastebin.com/Euhp1Lmz" target="_blank">http://pastebin.com/Euhp1Lmz</a></div><div>Now its is parsed: <a href="http://pastebin.com/x46pk4FF" target="_blank">http://pastebin.com/x46pk4FF</a><br></div></div><div>So this didn't help. </div><div><br></div><div>Yes, <span style="font-size:12.8000001907349px"> </span><span style="font-size:12.8000001907349px">"[TIMER]" part is also part of the message.</span></div><div><span style="font-size:12.8000001907349px"><br></span></div><div><span style="font-size:12.8000001907349px">@</span><span style="font-size:12.8000001907349px">Gyu I don't understand this part about length of message . Do You have link to documentation?</span></div><div><span style="font-size:12.8000001907349px"><br></span></div><div><br></div></div><div class="gmail_extra"><span class=""><br clear="all"><div><div><div dir="ltr"><div><div dir="ltr"><p><span lang="EN-US" style="font-family:Verdana,sans-serif;color:rgb(136,136,136)">-- <br></span><b><span lang="EN-US" style="font-size:13.5pt;font-family:Verdana,sans-serif;color:black">Jacek Drewniak</span></b><span lang="EN-US" style="font-family:Verdana,sans-serif;color:black"><br></span><font color="#000000" face="Verdana, sans-serif">R&D</font></p><p style="margin:0cm 0cm 0.0001pt;background-image:initial;background-repeat:initial"><b><span lang="PL" style="font-size:10pt;font-family:Verdana,sans-serif">email</span></b><span lang="PL" style="font-size:10pt;font-family:Verdana,sans-serif">: </span><span style="font-size:10pt;font-family:Verdana,sans-serif"><a href="mailto:jacek.drewniak@oort.in" target="_blank"><font color="#000000">jacek.drewniak@oort.in</font></a></span><span lang="PL" style="font-size:10pt;font-family:Arial,sans-serif"></span></p><p style="margin:0cm 0cm 0.0001pt;background-image:initial;background-repeat:initial"><b><span lang="PL" style="font-size:10pt;font-family:Verdana,sans-serif;color:black">mobile</span></b><span lang="PL" style="font-size:10pt;font-family:Verdana,sans-serif;color:black">: <u>+</u></span><span lang="PL" style="font-size:10pt;font-family:Verdana,sans-serif"><font color="#000000"><u>48 696 151 670</u></font></span><span lang="PL" style="font-size:10pt;font-family:Arial,sans-serif"></span></p><p style="margin:0cm 0cm 0.0001pt;background-image:initial;background-repeat:initial"></p><p style="margin:0cm 0cm 0.0001pt;background-image:initial;background-repeat:initial"><b><span style="font-size:10pt;font-family:Verdana,sans-serif;color:black">website</span></b><span style="font-size:10pt;font-family:Verdana,sans-serif"><font color="#000000">:</font><span style="color:black"> </span><a href="http://www.oort.in/" style="color:rgb(17,85,204)" target="_blank"><font color="#000000">www.oort.in</font></a></span><span style="font-size:10pt;font-family:Arial,sans-serif"></span></p><p style="margin:0cm 0cm 0.0001pt;background-image:initial;background-repeat:initial"><br></p><p><span lang="EN-US" style="font-family:Verdana,sans-serif;color:black"><img src="http://www.oort.in/oort-stuff/logo-mail2.png"><br></span></p><p><span style="color:rgb(153,153,153);font-family:verdana,sans-serif"><br></span></p><p><span style="color:rgb(153,153,153);font-family:verdana,sans-serif">AWARDS</span><br></p><p></p><p></p><p></p><p style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><font face="verdana, sans-serif"><span style="color:rgb(153,153,153)">Bluetooth Breakthrough Award Finalist</span><br><span style="color:rgb(153,153,153)">CES 2015 </span><span style="color:rgb(153,153,153)">Envisioneering</span><i style="color:rgb(153,153,153)"> </i><span style="color:rgb(153,153,153)">Innovation & Design Award Winner</span><br><span style="color:rgb(153,153,153)">Tech Trailblazers Awards Winner</span><br><span style="color:rgb(153,153,153)">Most exciting company at Bluetooth Media Event in New York 2014</span><br><span style="color:rgb(153,153,153)">Polish Agency for Enterprise Development Award Winner</span></font><br></p></div></div></div></div></div>
<br></span><div><div class="h5"><div class="gmail_quote">2015-08-14 15:10 GMT+02:00 PÁSZTOR György <span dir="ltr"><<a href="mailto:pasztor@linux.gyakg.u-szeged.hu" target="_blank">pasztor@linux.gyakg.u-szeged.hu</a>></span>:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi,<br>
<span><br>
"Jacek Drewniak" <<a href="mailto:jacek.drewniak@oort.in" target="_blank">jacek.drewniak@oort.in</a>> írta 2015-08-14 14:40-kor:<br>
> I am new in logging world.<br>
> I am formating my logs according to:<br>
> <a href="https://www.balabit.com/sites/default/files/documents/syslog-ng-ose-latest-guides/en/syslog-ng-ose-guide-admin/html/concepts-message-ietfsyslog.html" rel="noreferrer" target="_blank">https://www.balabit.com/sites/default/files/documents/syslog-ng-ose-latest-guides/en/syslog-ng-ose-guide-admin/html/concepts-message-ietfsyslog.html</a><br>
><br>
><br>
</span>> I am using *syslog* protocol.<br>
<span>><br>
> For example I am logging this: <a href="http://pastebin.com/4UtUYiJJ" rel="noreferrer" target="_blank">http://pastebin.com/4UtUYiJJ</a><br>
> But it is parsed to fields (I can see this on kibana) :<br>
> <a href="http://pastebin.com/cNX8PZJp" rel="noreferrer" target="_blank">http://pastebin.com/cNX8PZJp</a><br>
><br>
> Can You tell me what I am doing wrong?<br>
<br>
</span>Your format is not exactly the ietf syslog protocol's format.<br>
The beginning is okay, but:<br>
<15>1 2015-08-14T12:33:53Z jackahub oortApp - -<br>
<br>
Until this point it seems okay.<br>
And now the real but:<br>
"{_SDATA:{meta:{sequenceId:jackaSEQ,hubId:123456789}}"<br>
should be formatted in this way:<br>
[meta sequenceId="jackaSEQ" hubId="123456789"]<br>
<br>
Assuming that the "[TIMER]" part is also part of the message.<br>
<br>
Also, please care about the transport protocol.<br>
Eg. if your transfer this over tcp/tls channel, then you have to prefix the<br>
whole with the length of this message in bytes eg.<br>
print SOCK "".length($message)." ".$message;<br>
<br>
Cheers,<br>
Gyu<br>
<div><div>______________________________________________________________________________<br>
Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" rel="noreferrer" target="_blank">https://lists.balabit.hu/mailman/listinfo/syslog-ng</a><br>
Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" rel="noreferrer" target="_blank">http://www.balabit.com/support/documentation/?product=syslog-ng</a><br>
FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" rel="noreferrer" target="_blank">http://www.balabit.com/wiki/syslog-ng-faq</a><br>
<br>
</div></div></blockquote></div><br></div></div></div>
</blockquote></div><br></div>