<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<!-- Template generated by Exclaimer Signature Manager Exchange Edition on 02:02:54 Friday, 20 January 2017 -->
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<style type="text/css">P.ImprintUniqueID {
MARGIN: 0cm 0cm 0pt
}
LI.ImprintUniqueID {
MARGIN: 0cm 0cm 0pt
}
DIV.ImprintUniqueID {
MARGIN: 0cm 0cm 0pt
}
TABLE.ImprintUniqueIDTable {
MARGIN: 0cm 0cm 0pt
}
DIV.Section1 {
page: Section1
}
</style>
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-GB" link="blue" vlink="purple">
<br>
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US">Worked out http mode only supported not long after receiving the mail from Fabien, and now all working (thanks!).
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US">A more “generalised” question – I’ve used syslog-ng for years as a network engineer to receive Cisco network device input and output
it to file, both as individual host data and also a collected “all” file on which I’ve used a very simple “swatch” implementation to both screen out noise, and also highlight interesting network events (routing convergence etc), as per config below. I’d like
to replicate this somewhat with the syslog-ng/ES/Kibana build I now have, but I’m wondering the best way of doing it – should I filter “non-interesting” traffic at the syslog-ng level (if so, what is the best practice?) or do so at the Kibana level? In terms
of transportation from syslog-ng into ES, does anyone have any tips or pointers as to the best way of formatting Cisco switch/firewall/router logs to best be utilised within ES/Kibana?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US">Thank you very much in advance.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<div></div>
</div>
<br>
<br>
<table class="ImprintUniqueIDTable" style="BORDER-COLLAPSE: collapse; WIDTH: 100%" cellspacing="0" cellpadding="0" border="0">
<tbody>
<tr>
<td><font style="font-family:Calibri;font-size:10pt;color:#595959;font-weight:bold;">Damian</font> <font style="font-family:Calibri;font-size:10pt;color:#595959;font-weight:bold;">Bell</font><br>
<font style="font-family:Calibri;font-size:10pt;color:#595959;">Infrastructure Engineer</font><font style="font-family:Calibri;font-size:10pt;color:#595959;"> |
</font><font style="font-family:Calibri;font-size:10pt;color:#595959;">Support</font><font style="font-family:Calibri;font-size:10pt;color:#595959;"> |
</font><font style="font-family:Calibri;font-size:10pt;color:#595959;">H Clarkson & Co Ltd</font></td>
</tr>
<tr>
<td><br>
<font style="font-family:Calibri;font-size:10pt;color:#404040;">Email: </font><span style="font-family:Calibri;font-size:10pt;color:#0070C0;"><a href="mailto:Damian.Bell@clarksons.com" title="Click to send email to Damian Bell" target="" style="font-family:Calibri;font-size:10pt;color:#0070C0;"><span style="font-family:Calibri; font-size:10pt; color:#0070C0;">Damian.Bell@clarksons.com</span></a></span></td>
</tr>
</tbody>
</table>
<br>
<br>
<div class="WordSection1">
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif"> syslog-ng [mailto:syslog-ng-bounces@lists.balabit.hu]
<b>On Behalf Of </b>Laszlo Budai<br>
<b>Sent:</b> 18 January 2017 05:40<br>
<b>To:</b> Syslog-ng users' and developers' mailing list <syslog-ng@lists.balabit.hu>; Fabien Wernli <wernli@in2p3.fr><br>
<b>Subject:</b> Re: [syslog-ng] Error initializing message pipeline;<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div id="compose">
<div>
<p class="MsoNormal">hi,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">we support ES5.x only via http mode.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">regards,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt">Laszlo Budai<o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal">_____________________________<br>
From: Scot <<a href="mailto:scotrn@gmail.com">scotrn@gmail.com</a>><br>
Sent: Wednesday, January 18, 2017 3:33 AM<br>
Subject: Re: [syslog-ng] Error initializing message pipeline;<br>
To: Syslog-ng users' and developers' mailing list <<a href="mailto:syslog-ng@lists.balabit.hu">syslog-ng@lists.balabit.hu</a>>, Fabien Wernli <<a href="mailto:wernli@in2p3.fr">wernli@in2p3.fr</a>><br>
<br>
<br>
<br>
<o:p></o:p></p>
<div>
<p class="MsoNormal">Is <span style="font-size:9.5pt">client-mode("transport") now supported with ES 5.1? I thought it was only http mode for ES 5. </span>
<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:9.5pt">I got pipeline error then switched to http thinking it was the transport mode. http worked fine. </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:9.5pt"> </span><o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">On Tue, Jan 17, 2017 at 9:58 AM, Fabien Wernli <<a href="mailto:wernli@in2p3.fr" target="_blank">wernli@in2p3.fr</a>> wrote:<o:p></o:p></p>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<p class="MsoNormal" style="margin-bottom:12.0pt">Hi Damian,<br>
<br>
You need to specify the location to your elasticsearch installation, i.e.<br>
where the .jar files are installed.<br>
If you're using the official packages from <a href="http://elastic.co" target="_blank">
elastic.co</a>, they are most likely<br>
located here: /usr/share/elasticsearch/lib/<br>
<br>
So your config ought to look like the following instead:<br>
<br>
source s_syslog { udp(ip(0.0.0.0) port(514)); };<br>
<br>
destination d_elastic {<br>
elasticsearch2(<br>
client-lib-dir("/usr/share/elasticsearch/lib/")<br>
index("syslog-ng_${YEAR}.${MONTH}.${DAY}")<br>
type("test")<br>
cluster("someserver")<br>
client-mode("transport")<br>
template("$(format-json --scope rfc5424 --scope nv-pairs --exclude DATE --key ISODATE)")<br>
time-zone("UTC")<br>
);<br>
};<br>
<br>
Moreover, you might want to set the destination's timezone to UTC too, or<br>
you'll have surprises in kibana around midnight UTC: time-zone("UTC")<br>
<br>
______________________________________________________________________________<br>
Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" target="_blank">
https://lists.balabit.hu/mailman/listinfo/syslog-ng</a><br>
Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" target="_blank">
http://www.balabit.com/support/documentation/?product=syslog-ng</a><br>
FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" target="_blank">http://www.balabit.com/wiki/syslog-ng-faq</a><o:p></o:p></p>
</blockquote>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><o:p> </o:p></p>
</div>
</div>
<p class="ImprintUniqueID"><font size="1"></p>
<hr>
</font><font face="Microsoft Sans Serif" size="1">This message is private and confidential. If you have received it in error, you are on notice of its status. Please notify us immediately by reply email and then delete this message from your system. Please
do not copy it or use it for any purposes, or disclose its contents to any other person: to do so could be a breach of confidence.<br>
<br>
</font><font face="Microsoft Sans Serif" size="1">Emails may be monitored.<br>
<br>
Details of Clarkson group companies and their regulators (where applicable) can be found at this url:
</font><a title="Disclosure" href="http://www.clarksons.com/disclosure/"><font face="Microsoft Sans Serif" size="1">Disclosure</font></a>
<hr>
<p></p>
</body>
</html>