<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
Hi Allen,</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
Maybe just I don't get it entirely, but I guess your issue is that $LOCATION $RULE nv-pairs are empty, because Wazuh log messages are not parsed correctly, right?</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
We can usually solve custom parsing issues with SCLs, which are built up by common filters/parsers according to the custom message format.</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
Some examples of these </div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
<ul>
<li>netskope parser, websense-parser, checkpoint-parser</li></ul>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
(you can find these in syslog-ng's install directory, in "<install_prefix>/share/syslog-ng/include/scl")</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
Some documentation about creating your on SCL:<br>
<a href="https://www.syslog-ng.com/technical-documents/doc/syslog-ng-open-source-edition/3.22/administration-guide/15#TOPIC-1209122" id="LPlnk386403">https://www.syslog-ng.com/technical-documents/doc/syslog-ng-open-source-edition/3.22/administration-guide/15#TOPIC-1209122</a><br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
If you can share an example log iof Wazuh, we can help how you should parse it.</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
As far as I see, it can log in "plain" and "JSON" formats, the latter sounds easy as well, as syslog-ng does have a json-parser.</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
<a href="https://documentation.wazuh.com/current/user-manual/reference/ossec-conf/logging.html" id="LPNoLP341220">https://documentation.wazuh.com/current/user-manual/reference/ossec-conf/logging.html</a><br>
</div>
<br>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
Regards,</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
Gabor</div>
<div id="appendonsend"></div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> syslog-ng <syslog-ng-bounces@lists.balabit.hu> on behalf of Allen Olivas <allen.olivas@infodefense.com><br>
<b>Sent:</b> Thursday, August 1, 2019 23:35<br>
<b>To:</b> Syslog-ng users' and developers' mailing list <syslog-ng@lists.balabit.hu><br>
<b>Subject:</b> [syslog-ng] Parse message fields for use as columns in MySQL</font>
<div> </div>
</div>
<style>
<!--
@font-face
{font-family:"Cambria Math"}
@font-face
{font-family:Calibri}
p.x_MsoNormal, li.x_MsoNormal, div.x_MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif}
a:link, span.x_MsoHyperlink
{color:#0563C1;
text-decoration:underline}
a:visited, span.x_MsoHyperlinkFollowed
{color:#954F72;
text-decoration:underline}
span.x_EmailStyle17
{font-family:"Calibri",sans-serif;
color:windowtext}
.x_MsoChpDefault
{font-family:"Calibri",sans-serif}
@page WordSection1
{margin:1.0in 1.0in 1.0in 1.0in}
div.x_WordSection1
{}
-->
</style>
<div lang="EN-US" link="#0563C1" vlink="#954F72">
<div style="background-color:#FFEB9C; width:100%; border-style:solid; border-color:#9C6500; border-width:1pt; padding:2pt; font-size:10pt; line-height:12pt; font-family:'Calibri'; color:Black; text-align:left">
<span style="color:#9C6500; font-weight:bold">CAUTION:</span> This email originated from outside of the organization. Do not follow guidance, click links, or open attachments unless you recognize the sender and know the content is safe.</div>
<br>
<div>
<div class="x_WordSection1">
<p class="x_MsoNormal">Hello, </p>
<p class="x_MsoNormal"> </p>
<p class="x_MsoNormal">I’m really scratching my head trying to make this work and thought maybe the community has experienced this before. I’m collecting logs from Wazuh and Syslog-NG. Those logs are sent from my Wazuh server with Sylog-NG configured to send
to my MySQL server. The Syslog-ng.conf file on the MySQL server is configured with a destination to mysql.
</p>
<p class="x_MsoNormal"> </p>
<p class="x_MsoNormal">In the declared the destination and list out columns and values.
</p>
<p class="x_MsoNormal"># MySQL define destination</p>
<p class="x_MsoNormal">destination d_mysql {</p>
<p class="x_MsoNormal">sql(</p>
<p class="x_MsoNormal">type(mysql)</p>
<p class="x_MsoNormal">username("syslog")</p>
<p class="x_MsoNormal">password("xxxxxxx")</p>
<p class="x_MsoNormal">database("syslog")</p>
<p class="x_MsoNormal">host("127.0.0.1")</p>
<p class="x_MsoNormal">table("logs")</p>
<p class="x_MsoNormal">columns("host", "id", "location", "facility", "rule", "priority", "level", "tag", "datetime", "program", "msg")</p>
<p class="x_MsoNormal">values("$HOST", "$ID", "$LOCATION","$FACILITY", "$RULE", "$PRIORITY", "$LEVEL", "$TAG","$YEAR-$MONTH-$DAY $HOUR:$MIN:$SEC","$PROGRAM", "$MSG")</p>
<p class="x_MsoNormal">indexes("datetime", "host", "id", "location", "rule")</p>
<p class="x_MsoNormal">);</p>
<p class="x_MsoNormal">};</p>
<p class="x_MsoNormal"> </p>
<p class="x_MsoNormal">So here’s the problem. The Message data contains information like Rule and Location that really equate to the Wazuh Rule and Location = the Wazuh Agent that’s reporting it. I had hoped “location” column would populate with the Location
date from the message. Same with Rule. </p>
<p class="x_MsoNormal"> </p>
<p class="x_MsoNormal">SO my question to the community is how on earth do I parse the data in the message field to populate columns (existing or new)? Any thoughts, guidance, recommendations are greatly appreciated.
</p>
<p class="x_MsoNormal"> </p>
<p class="x_MsoNormal"><span style="font-size:10.0pt; font-family:"Arial",sans-serif; color:black">Thanks,</span></p>
<p class="x_MsoNormal"><span style="font-size:10.0pt; font-family:"Arial",sans-serif; color:black"><br>
</span><span style="font-size:12.0pt; font-family:"Arial",sans-serif; color:black">Allen Olivas</span><span style="font-size:10.0pt; font-family:"Arial",sans-serif; color:black"><br>
</span><b><i><span style="font-size:18.0pt; font-family:"Arial",sans-serif; color:#3A67B8">Info</span></i></b><b><i><span style="font-size:18.0pt; font-family:"Arial",sans-serif; color:#5F5F5F">Defense</span></i></b><span style="font-size:10.0pt; font-family:"Arial",sans-serif; color:black"><br>
Office: (972) 848-7910<br>
Email: allen.olivas@infodefense.com<br>
Toll Free: (877) INFODEFENSE<br>
</span><a href="https://nam05.safelinks.protection.outlook.com/?url=https%3A%2F%2Fl.shatrk.com%2Fr%2Fe%2FDblvLSPvKY2IxMPE%3Fr%3Dhttps%3A%2F%2Fapp.salesforceiq.com%2Fr%3Ftarget%3D5c77291cc9e77c007aa6cb3e%26t%3DAFwhZf0O7sC6c6N-x691ne-Q9q_27TNhu1ayis_kAJ00Z7HL-lH9bPLytoPohWYrCc5EpGO_mM--1dDX-GDgklCQ_2ZINq3F1wwLoCnz9aRhfWm9RG1fC4RVQcHYR5hMwHruEmd00J_U%26url%3Dhttp%3A%2F%2Fwww.infodefense.com%2F&data=02%7C01%7Cgabor.nagy%40oneidentity.com%7C06aa5cd3ebd14695670808d716c83cdb%7C91c369b51c9e439c989c1867ec606603%7C0%7C0%7C637002921605460940&sdata=3VdqqIxIp9Ls2xz8odH%2B%2FD6iHutUZ%2FzkmZ81mLN9Vxc%3D&reserved=0" originalsrc="https://l.shatrk.com/r/e/DblvLSPvKY2IxMPE?r=https://app.salesforceiq.com/r?target=5c77291cc9e77c007aa6cb3e&t=AFwhZf0O7sC6c6N-x691ne-Q9q_27TNhu1ayis_kAJ00Z7HL-lH9bPLytoPohWYrCc5EpGO_mM--1dDX-GDgklCQ_2ZINq3F1wwLoCnz9aRhfWm9RG1fC4RVQcHYR5hMwHruEmd00J_U&url=http://www.infodefense.com/" shash="wXq+azgdGm8qiJ55aDKfHhM3uwe9CFUf4agVT5ppUqNR2cqFMyAIdfn2fAFY8G5BL01/NPekAju2Dzj++/PRm8Xv4nruo5mkf7BR9sIgxri9In2pVOx82Wq514f6Fe8LPFnsMR7zIlTcVY8M9WPkMvFTiHG7wks9R/PWIjyl8Zg=" title="http://www.infodefense.com/"><span style="font-size:10.0pt; font-family:"Arial",sans-serif; color:blue">www.infodefense.com</span></a><span style="font-size:13.5pt; font-family:"Times New Roman",serif; color:black"><br>
<br>
</span><span style="font-size:10.0pt; font-family:"Arial",sans-serif; color:black"></span></p>
<p class="x_MsoNormal"> </p>
</div>
</div>
</div>
</body>
</html>