<div dir="ltr"><span class="gmail-im" style="font-size:12.8px"><div><div class="gmail_extra"><div class="gmail_quote">Hi, </div><div class="gmail_quote"><br></div><div class="gmail_quote">Thanks your answer.</div><div class="gmail_quote"><br></div><div class="gmail_quote"><br></div><div class="gmail_quote">On Jun 8, 2017 08:38, "Sandbox" <<a href="mailto:sandboxheh@gmail.com" target="_blank">sandboxheh@gmail.com</a>> wrote:<br type="attribution"><blockquote class="gmail-m_-725075846151490485quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Hi,<div><br></div><div>I just started to test, learn etc syslog-ng, my server configuration is really basic:</div><div><br></div><div>Q: Can i filter (and mark them on client) the incoming logs, so i dont have to open multiple ports for different logs?</div></div></blockquote></div></div></div><div dir="auto"><br></div></span><div dir="auto" style="font-size:12.8px">| Sure, you can open one port and have it filtered using source ip (netmask() filter), embedded hostname (host() filter) or even message content.</div><div dir="auto" style="font-size:12.8px"><br></div><div style="font-size:12.8px">I made some filters, eg:</div><div style="font-size:12.8px"><div style="font-size:12.8px"><br></div><div style="font-size:12.8px">filter testsrv_apache_access {</div><div style="font-size:12.8px"> match("apache-access-log")</div><div style="font-size:12.8px">};</div><div style="font-size:12.8px"><br></div><div style="font-size:12.8px">Q: i tried to use the "program" filter, but for some reason would't work, as you mentioned im using <span style="font-size:12.8px">program-override("apache-</span><wbr style="font-size:12.8px"><span style="font-size:12.8px">access-log") on the client and sat up the </span><span style="font-size:12.8px">apache-</span><wbr style="font-size:12.8px"><span style="font-size:12.8px">access-log as filter match on the server side. With this setting it complains about missing value setting.</span></div></div><span class="gmail-im" style="font-size:12.8px"><div dir="auto"><br></div><div dir="auto"><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail-m_-725075846151490485quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><br></div><div><br></div><div>The stored log:</div><div><br></div><div>Q: Why does it store the date 3 times in every logs?</div></div></blockquote></div></div></div><div dir="auto"><br></div></span><div dir="auto" style="font-size:12.8px">| You seem to have received an rfc5424 formatted message, but it was not parsed, maybe because you were using the wrong source driver (syslog() is the one that should handle this format).</div><div dir="auto" style="font-size:12.8px">|</div><div dir="auto" style="font-size:12.8px">| Since it wasnt parsed, syslog-ng assumed the entire line is a $MSG, and prepended its own syslog header. Also, apache itself contains date as well.</div><div dir="auto" style="font-size:12.8px">|</div><div dir="auto" style="font-size:12.8px">| The solution depends on your exact use case. If you want to transport non-syslog data (like apache.log), you'll probably want to dedicate a port to it (so it doesnt mix syslog), or you make sure you | can identify it on the server side.</div><div dir="auto" style="font-size:12.8px"><br></div><div dir="auto" style="font-size:12.8px">| E.g.</div><div dir="auto" style="font-size:12.8px">|</div><div dir="auto" style="font-size:12.8px">| source { file("/var/log/apache/access.<wbr>log" host-override("hostname") program-override("apache-<wbr>access-log") flags(no-parse))); };</div><div dir="auto" style="font-size:12.8px">|</div><div dir="auto" style="font-size:12.8px">| This would read the log file without parsing it, adds $HOST and $PROGRAM fields, which would otherwise be missing.</div><div dir="auto" style="font-size:12.8px">|</div><div dir="auto" style="font-size:12.8px">| Then:</div><div dir="auto" style="font-size:12.8px">|</div><div dir="auto" style="font-size:12.8px">| * send it on to the server using whatever means (tcp and syslog both works), on the wire, the syslog header will be prepended.</div><div dir="auto" style="font-size:12.8px"><br></div><div style="font-size:12.8px">I sat up tcp driver and it stopped to send any log to the server. :)</div><div dir="auto" style="font-size:12.8px"><br></div><div dir="auto" style="font-size:12.8px">|</div><div dir="auto" style="font-size:12.8px">| * On the server, identify that these are apache logs (based on the $PROGRAM value), then write a file using a custom template, where you only use $MSG:</div><div dir="auto" style="font-size:12.8px">|</div><div dir="auto" style="font-size:12.8px">|file("logfile" template("$MSG\n"));</div><div dir="auto" style="font-size:12.8px"><br></div><div style="font-size:12.8px">I am still got this: <13>1 2017-06-08T14:53:54+02:00 testsrv_access apache-access-log - - - 192.168.56.48 - - [08/Jun/2017:14:53:54 +0200] "GET /index.php HTTP/1.1" 304 -</div><div style="font-size:12.8px"><br></div><div style="font-size:12.8px"><br></div><div dir="auto" style="font-size:12.8px">| This would remove the syslog header in your output file.</div><div dir="auto" style="font-size:12.8px">| Hope this helps</div><div dir="auto" style="font-size:12.8px">| Bazsi</div><div dir="auto" style="font-size:12.8px"><br></div><div style="font-size:12.8px">Thanks, Robert</div></div><div class="gmail_extra"><br><div class="gmail_quote">2017-06-08 10:19 GMT+02:00 Scheidler, Balázs <span dir="ltr"><<a href="mailto:balazs.scheidler@balabit.com" target="_blank">balazs.scheidler@balabit.com</a>></span>:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="auto"><span class=""><div><br><div class="gmail_extra"><br><div class="gmail_quote">On Jun 8, 2017 08:38, "Sandbox" <<a href="mailto:sandboxheh@gmail.com" target="_blank">sandboxheh@gmail.com</a>> wrote:<br type="attribution"><blockquote class="m_-725075846151490485quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi,<div><br></div><div>I just started to test, learn etc syslog-ng, my server configuration is really basic:</div><div><br></div><div>Q: Can i filter (and mark them on client) the incoming logs, so i dont have to open multiple ports for different logs?</div></div></blockquote></div></div></div><div dir="auto"><br></div></span><div dir="auto">Sure, you can open one port and have it filtered using source ip (netmask() filter), embedded hostname (host() filter) or even message content.</div><span class=""><div dir="auto"><br></div><div dir="auto"><div class="gmail_extra"><div class="gmail_quote"><blockquote class="m_-725075846151490485quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><br></div><div><br></div><div>The stored log:</div><div><br></div><div>Q: Why does it store the date 3 times in every logs?</div></div></blockquote></div></div></div><div dir="auto"><br></div></span><div dir="auto">You seem to have received an rfc5424 formatted message, but it was not parsed, maybe because you were using the wrong source driver (syslog() is the one that should handle this format).</div><div dir="auto"><br></div><div dir="auto">Since it wasnt parsed, syslog-ng assumed the entire line is a $MSG, and prepended its own syslog header. Also, apache itself contains date as well.</div><div dir="auto"><br></div><div dir="auto">The solution depends on your exact use case. If you want to transport non-syslog data (like apache.log), you'll probably want to dedicate a port to it (so it doesnt mix syslog), or you make sure you can identify it on the server side.</div><div dir="auto"><br></div><div dir="auto">E.g.</div><div dir="auto"><br></div><div dir="auto">source { file("/var/log/apache/access.<wbr>log" host-override("hostname") program-override("apache-<wbr>access-log") flags(no-parse))); };</div><div dir="auto"><br></div><div dir="auto">This would read the log file without parsing it, adds $HOST and $PROGRAM fields, which would otherwise be missing.</div><div dir="auto"><br></div><div dir="auto">Then:</div><div dir="auto"><br></div><div dir="auto">* send it on to the server using whatever means (tcp and syslog both works), on the wire, the syslog header will be prepended.</div><div dir="auto"><br></div><div dir="auto">* On the server, identify that these are apache logs (based on the $PROGRAM value), then write a file using a custom template, where you only use $MSG:</div><div dir="auto"><br></div><div dir="auto">file("logfile" template("$MSG\n"));</div><div dir="auto"><br></div><div dir="auto">This would remove the syslog header in your output file.</div><div dir="auto">Hope this helps</div><div dir="auto">Bazsi</div><span class=""><div dir="auto"><br></div><div dir="auto"><br></div><div dir="auto"><br></div><div dir="auto"><br></div><div dir="auto"><div class="gmail_extra"><div class="gmail_quote"><blockquote class="m_-725075846151490485quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><br></div><div>Jun 8 08:20:11 192.168.7.30 133 <13>1 2017-06-08T08:20:11+02:00 testweb01 - - - [meta sequenceId="24"] :1 - - [08/Jun/2017:08:20:10 +0200] "GET / HTTP/1.1" 200 3004<br></div><div><br></div><div><br></div><div><br></div></div></blockquote></div></div></div></span></div>
<br>______________________________<wbr>______________________________<wbr>__________________<br>
Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" rel="noreferrer" target="_blank">https://lists.balabit.hu/<wbr>mailman/listinfo/syslog-ng</a><br>
Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" rel="noreferrer" target="_blank">http://www.balabit.com/<wbr>support/documentation/?<wbr>product=syslog-ng</a><br>
FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" rel="noreferrer" target="_blank">http://www.balabit.com/wiki/<wbr>syslog-ng-faq</a><br>
<br>
<br></blockquote></div><br></div>