<div dir="ltr">Hi, in OSE 3.9 and later there is a dedicated apache parser: <a href="https://www.balabit.com/documents/syslog-ng-ose-latest-guides/en/syslog-ng-ose-guide-admin/html/apache-access-log-parser.html">https://www.balabit.com/documents/syslog-ng-ose-latest-guides/en/syslog-ng-ose-guide-admin/html/apache-access-log-parser.html</a><div><br></div><div>You might want to try it.</div><div><br></div><div>HTH, </div><div><br></div><div>Robert</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Jul 11, 2017 at 3:11 PM, Filipe Cifali <span dir="ltr"><<a href="mailto:cifali.filipe@gmail.com" target="_blank">cifali.filipe@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div><div>Hi all,<br><br></div>reading the docs I got into this config:<br><br>source s_apache_access_log { <wbr> <wbr> <wbr> <br> file( <wbr> <wbr> <wbr> <wbr> <br> "/var/logs/apache2/access_log"<wbr> <wbr> <wbr> <wbr> <br> follow-freq(1) <wbr> <wbr> <wbr> <wbr> <br> flags(no-parse) <wbr> <wbr> <wbr> <wbr> <br> ); <wbr> <wbr> <wbr> <wbr> <br>}; <wbr> <wbr> <wbr> <wbr> <br> <wbr> <wbr> <wbr> <wbr> <br>filter f_apache_access_log { <wbr> <wbr> <wbr> <br> match( <wbr> <wbr> <wbr> <wbr> <br> '(.*) (.*) - - \[[0-9]{2}\/[A-Z][a-z]{2}\/[0-<wbr>9]{4}:[0-9]{2}:[0-9]{2}:[0-9]{<wbr>2} -0300\] \"(.*) (.*) (.*)\" (.*) (.*) \"-\" (.*)' <br> type("pcre") <wbr> <wbr> <wbr> <wbr> <br> flags("store-matches") <wbr> <wbr> <wbr> <wbr> <br> ); <wbr> <wbr> <wbr> <wbr> <br>}; <wbr> <wbr> <wbr> <wbr> <br> <wbr> <wbr> <wbr> <wbr> <br>rewrite r_apache_access_log { <wbr> <wbr> <wbr> <br> set("$1", value("DOMAIN") condition(filter(f_apache_<wbr>access_log))); <wbr> <wbr> <wbr> <br> set("$2", value("IP") condition(filter(f_apache_<wbr>access_log))); <wbr> <wbr> <wbr> <br> set("$3", value("HTTP_METHOD") condition(filter(f_apache_<wbr>access_log))); <wbr> <wbr> <br> set("$4", value("URI") condition(filter(f_apache_<wbr>access_log))); <wbr> <wbr> <wbr> <br> set("$6", value("HTTP_STATUS") condition(filter(f_apache_<wbr>access_log))); <wbr> <wbr> <br> set("$7", value("SIZE") condition(filter(f_apache_<wbr>access_log))); <wbr> <wbr> <wbr> <br> set("$8", value("USER_AGENT") condition(filter(f_apache_<wbr>access_log))); <wbr> <wbr> <br>}; <wbr> <wbr> <wbr> <wbr> <br> <wbr> <wbr> <wbr> <wbr> <br>destination d_apache_access_log { <wbr> <wbr> <wbr> <br> mongodb( <wbr> <wbr> <wbr> <wbr> <br> # <a href="https://docs.mongodb.com/manual/reference/connection-string/" target="_blank">https://docs.mongodb.com/<wbr>manual/reference/connection-<wbr>string/</a> <wbr> <wbr> <br> persist-name("apache-access-<wbr>logs") <wbr> <wbr> <wbr> <br> uri("mongodb://$server_and_<wbr>port/syslog?wtimeoutMS=60000&<wbr>socketTimeoutMS=60000&<wbr>connectTimeoutMS=60000") <wbr> <br> collection("logs") <wbr> <wbr> <wbr> <wbr> <br> retries(3600) <wbr> <wbr> <wbr> <wbr> <br> value-pairs( <wbr> <wbr> <wbr> <wbr> <br> pair("HOST", "${HOST}") <wbr> <wbr> <wbr> <wbr> <br> pair("SERVICE", "APACHE") <wbr> <wbr> <wbr> <br> pair("DATE", "${DAY}/${MONTH}/${YEAR}") <wbr> <wbr> <wbr> <wbr> <br> pair("TIME", "${HOUR}:${MIN}") <wbr> <wbr> <wbr> <wbr> <br> pair("MESSAGE", "${MESSAGE}") <wbr> <wbr> <wbr> <br> pair("DOMAIN", "${DOMAIN}") <wbr> <wbr> <wbr> <br> pair("HTTP_STATUS", "${HTTP_STATUS}") <wbr> <wbr> <wbr> <br> pair("HTTP_METHOD", "${HTTP_METHOD}") <wbr> <wbr> <wbr> <br> pair("USER_AGENT", "${USER_AGENT}") <wbr> <wbr> <wbr> <br> pair("SIZE", "${SIZE}") <wbr> <wbr> <wbr> <wbr> <br> pair("URI", "${URI}") <wbr> <wbr> <wbr> <wbr> <br> pair("IP", "${IP}") <wbr> <wbr> <wbr> <wbr> <br> ) <wbr> <wbr> <wbr> <wbr> <br> ); <wbr> <wbr> <wbr> <wbr> <br>}; <wbr> <wbr> <wbr> <wbr> <br> <wbr> <wbr> <wbr> <wbr> <br>log { <wbr> <wbr> <wbr> <wbr> <br> source(s_apache_access_log); <wbr> <wbr> <wbr> <wbr> <br> filter(f_apache_access_log); <wbr> <wbr> <wbr> <wbr> <br> rewrite(r_apache_access_log); <wbr> <wbr> <wbr> <wbr> <br> destination(d_apache_access_<wbr>log); <wbr> <wbr> <wbr> <br>}; <wbr> <wbr> <wbr> <wbr> <br><br></div>but I think something is not ok, I'm not sure this is the right way to do it.<br><br></div>This log produces an strange behavior: <br><br><a href="http://www.cifa.li" target="_blank">www.cifa.li</a> 127.0.0.1 - - [11/Jul/2017:09:18:56 -0300] "GET / HTTP/1.1" 200 18652 "<a href="http://cifa.li/" target="_blank">http://cifa.li/</a>" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:54.0) Gecko/20100101 Firefox/54.0"<br clear="all"><div><div><div><div><div><div><div><br></div><div>but this one doesn't<br><br><a href="http://cifa.li" target="_blank">cifa.li</a> 127.0.0.1 - - [11/Jul/2017:09:18:56 -0300] "GET / HTTP/1.1"
200 18652 "<a href="http://cifa.li/" target="_blank">http://cifa.li/</a>" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64;
rv:54.0) Gecko/20100101 Firefox/54.0"</div><div><br></div><div>The behavior is (only for subdomains):<br><br></div><div>DOMAIN: ': <a href="http://www.cifa.li" target="_blank">www.cifa.li</a>' <br><br></div><div>corret one<br><br></div><div>DOMAIN: '<a href="http://www.cifa.li" target="_blank">www.cifa.li</a>'<br><br></div><div>The subdomain seems like it's adding stuff that I didn't (or want) to add.<br><br><br></div><div>Am I missing something?<br><br></div><div>Thanks in advance.<span class="HOEnZb"><font color="#888888"><br></font></span></div><span class="HOEnZb"><font color="#888888"><div><br></div><div><br>-- <br><div class="m_-8953235465465869119gmail_signature">[ ]'s<br><br>Filipe Cifali Stangler<br></div>
</div></font></span></div></div></div></div></div></div></div>
<br>______________________________<wbr>______________________________<wbr>__________________<br>
Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" rel="noreferrer" target="_blank">https://lists.balabit.hu/<wbr>mailman/listinfo/syslog-ng</a><br>
Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" rel="noreferrer" target="_blank">http://www.balabit.com/<wbr>support/documentation/?<wbr>product=syslog-ng</a><br>
FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" rel="noreferrer" target="_blank">http://www.balabit.com/wiki/<wbr>syslog-ng-faq</a><br>
<br>
<br></blockquote></div><br></div>