<div dir="ltr"><div dir="ltr"><div dir="ltr"><div>Currently I wrote a python script that logs to disk and sends some metrics via statsd, this only needs to be declared once for the entire apache/httpd instance and it works for all requests, including mod_proxy, mod_jk, and errors:<br></div><div><br></div><div>LogFormat "%v %h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-agent}i\"" combined_virtual<br>CustomLog "||/usr/local/bin/vlog.py access /var/log/httpd" combined_virtual<br>ErrorLog "||/usr/local/bin/vlog.py error /var/log/httpd"<br><br></div><div>This has worked fine for years and there are no real problems, I've just been thinking about a "better" way to do it. If i can ship logs to a centralized server *and* write to local disk then I could kill off my daily jobs to fetch and archive access logs.</div><div><br></div><div>The question i'm asking is if there's a better mechanism than piping to logger. I use logger for other projects and while it's a very old way of doing things it still works great -<br></div><div> </div><div>LogFormat "blahblahblah" my_custom_log_format</div><div>CustomLog "|exec /usr/bin/logger -t apache-access my_custom_log_format<br></div><div>ErrorLog "|exec /usr/bin/logger -t apache-error</div><div><br></div><div><br></div></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Mar 26, 2019 at 10:53 AM Evan Rempel <<a href="mailto:erempel@uvic.ca" target="_blank">erempel@uvic.ca</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF">
<div class="gmail-m_1929721981519321611gmail-m_-7300310234177618046moz-cite-prefix">I agree that it is much preferred to
NOT use apache's built in file logging. Apache can log its error
log to syslog ... which is great, but it can not log its access
log, or mod_jk log or others.</div>
<div class="gmail-m_1929721981519321611gmail-m_-7300310234177618046moz-cite-prefix"><br>
</div>
<div class="gmail-m_1929721981519321611gmail-m_-7300310234177618046moz-cite-prefix">We wanted the ability to be able to
recreate log files at a syslog destination for web site analytics
(which some tools require) so we wrote a small perl program that
takes</div>
<div class="gmail-m_1929721981519321611gmail-m_-7300310234177618046moz-cite-prefix">a couple of arguments for the log type
(access, error, mod_jk etc) and for the site name. Then we stick
them into the log with a syslog ident/tag of httpd and a message
of the form<br>
</div>
<div class="gmail-m_1929721981519321611gmail-m_-7300310234177618046moz-cite-prefix"><br>
</div>
<div class="gmail-m_1929721981519321611gmail-m_-7300310234177618046moz-cite-prefix">${log_type}: ${site_name}: {$message}</div>
<div class="gmail-m_1929721981519321611gmail-m_-7300310234177618046moz-cite-prefix"><br>
</div>
<div class="gmail-m_1929721981519321611gmail-m_-7300310234177618046moz-cite-prefix">where the message is the full message
produced by apache using the normal apache log format
specification.</div>
<div class="gmail-m_1929721981519321611gmail-m_-7300310234177618046moz-cite-prefix"><br>
</div>
<div class="gmail-m_1929721981519321611gmail-m_-7300310234177618046moz-cite-prefix">One of the advantages of using live
central logging like this is that I can "see/watch" all of the
logs for a "site_name" when the site is being run on a load
balanced pool of apache servers.</div>
<div class="gmail-m_1929721981519321611gmail-m_-7300310234177618046moz-cite-prefix">The other advantage is that I can
stream these to my analytics system and generate log files of the
original apache format</div>
<div class="gmail-m_1929721981519321611gmail-m_-7300310234177618046moz-cite-prefix"><br>
</div>
<div class="gmail-m_1929721981519321611gmail-m_-7300310234177618046moz-cite-prefix">access_log-${site_name}.log</div>
<div class="gmail-m_1929721981519321611gmail-m_-7300310234177618046moz-cite-prefix">error_log-${site_name}.log</div>
<div class="gmail-m_1929721981519321611gmail-m_-7300310234177618046moz-cite-prefix"><br>
</div>
<div class="gmail-m_1929721981519321611gmail-m_-7300310234177618046moz-cite-prefix">which can then be consumed by the
analytics engine.<br>
</div>
<div class="gmail-m_1929721981519321611gmail-m_-7300310234177618046moz-cite-prefix"><br>
</div>
<div class="gmail-m_1929721981519321611gmail-m_-7300310234177618046moz-cite-prefix">Just my $0.02 for anyone that likes it
:-)</div>
<div class="gmail-m_1929721981519321611gmail-m_-7300310234177618046moz-cite-prefix"><br>
</div>
<div class="gmail-m_1929721981519321611gmail-m_-7300310234177618046moz-cite-prefix">Evan<br>
</div>
<div class="gmail-m_1929721981519321611gmail-m_-7300310234177618046moz-cite-prefix"><br>
</div>
<div class="gmail-m_1929721981519321611gmail-m_-7300310234177618046moz-cite-prefix">On 3/26/19 7:14 AM, Nik Ambrosch wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div>I would like to replace apache's file logger entirely,
using syslog-ng to write to both network and local disk
instead of just using apache to write to disk and syslog-ng to
write to network.</div>
<div><br>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Tue, Mar 26, 2019 at 10:08
AM Balazs Scheidler <<a href="mailto:bazsi77@gmail.com" target="_blank">bazsi77@gmail.com</a>> wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="auto">I would follow the files using the
wildcard-file() source, possibly with marking them up with
apache specific name-value pairs that we extract from
filenames and/or content.</div>
<br>
<div class="gmail_quote">
<div dir="ltr">On Tue, Mar 26, 2019, 06:48 Nik Ambrosch <<a href="mailto:nik@ambrosch.com" target="_blank">nik@ambrosch.com</a> wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div>How are people managing their apache access logs
using syslog-ng these days - still just piping through
logger? I'm looking to take over logging entirely
using syslog-ng, not just reading files from disk and
sending to ES or something.<br>
</div>
<div><br>
</div>
<div>In the past I've used cronolog, which works fine,
but I'd love the flexibility of sending the logs
through syslog-ng.<br>
</div>
</div>
</blockquote>
</div>
</blockquote>
</div>
</blockquote>
<br>
</div>
______________________________________________________________________________<br>
Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" rel="noreferrer" target="_blank">https://lists.balabit.hu/mailman/listinfo/syslog-ng</a><br>
Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" rel="noreferrer" target="_blank">http://www.balabit.com/support/documentation/?product=syslog-ng</a><br>
FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" rel="noreferrer" target="_blank">http://www.balabit.com/wiki/syslog-ng-faq</a><br>
<br>
</blockquote></div>