<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
<title></title>
</head>
<body text="#000000" bgcolor="#ffffff">
Hello,<br>
<br>
On 11/18/2010 06:09 PM, Peter Gyongyosi wrote:
<blockquote cite="mid:4CE55DDB.1080204@balabit.hu" type="cite">
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
Hi,<br>
<br>
On 11/18/2010 08:31 AM, Peter Czanik wrote:
<blockquote cite="mid:4CE4D650.6060500@balabit.hu" type="cite">
<meta http-equiv="content-type"
content="text/html; charset=ISO-8859-1">
Hello,<br>
<br>
As part of the patterndb project, we plan to start a log sample
collecting project. At <a moz-do-not-send="true"
class="moz-txt-link-freetext"
href="http://czanik.blogs.balabit.com/2010/11/log-sample-collecting-project/">http://czanik.blogs.balabit.com/2010/11/log-sample-collecting-project/</a>
you can read a document, which describes it.<span id="more-92"></span>
It has three main parts:
<ol>
<li>background / what is it good for</li>
<li>methods</li>
<li>technical requirements</li>
</ol>
<p>It still has some “FIXME” parts in it, but already enough to get
started. Please let us know what you think about it, if you have any
questions, miss any information, etc.!</p>
</blockquote>
<br>
First of all, it's a great initiative -- this is something a lot of
people could profit from. Here are my remarks:<br>
<br>
1) If I get it right, this is just an RFC for the initiative. When the
project is started, we'd definitely need an easy-to-use interface that
makes it easy to browse and/or submit log samples. Something like what
<a moz-do-not-send="true" class="moz-txt-link-freetext"
href="http://www.pcapr.net">http://www.pcapr.net</a> does for network
captures, though without the ads
and the annoying mandatory registration stuff. We can get started by
using a git repo for the samples just like for patterndb, but in the
long run, it'd put the barrier much lower and thus result in more log
submissions to have a nice'n'shiny website for this. In either case, we
need very clear and short instructions on how to submit logs, because
this blog post is a bit too long to read just for that.<br>
</blockquote>
Oops, I really forgot this one. For the short term it is definitely
e-mail, either to the mailing list or to me directly. In either way
I'll make the samples available in git. Of course we can add a web
interface later on, but designing and implementing it takes
considerable time and we can live without it for the start...<br>
<br>
<blockquote cite="mid:4CE55DDB.1080204@balabit.hu" type="cite">2) I'm
not entirely sure that it's a good idea to add explenatory
comments to to logs in such an "in-band" way -- they're way to easy to
mistake for real log messages.</blockquote>
I stored my log files this way and had no such problem. And dealing
with one file is easier than dealing with possible 50+ little files for
some applications.<br>
<br>
<blockquote cite="mid:4CE55DDB.1080204@balabit.hu" type="cite"> I think
sample log files with single
events along with a .nfo file with the necessary meta information would
be much more usable. Yes, as you've written, it would make it a bit
more problematic to handle them, but it'd worth the trouble IMHO.<br>
</blockquote>
Could you write a short proposal, how would you organize log samples
and documentation? Then we could ask the list which they like better.<br>
<br>
<blockquote cite="mid:4CE55DDB.1080204@balabit.hu" type="cite">3) The
sections "All logs", "Application settings" and "Host names" got
me confused. These instructions can be useful but only apply to the
scenario when the submitter tries to create logs for the specifically
for the project. In a final documentation it should be noted
accordingly, something like "Tips for generating high-quality log
samples."<br>
</blockquote>
Well, the whole document is supposed to be about generating
high-quality log samples :-) The quality of the resulting patterns is
dependent on the quality of log samples...<br>
<br>
<blockquote cite="mid:4CE55DDB.1080204@balabit.hu" type="cite">4)
You've left out one way of generating logs, which can also be
important but, I admit, is a lot different from the mentioned two
collecting modes: investigating the source code of applications. This
can reveal possible log messages that are almost impossible to record
in real-life scenarios or to trigger in a laboratory environment but
can notify about very important events. We should think about this way
of getting log messages, too.<br>
</blockquote>
That's right. I considered my target audience to be sysadmins. Of
course programming skills are a bonus :-)<br>
Bye,<br>
<pre class="moz-signature" cols="72">--
Peter Czanik (CzP) <a class="moz-txt-link-rfc2396E" href="mailto:czanik@balabit.hu"><czanik@balabit.hu></a>
BalaBit IT Security / syslog-ng upstream
<a class="moz-txt-link-freetext" href="http://czanik.blogs.balabit.com/">http://czanik.blogs.balabit.com/</a>
</pre>
</body>
</html>