[syslog-ng]Feature Request

[Chad C. Walstrom]

--ep0oHQY+/Gbo/zt0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Tue, Mar 27, 2001 at 05:56:39PM -0500, jon@dumbo.pobox.com wrote:
> quick question:
> why not just output to a program? and have that program deal with the inp=
ut
> in any way you choose...

Because the syslog entry that is being output does not contain the
facility, level, or host that the message was received from.  The only
thing that an external program can do is parse the actual syslog entry
in hopes that enough detail is there to parse out error severities and
hostname.  Not all syslog clients actually report their own hostname
in the syslog message; case in point, the Cisco 675's.

By allowing the administrator to specify the output format of the
syslog string, or by adding things like a prepend driver, we can get
more valuable data.

I don't believe daemons like syslogd and klogd should be talking to
database engines directly or even managing databases, but they should
allow us to retrieve all information that may be considered valuable.
In that case, I agree with being able to pipe out to an external
program.  Still, the syslog entry in its current state is not as
valuable as it could be.  The filters module does help out in that
sense.

Another reason I don't want to pipe the output directly to a program
is the overhead of program startup.  If I want to connect to a
database, I have to start the program (and any virtual machine it
relies upon, such as perl), initialize its class structures, make the
connection to the database, process the input from file/stdin,
process, close connection, exit.

Better is to pipe to a special device file, such as a named pipe/fifo
file.

    destination( file("/dev/syslog_output"); );

A daemon perl program, which by its nature only needs to be
initialized once, can process log entries as they are output to the
fifo by syslogd.   Using placeholders in SQL statements with the DBI
perl module is a good example of optimizing code:

       Without using placeholders, the insert statement shown
       previously would have to contain the literal values to be
       inserted and would have to be re-prepared and re-executed
       for each row. With placeholders, the insert statement only
       needs to be prepared once. The bind values for each row
       can be given to the `execute' method each time it's
       called. By avoiding the need to re-prepare the statement
       for each row, the application typically runs many times
       faster. Here's an example:

         my $sth =3D $dbh->prepare(q{
           INSERT INTO sales (product_code, qty, price) VALUES (?, ?, ?)
         }) || die $dbh->errstr;
         while (<>) {
             chop;
             my ($product_code, $qty, $price) =3D split /,/;
             $sth->execute($product_code, $qty, $price) || die $dbh->errstr;
         }
         $dbh->commit || die $dbh->errstr;

       See the executeand bind_param entries elsewhere in this
       document for more details.

Granted, we wouldn't have to worry about this particular statement if
we could output the syslogd entry as a configurable SQL statement and
pipe it to a fifo and let the database client handle all the details
about reading the statement and executing the query.

An added advantage to this very generic idea of a string formatter for
syslog-ng is that if you want your messages to look like Windoze NT
logs, you can do so with a simple template.

--=20
Chad Walstrom <chewie@wookimus.net>                 | a.k.a. ^chewie
http://www.wookimus.net/                            | s.k.a. gunnarr
Key fingerprint =3D B4AB D627 9CBD 687E 7A31  1950 0CC7 0B18 206C 5AFD


--ep0oHQY+/Gbo/zt0
Content-Type: application/pgp-signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.4 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE6wSOJDMcLGCBsWv0RAlX0AJ921niXsamk6O44DZ16Lj5xFgBX2gCg09fh
G1yOJupYtz1n1vvLhOu2J1M=
=WTPG
-----END PGP SIGNATURE-----

--ep0oHQY+/Gbo/zt0--