On Tue, Mar 27, 2001 at 05:56:39PM -0500, jon@dumbo.pobox.com wrote:
quick question: why not just output to a program? and have that program deal with the input in any way you choose...
Because the syslog entry that is being output does not contain the facility, level, or host that the message was received from. The only thing that an external program can do is parse the actual syslog entry in hopes that enough detail is there to parse out error severities and hostname. Not all syslog clients actually report their own hostname in the syslog message; case in point, the Cisco 675's. By allowing the administrator to specify the output format of the syslog string, or by adding things like a prepend driver, we can get more valuable data. I don't believe daemons like syslogd and klogd should be talking to database engines directly or even managing databases, but they should allow us to retrieve all information that may be considered valuable. In that case, I agree with being able to pipe out to an external program. Still, the syslog entry in its current state is not as valuable as it could be. The filters module does help out in that sense. Another reason I don't want to pipe the output directly to a program is the overhead of program startup. If I want to connect to a database, I have to start the program (and any virtual machine it relies upon, such as perl), initialize its class structures, make the connection to the database, process the input from file/stdin, process, close connection, exit. Better is to pipe to a special device file, such as a named pipe/fifo file. destination( file("/dev/syslog_output"); ); A daemon perl program, which by its nature only needs to be initialized once, can process log entries as they are output to the fifo by syslogd. Using placeholders in SQL statements with the DBI perl module is a good example of optimizing code: Without using placeholders, the insert statement shown previously would have to contain the literal values to be inserted and would have to be re-prepared and re-executed for each row. With placeholders, the insert statement only needs to be prepared once. The bind values for each row can be given to the `execute' method each time it's called. By avoiding the need to re-prepare the statement for each row, the application typically runs many times faster. Here's an example: my $sth = $dbh->prepare(q{ INSERT INTO sales (product_code, qty, price) VALUES (?, ?, ?) }) || die $dbh->errstr; while (<>) { chop; my ($product_code, $qty, $price) = split /,/; $sth->execute($product_code, $qty, $price) || die $dbh->errstr; } $dbh->commit || die $dbh->errstr; See the executeand bind_param entries elsewhere in this document for more details. Granted, we wouldn't have to worry about this particular statement if we could output the syslogd entry as a configurable SQL statement and pipe it to a fifo and let the database client handle all the details about reading the statement and executing the query. An added advantage to this very generic idea of a string formatter for syslog-ng is that if you want your messages to look like Windoze NT logs, you can do so with a simple template. -- Chad Walstrom <chewie@wookimus.net> | a.k.a. ^chewie http://www.wookimus.net/ | s.k.a. gunnarr Key fingerprint = B4AB D627 9CBD 687E 7A31 1950 0CC7 0B18 206C 5AFD