[syslog-ng] Buffering AF_UNIX Destination, Batch Post Processing Messages

[Martin Holste]

> I guess syslog-ng could also write tab separated data into files and can
> also do per-minute batches (by using the $MIN macro). Are there any
> other things the perl stuff does?

Yes, you certainly could get Syslog-NG to write TSV in minute batches
without any other program's intervention.  I pipe to Perl because I do
some minor data alterations before writing to file.  Specifically, I
CRC the program name to generate a program ID so that I can store the
program in a programs table and the program_id in the main logs table
to conserve space and keep the database in as close to 3rd normal form
as I can.  I also do basic conversions like INET_ATON all IP addresses
to store them in integer columns.  In the future, I would put advanced
correlation capabilities there (probably nothing like what Matthew is
cooking up, though!) as well as any real-time cluster messaging I need
to do.