[syslog-ng] Anyone got a well performing search interface for syslog data?

[Jason Haar]

We're generating around 4Gb syslog data per week, and I'm looking for a
good search interface into it.

I can cut my way through it with egrep/etc, but waiting 10-15min for a
result really isn't going to break any speed records. Especially when I
then need to re-run it with another "grep" on the end of it! ;-)

I have tried injecting it into a MySQL database using some schemas I've
found on the Internet - but the performance didn't seem much better to
me - and you lost the "free-text" attributes of grep (or more
specifically, the sorts of searches I find I want to do aren't
SQL-friendly).

Has anyone come up with a good speedy way of coping with Gbytes of
syslog data? Or is it time to invest in some Appliance or the like?

-- 
Cheers

Jason Haar
Information Security Manager, Trimble Navigation Ltd.
Phone: +64 3 9635 377 Fax: +64 3 9635 417
PGP Fingerprint: 7A2E 0407 C9A6 CAF6 2B9F 8422 C063 5EBB FE1D 66D1