Sent: Fri Aug 19 2011 10:57:57 GMT-0600 (MST)
From: Jakub Jankowski <shasta@toxcorp.com>
To: Syslog-ng users' and developers' mailing list <syslog-ng@lists.balabit.hu>
Subject: Re: [syslog-ng] Store syslog occurrence frequency instead of adding all of them to the DB

On 2011-08-19, syslogng@feystorm.net wrote:

Secondly using a 32-bit checksum of the message text to determine uniqueness 
is risky. It would be farily easy to end up with 2 different messages that 
have the same checksum. A md5 checksum would be much better, but I dont 
believe syslog-ng has a function to compute md5 sums.

One can delegate this task to the database itself. MySQL has MD5() as well 
as SHA1() built in.

Its been a while since I've had syslog-ng talk to a database directly, but when I did, you couldnt use database functions when storing data. You just gave syslog-ng the names of the fields, and then the macros to stick in those fields and syslog-ng went and assembled the query for you. Does syslog-ng let you construct the query yourself now? If so, then yes, using the database's hashing functions would work fine.