[syslog-ng] Store syslog occurrence frequency instead of adding all of them to the DB

[Martin Holste]

I'd add on to this by using a crc function to hash the message and
store that in a column to make the unique check very fast:
CREATE TABLE mylogs (
  id BIGINT UNSIGNED NOT NULL PRIMARY KEY AUTO_INCREMENT,
  timestamp TIMESTAMP NOT NULL,
  crc INT UNSIGNED NOT NULL,
  count INT UNSIGNED NOT NULL DEFAULT 0,
  msg TEXT,
  UNIQUE KEY (crc)
);
INSERT INTO mylogs (timestamp, crc, msg) values($timestamp, CRC32($msg),
$msg) ON DUPLICATE KEY UPDATE count=count+1, timestamp=$timestamp;

On Thu, Aug 18, 2011 at 1:33 PM, system at ra-schaal.de
<system at ra-schaal.de> wrote:
> Am 18.08.2011 19:52, schrieb Marcos Tang:
>
>> Can those records being "processed" some how and when I search the MySQL
>> DB, I only see ONE record list the following only?
>>
>> Total occurrenceMessage content
>> =====================
>> 14,400File system is full
>>
>
> can´t you use something like
>
> INSERT INTO tables VALUES (date,logstring) ON duplicate KEY UPDATE date;
>
> ?
>
> florian
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>