[syslog-ng] MongoDB destination driver

Martin Holste mcholste at gmail.com
Fri Dec 31 21:40:00 CET 2010


> I plan to do bulk inserts within the driver - much like how the sql
> driver does bulk commits with explicit-commit turned on.
>

If that's the equivalent of insert () values (), (), () batching, then
that's not what I mean.  In MySQL, and similarly though not to the
same degree in other DBMS's, bulk insert is very different from
batched inserts.  Specifically, a LOAD DATA in MySQL will yield
upwards of 100k inserts/sec because it gets a full table write lock
and buffers the inserts at the columnar data level, not the statement
level.  It's closer to batch writing sectors of disk, not rows of
data.  MS-SQL bcp and mongoimport behave similarly, though the
difference isn't quite as pronounced as it is with mysqlimport on a
MyISAM table.


More information about the syslog-ng mailing list