[syslog-ng] abandon libdbi and write a new sql driver?

[Patrick H.]

Sent: Mon Jan 24 2011 04:12:37 GMT-0700 (Mountain Standard Time)
From: Balazs Scheidler <bazsi at balabit.hu>
To: Syslog-ng users' and developers' mailing list 
<syslog-ng at lists.balabit.hu>
Subject: Re: [syslog-ng] abandon libdbi and write a new sql driver?
> On Thu, 2011-01-20 at 13:17 -0700, Patrick H. wrote:
>   
>> Reading through the mailing list lately I've seen a fair amount of
>> problems reported with libdbi. Given that it also doesnt support
>> prepared statements which is a very important feature, I'm wondering
>> if libdbi should just be dropped entirely. Libdbi seems to be of use
>> to applications that want just very basic sql support, and not for
>> anything involving high performance or reliability.
>>
>> I know its no small undertaking to write a new DBI for all the common
>> databases out there, but if syslog-ng is supposed to be
>> high-performing, libdbi does not fit in with that. I dont know if it'd
>> be best to start a new project to replace libdbi but accomplish the
>> same thing of providing a common API for all databases, or to just
>> write separate modules for syslog-ng 3.2+ for each database out there.
>> Writing modules would certainly be easier as they could be written
>> independently by people who best know each database API.
>>
>>
>> This is all just my opinion, but I for one am all for it. In my
>> production environment, we do logging to a database, and libdbi just
>> didnt have the flexibility or performance we needed, so I ended up
>> writing a separate program which connects to the database to perform
>> the inserts. Would be nice to avoid having to do that.
>> I know I've also heard many recommendations of piping out to a perl
>> script to do the inserts. A perl script should never do its job faster
>> than C code.
>>     
>
> Honestly I was thinking about forking libdbi/libdbi-drivers as we do
> have a couple of changes and upstream is not really doing regular
> releases. And that's the key problem. libdbi is basically stalled as a
> project and without them releasing bugfixes is causing us and our users
> pain.
>
> But I don't see anything against libdbi per-se, within syslog-ng a
> similar API would be needed to cover all database APIs, so why reinvent
> the wheel?
>   
Well thats certainly an option. I just remember looking at the code once 
and it wasnt very clean. But that was something like a year ago and I 
may be remembering some other project I looked at. Primarily I just 
didnt think libdbi in its current state is in line with what syslog-ng 
is trying to accomplish. How that gets resolved isnt that important.

I know I would love to have a mature common database API I could use in 
some of my other projects :-P

-Patrick
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.balabit.hu/pipermail/syslog-ng/attachments/20110124/b1a72e2e/attachment.htm