[Database] Toward better databases

[Larry Garfield]

Hi all.  Curiously Dries hasn't joined the list yet, but everyone else has so 
I'll make a welcome post. :-)

I'd like to introduce:

Joshua Drake - PostgreSQL guru extraordinaire that Dries put me in touch with 
when we hit a wall on the Postgres driver.

Barry Jaspan - Barry stepped forward to write the postgres driver for the new 
database API, which is a good thing, too, because it forced me to rewrite 
most of it. :-)

David Strauss - Drupal's resident MySQL-fu master.

Narayan Newton - The guy who actually runs our server cluster, and also knows 
MySQL better than most.

Larry Garfield - The poor sod who thought rewriting the database layer would 
only take a month or two. :-)

If there's any other MySQL or Postgres or SQLite or PDO gurus you know, feel 
free to invite them.  Long term there is plenty that we can and should do to 
improve our database handling.  Short-term, though, I just want to get the 
damned patch[1] committed. :-)

Right now, as far as I can determine the core system and MySQL are done and 
working and solid.  The problem right now is the PostgreSQL driver.  I 
finally have a working postgres install, but every time I try to install 
Drupal with the Postgres driver PHP itself dies on me.  I've been trying to 
get a debugger working at home so that I can investigate, but so far nothing 
has actually worked. :-)  Barry, is that something you'll be able to look 
into in the near future?  You've spent far more time with the postgres driver 
than I have.  If not, I'll keep trying to get a debugger working or Josh, if 
you're willing I can bring you up to speed on the architecture and you can 
have a look-see.

The other problem we realized recently is that we are relying on Schema API to 
tell us if we need to BLOB-escape a bytea field in PostgreSQL (which does 
seem to be the case), and then similar information for other, sillier 
databases like Oracle.  Schema API only tells us about the main Drupal 
database.  That means only MySQL and SQLite can be used for foreign 
databases.  This is a problem.  

The solution we came up with is to integrate a table-schema-derivation method 
into the new API so that we can pick such things up on the fly as needed.  At 
that point I just sort of ran out of steam and Barry got busy, so that's not 
happened yet.  Since you can't mix and match database drivers in D6 anyway, I 
am tempted to ask Dries if we can add that in a follow-up patch because right 
now this patch is blocking a lot of other work (including cleaning up Schema 
API, which may not even happen in D7 but needs to now that it's been 
OOPified).

So, that's where we stand right now.  Postgres support is blocking pretty much 
everything.  So, Barry, Josh, can either of you lend a hand while I keep 
trying to get a debugger to play nice?

[1] http://drupal.org/node/225450

-- 
Larry Garfield
larry at garfieldtech.com