[development] Database / SQL future thoughts

[Chris Johnson]

Perhaps before this discussion gets too far astray -- or maybe such
digression is impossible to avoid!  :-)

My point in posting this was to elicit folks who were interested in
the database usage and performance for the core parts of Drupal, and
in particular the folks who I through the past years have observed as
being especially astute and involved (and if I left your name off that
short list, it's only because I forgot your name or other such failure
on my part -- no slight or insult intended).

I was particularly hopeful that folks would read the comments, and
take to heart those written in favor of SQL, such as those by Robert
Young.  I didn't want to bias the conversation by injecting my opinion
too early, but I'm in the camp that says things like BigTable are
really just reinventions of older technology, AND that the death of
RDBMS/SQL is not nearly as soon as some might think.

For Drupal, I think it may make sense to attempt to make the SQL as
simple as possible, avoiding the really complex capabilities and doing
more of that work in PHP, where most of the developers actually have
some expertise.  This is especially true since MySQL's MyISAM table
engine is really poor at doing anything beyond that.  If we were to
move to using MVCC[1] engines like Postgres, Oracle and MySQL with
InnoDB or Falcon, then more complex SQL would be useful.
Unfortunately, most Drupal developers don't have the SQL skills
needed, and even worse, most people are not going to have the ability
to physically optimize the database for performance (i.e. skilled DBA
person, access to MySQL settings at most hosting services, etc.).

Hence, the former course of using only the simplest, most primitive
SQL statements appeals to me for Drupal.

It's that last point I'd really like to have criticized.  Is it
possible to do that?  Especially in light of DBTNG, where is the line
between simple statements and complex statements?

Lastly, although many Drupal-based sites are database limited, many
are PHP-code interpretation (# of files/lines parsed) or bandwidth
(fast JS/image pages) limited, too.  My goal is simple:  make my
hundred module websites respond in less than 2 seconds!  :-D

Thanks for participating in this discussion, no matter which tangent you're on.

..chris

[1] http://en.wikipedia.org/wiki/Multiversion_concurrency_control