[development] Database / SQL future thoughts

[Ivan Sergio Borgonovo]

On Mon, 4 May 2009 14:11:07 -0500
Chris Johnson <cxjohnson at gmail.com> wrote:

> Folks who care about the direction and performance of the Drupal
> database (and developers just generally interested in database
> backends for applications) should read this blog posting by Bob
> Cringley -- and more importantly, the comments written by some
> folks who know way more about databases than Mr. Cringely:
> 
> http://www.cringely.com/2009/05/the-sequel-dilemma/

> David Strauss and Larry Garfield, as our two current DB experts
> (IMHO, of course) -- I'd especially like you to digest the more
> pertinent comments and think about how we can simplify our
> database access, and hence optimize our performance.  I'm about to
> go off an research Hadoop a bit, but my gut feeling is the
> comments which say SSD and proper, simple use of SQL and good data
> paths will be the right answer for the next decade or so are
> probably on the right mark.

I'd say it depends on the application (and no... drupal is not *one*
application).

If you're google you really don't care to lose some data and you
don't care about transactions (most of the times). Most of what
drupal does don't need transactions even if once you've several
thousands of custom node types that are valuable and not just crap
on the net you'd be a bit disappointed if the node-kernel remains
orphan of the extra data just because something went wrong halfway...

Furthermore... don't expect that anything that is not SQL and works
better will be easy to understand... unless once more your data is
crap and your application is simple (concurrency, coherency...).

At the language level I won't bet we are going to see any
revolution, so it will still be a good bet to have a good SQL
abstraction layer, hopefully that will support some more
coherency/concurrency features.

Visual Basic, PHP, MySQL etc... were/are popular because they were
easy and cheap and mostly they dealt with crap (nothing you will
kill a self-proclaimed programmer for[1]). What alternative to SQL
are we going to see that will overcome the cost of learning
something new and offer anything better?
For a long time MySQL was just a SQL interface to the filesystem...
that should be a starting point to understand the real value and
importance of SQL and not confusing it with the RDBMS "debate".

Transaction, concurrency, coherency and building good system design
aren't easy.

A SQL interface is going to stay there for longer (linq anyone?).

So part of the future is Hbase, part is not... and there are money
in both camps. I'd say anyway that generally there are more money in
the business that care about transactions and coherency (unless it
is politics).

BTW Amazon does have to care much more about transactions than
Google. No surprise they aren't using MySQL.

[1]
F: hey Silvio, we lost all our invoices record
S: never mind we just deleted the false accounting crime

-- 
Ivan Sergio Borgonovo
http://www.webthatworks.it