On Tuesday 05 May 2009 8:39:08 pm David Metzler wrote:
Hey Chris,
I'm probably not the person you aimed this discussion at but I do have something that might be relevant. (I'm just a lowly contrib developer, but I do have a few years of drupal dev under my belt).
If I understand correctly where Drupal is going with storage abstraction, I can't see Bigtable as a viable replacement for SQL, so I think I'm in the same camp as you, but I do see folks starting to get interested in using the right storage mechanism for the right task. The PDO SQLLite discussion we had a while ago was I think somewhat analogous. (although admittedly not the same)
I could see using some kind of BigTable style storage for storing the cached data for drupal and therefore scaling cached pages across servers. Then the cache wouldn't rely strongly on the important relational data in a backend mysql or postgres engine. This seems like a productive use of those kind of storage techniques. I could even see caching common search results using the cache api in the search module (if it doesn't already).
There are many in this community who will, rightly so, express concern about the added complexity of relying on multiple storage engines, but this is one case where I think the results might be worth it.
Dave
In many many cases, the storage engine depends on your scale. For the run of the mill Drupal site, for instance, MySQL-based caching is perfectly fine and is also the lowest barrier to entry (since the main system runs on SQL). For higher end sites, memcache, CouchDB, Flatfile, or whatever else could make more sense. Fortunately those high end sites are also the ones run by people who, if we provide them with proper mechanisms to swap out backends cleanly, would know how to do so and why. So we just need to make more key parts of Drupal pluggable and those who actually care about such differences can leverage them as appropriate, and those who don't need to care won't have to. -- Larry Garfield larry@garfieldtech.com