[consulting] Drupal server requirements

[Michael Haggerty]

> -----Original Message-----
> From: John Handelaar [mailto:john at userfrenzy.com] 
> Sent: Wednesday, March 29, 2006 1:19 PM
> To: mhaggerty at trellon.com; A list for Drupal consultants and 
> Drupal service/hosting providers
> Subject: Re: [consulting] Drupal server requirements
> 
> > The big thing to remember is most people using Drupal are 
> NEVER going 
> > to run into race conditions in their database, and a clustering 
> > solution is inappropriate for about 99% of the Drupal sites 
> out there.
> 
> Yes, but that wasn't our discussion.
> 
> We know that all the well-known massively-scaling MySQL-PHP 
> apps we're aware of go for the shared-nothing trifecta.

If I understand you correctly, I would disagree that all well-known
massively scaling MySQL-PHP apps go for a 'shared-nothing' trifecta. A lot
of massively scaling MySQL-PHP applications go for the
cache-queries-in-memory omnibus approach, utilizing cool mods like memcached
to store results until those result sets are affected. You can get a lot
more traction on some platforms by making calls to a caching solution then
you can by breaking off read-write functionality.

> I don't think it's responsible of me to not point out that 
> one of its key pillars is not (yet) ready in Drupal.

Heh, I'm not the one to tell other people what to suggest. But again, it's
not really the application's role to determine where to write data, the
database itself is usually much better at making this decision. This kind of
architecture sounds unwieldy after adding a few servers.

> Meanwhile, as I tell anyone who asks me directly, the first 
> step is usually to throw memory at any performance issues, 
> the second is to physically seperate the DB from the web 
> server, and only when that stops working do we get into clustering...

Well, yeah, that's probably what most people advise. But when you have a few
dozen sites, it's a little nicer to be able to just toss everything into the
cluster and not worry about scaling. Considering how cheap it is to set up
and maintain a cluster, why take everyone through the same path? Couldn't
the cost to maintain the cluster be amoritized over a few of the bigger
sites and save each new client some money? As the pool of clients grows, so
could the cluster, reducing everyone's utilization of resources.

> ...by which time, I have no doubt, the read/write split in 
> Drupal will be solid and existent.

Unsure what this is really going to accomplish. I mean, the point here is to
scale what happens at the database level.

How is Drupal going to know where to write the data when you have 2 write
servers? Obviously, you don't want to do all commits to a single database or
you could end up with a massive bottleneck that is worse than not segmenting
operations in the first place. Outside of writing a separate module to
determine where the activity is being directed, wouldn't something like this
effectively limit the platform to a single write database?

For that matter, isn't traffic control exactly what a cluster is supposed to
do? Why is this effort being duplicated?

M

> -------------------------------------------
> John Handelaar
> 
> E  john at handelaar.org    T +353 21 427 9033
> M  +353 85 748 3790    http://handelaar.org
> -------------------------------------------
> Work in progress: http://dev.vocalvoter.com
> -------------------------------------------