[development] Re: enterprise needs

[Darrel O'Pry]

On Mon, 2006-02-27 at 10:45 -0800, Benson Wong wrote:
> > I've never used mod_gzip for performance, mainly as a cost control tool.
> 
> Performance is essentially fast you can deliver the content to the
> user, including making the transport of data more efficient.
> 
> >
> > > 3. Get a faster DB server....
> 
> > A local database server is far faster in terms of throughput as long as
> > it is not tight on memory, and has a sizable query cache.
> >
> 
> What you save in network transport, you lose in sharing resources with
> the web server. In a busy environment, more hardware = more
> performance. In a small environment, debating this will be splitting
> hairs.

Everyone in a large environment had to transition from a small
environment. Not everyone has the resources to go from, oh I have
everything on one machine, to I have a database cluster feeding my web
cluster, with load balancer in front, proxied by a squid cluster. 

And not all web sites and communities, explode, some just grow gradually
and consistently.

The normal requirements I try to meet during a cluster build out is to
maintain active failover capabilities, and at least minimal load sharing
capabilities.

That said my transitions normally go something like...

1) stand alone server working all by itself...
- gotta start somewhere

2) 2 servers... 1 db master/http, 1 db slave/http
- provides a failover HA cluster, that can provide performance
gains through RR dns. You will need nfs or some other solution to keep
file systems in sync at this juncture. (I prefer nfs and a rsync
backup).


3) 3 servers... 1) db master, 2 http/db slaves
- Here you split all your write access db, 

4) 4 servers...  at this juncture I would suggest playing with different
configurations and technologies... completely seperate db cluster, load
balancing, proxies, seperate static server, etc. I haven't really had to
progress past 3, without getting the budget I needed to get 10 servers
in the mix and play...


There are also different ideas some people just expand on the 2 servers
idea adding more servers as db slave/http servers... Its easy to
replicate. A monkey can setup system imager, and you can have an easily
scalable cluster until you get to the point that you have to seperate
services to get any more effectiveness out of it, and the administrative
issues are easy... Oh something is broken, re image it. oh the master is
dead... ok well re-ip slave one(hearbeat and mon can automate this)...

There are  administrative, economical, and technical factors which must
all be satisfied which will probably be unique for each individual build
out. 

My experience with HA build outs and web clustering is a bit more
general than drupal. I'd like to hear from more drupalites who have
built out large clusters about their progression and growth paths... I
think it is something that can benefit a lot of people.