[consulting] scaling sites with restricted downloads

[Boris Mann]

On 30-Mar-06, at 8:45 AM, Laura Scott wrote:

> 1) We're about to build two different music community sites that  
> are going to need to start small but (obviously) be able to scale.  
> Controlling access to uploaded mp3s etc. will be key aspects of  
> this install. To (hopefully) reduce their server needs -- because,  
> after all, they're musicians and thus aren't exactly flush -- I'm  
> considering using public fileserver settings, and chmod-ing the  
> folder to limit access to only via Drupal. Would that work? Are  
> there limitations to this approach?

See FileRequest (http://drupal.org/node/43202) -- it's one of the  
things on my radar for trying to get applied as patches to core. But,  
with 4.7 out the door, we may do something like try and backport a  
"new" file API...don't know. More dev talk.

> 2) More generally, has anyone flipped an existing site's file  
> system from private to public, or vice versa? I know this can wreak  
> utter havoc, but it seems that some scrupulous editing of db data  
> should be able to effect retroactive settings/locations fixes. I  
> would love to hear any anecdotes in this area.

You may have issues, treat it like a manual upgrade and/or import,  
and yes, you can fix with scripting one way or another.

> 3) Same question for moving an existing site into having uploads  
> managed by filemanager+attachment?

Don't use those. They are not core, and you will be in a world of  
pain should, say, the new improved file api be in 4.7/4.8. But that's  
development, so don't want to get too deep into it. Talk to James  
Walker if you want to help work on the next file API.

So, in short, you REALLY need to know what you are doing in scoping  
large static file storage. Drupal's complexity lets you shoot  
yourself in the foot, as do default Apache/MySQL configurations. You  
need to be experienced in optimizing the entire stack. Use the dev  
module to look for slow queries (esp. if you are throwing in lots of  
edgy contrib modules and/or handcoded PHP snippets that do DB access).

In summary, it's nice that you want to offer something cheap, but be  
aware that if gets popular, you may not be able to pay for/support it.

Cheers,

--
Boris Mann
Vancouver 778-896-2747 San Francisco 415-367-3595
SKYPE borismann
http://www.bryght.com