On Mon, 15 Oct 2007 23:33:38 +0200 Daniel Carrera daniel.carrera@zmsl.com wrote:
Question: I know that you can have multiple sites on the same Drupal installation. But can each site have a different set of modules? I admin more than one site, but they are all totally different.
Is there any good documentation about sharing modules and codebase for several sites?
I see some pros and cons but I wonder if it is worth the effort: pros: - you can share tables - you fix/tune stuff in one place - you upgrade in one place - you save some space cons: - less tedious but more complex/risky install/upgrade process
any hands-on experience? comments?
On 16/10/2007, Ivan Sergio Borgonovo mail@webthatworks.it wrote:
Is there any good documentation about sharing modules and codebase for several sites?
I see some pros and cons but I wonder if it is worth the effort: pros:
- you can share tables
You don't need to share a codebase (multisite) for that. The settings that are used for mixing up tables, table prefixes and multiple databases don't need a multisite set up to work.
- you fix/tune stuff in one place
Or more accurately: you can fix/tune a small fraction of stuff in one place. Most fixing/tuning seems to be driven out of the database/admin screens rather than the codebase.
- you upgrade in one place
You still need to upgrade each sites database individually.
- you save some space
A very minor benefit with the current cost of disk space. The Drupal codebase is pretty small, and could quickly get dwarfed by the database size or uploaded files/images.
cons:
- less tedious but more complex/risky install/upgrade process
Yep. You also need to upgrade every site at the same time - which could be a pain if each site has different clients or module requirements.
any hands-on experience? comments?
Multisite makes the easy stuff (eg uploading files) easier at the expense of making the harder stuff harder and reducing flexibility. The codebase management is the easily scriptable and version controllable part. It is the database management that is the harder part of managing multiple Drupal sites.
Multisite still has its uses though. I use it to manage different instances (eg dev, test, production) of the same site so that each instances unique settings.php stuff can be version controlled along with the other instances in the same codebase. That way I can check out the codebase onto any of our servers or virtual hosts and it will pick up the appropriate database connection and other settings.
My (personal) opinion is that if you don't fully understand how multisite works and why you need it, then you probably don't need it. Others might feel differently though.
On Wed, 17 Oct 2007 11:44:09 +1300 Anton anton.list@gmail.com wrote:
- you fix/tune stuff in one place
Or more accurately: you can fix/tune a small fraction of stuff in one place. Most fixing/tuning seems to be driven out of the database/admin screens rather than the codebase.
I use pgsql frequently and I generally have to fix modules to make them work with pgsql. Recently I had to fix the watchdog function in core for the same reason. Modules in different sites tend to have different versions. Adding sites to the same code base may mitigate these problems.
- you upgrade in one place
You still need to upgrade each sites database individually.
You still don't have to upgrade all the code bases.
- you save some space
A very minor benefit with the current cost of disk space. The Drupal codebase is pretty small, and could quickly get dwarfed by the database size or uploaded files/images.
Absolutely true unless your site is on your Linux router ;)
cons:
- less tedious but more complex/risky install/upgrade process
Yep. You also need to upgrade every site at the same time - which could be a pain if each site has different clients or module requirements.
Core should stay untouched, can't modules be divided in /all/ and site specific modules? Shouldn't the overhead of having them in the all directory be negligible if they are not activated? I'd keep the ones that all the web sites need in all and then the not shared ones in a different place.
I agree that putting everything together you risk to kill everything at one time when you upgrade/change some stuff. I wouldn't put the codebase of different clients in the same place too.
any hands-on experience? comments?
The codebase management is the easily scriptable and version controllable part. It is the database management that is the harder
I can't see how it can be easily scriptable and version controllable. I see maintaining different code bases a management burden. I tweak something in a module (no matter how I would like to avoid to touch work maintained by someone else, but sometimes you've to do it). Then I've to replicate the same change everywhere or a) I'll forget b) changes in other modules won't be in sync.
It is trivial but it is one more management step.
part of managing multiple Drupal sites.
Multisite still has its uses though. I use it to manage different instances (eg dev, test, production) of the same site so that each instances unique settings.php stuff can be version controlled along with the other instances in the same codebase. That way I can check out the codebase onto any of our servers or virtual hosts and it will pick up the appropriate database connection and other settings.
My (personal) opinion is that if you don't fully understand how multisite works and why you need it, then you probably don't need it. Others might feel differently though.
mmm let me understand... you share *all* the codebase including modules etc... with the exception of settings... just to edit the production settings before checkout and let it land on the server as you need it and not as on your local copy?
Just thinking how multisite could be exploited.
thx
On 17/10/2007, Ivan Sergio Borgonovo mail@webthatworks.it wrote:
Modules in different sites tend to have different versions. Adding sites to the same code base may mitigate these problems.
I use the svn:externals property for that (developing modules branches outside the main Drupal core codebase). More write up here:
We have changed our repository a bit since then to make it a bit simpler, but you get the general idea and should be able to adapt it to your needs.
I can't see how it can be easily scriptable and version controllable. I see maintaining different code bases a management burden. I tweak something in a module (no matter how I would like to avoid to touch work maintained by someone else, but sometimes you've to do it). Then I've to replicate the same change everywhere or a) I'll forget b) changes in other modules won't be in sync.
It is trivial but it is one more management step.
That's why separating out your module development and using svn:externals (see link above) is so useful. You can check your module tweaks in once for a certain branch, and every site that uses that module branch automatically gets the updated module.
mmm let me understand... you share *all* the codebase including modules etc... with the exception of settings... just to edit the production settings before checkout and let it land on the server as you need it and not as on your local copy?
All settings live in the repo - we don't exclude anything from the repo.
No editing of production settings required. The production settings live in the repo under the production sites directory (eg /sites/example.com). Our deployment script does an svn export into a temp directory, then scps the files to the production server.
On Wed, 17 Oct 2007 13:11:16 +1300 Anton anton.list@gmail.com wrote:
On 17/10/2007, Ivan Sergio Borgonovo mail@webthatworks.it wrote:
Modules in different sites tend to have different versions. Adding sites to the same code base may mitigate these problems.
I use the svn:externals property for that (developing modules branches outside the main Drupal core codebase). More write up here:
OK. But nothing to do with multi-site... that's a different trick ;)
So different projects are just a collection of links to your drupal core repo, contrib modules and your own work (modules + themes).
How do you plug in core and contrib? You'd have your own repo cos you can't commit to the original repo. If you've to tweak core or contrib, how do you diff with upstream updates? You plug it in somewhere with a different tag and diff with the previous then you apply the tweaks to the new imported version, you open a new branch for all your projects, make the links to the new components, test. Repeat for all the projects... maybe a bit faster cos you already tested in one place...
Do you have someone in charge just to keep versions in sync or the person that modify a module, is responsible for upgrading all the projects? Who/when do you upgrade contrib modules?
Start to sound interesting if you deployed a lot of automation... and nothing should stop you to forget to link projects to newer version, but at least you'll have a trace of what you did for other projects that were upgraded.
Tricky things like modifying your own modules "on place" can be risky cos you'll modify all your projects. You should branch the module, modify one project to link to the new version, test it, change the link of all the others... The workflow should require a lot of discipline.
There is still too much manual work in programming ;) wasn't it all about automation?
what did you simplify in your svn layout? what is the exact workflow? including the launch of scripts?
mmm let me understand... you share *all* the codebase including modules etc... with the exception of settings... just to edit the production settings before checkout and let it land on the server as you need it and not as on your local copy?
All settings live in the repo - we don't exclude anything from the repo.
Yep, but you'll still have to change production settings *and* dev settings I suppose just for the DB connection.
No editing of production settings required. The production settings live in the repo under the production sites directory (eg /sites/example.com). Our deployment script does an svn export into a temp directory, then scps the files to the production server.
Why 2 steps? tmp -> scp?
thx
On 17/10/2007, Ivan Sergio Borgonovo mail@webthatworks.it wrote:
How do you plug in core and contrib? You'd have your own repo cos you can't commit to the original repo. If you've to tweak core or contrib, how do you diff with upstream updates?
Core is handled by the svn vendor drop technique documented in the svn book. We don't bother with that for contrib though - it would get too fiddly.
And the long time between 5.x patch releases means that we seem to hardly ever do vendor drops any more :)
We handled contrib by doing selected cvs checkouts (via a script) from drupal.org into a svn like hierarchy (eg a subdir for each branch/tag we end up using), and then checking that into svn. So a single cvs update at the root dir updates the working copies of all the modules, then we do a svn commit to apply them.
If any local changes we have made (which isn't many, and only tend to be in unmaintained modules anyway) don't merge cleanly we deal with the conflict then.
Having the different cvs branches/tags in different directories allows you to pick and choose versions by changing your svn:externals path. If you just want something to be stable pick a release tag, if you want to track development pick a dev branch.
You plug it in somewhere with a different tag and diff with the previous then you apply the tweaks to the new imported version, you open a new branch for all your projects, make the links to the new components, test.
Not really, it sounds like we don't have anywhere near the customisations you have. merges take care of most things automatically.
Repeat for all the projects... maybe a bit faster cos you already tested in one place...
We're not really a consultancy, we just have a couple of projects for us that don't get worked on full time. But I still think these techniques would still help a consultancy with lots of projects more than ad hoc development would.
Do you have someone in charge just to keep versions in sync or the person that modify a module, is responsible for upgrading all the projects? Who/when do you upgrade contrib modules?
Yep - me. I'm the only one that knows enough svn/cvs and does any module development anyway (plus I'm the company sysadmin). The others who are/were designers who basically only needed to know svn update, add and commit and stuck to working on our custom themes.
With a bigger team of devs, things could be automated more eg we don't even need to use svn hook scripts yet.
Start to sound interesting if you deployed a lot of automation... and nothing should stop you to forget to link projects to newer version, but at least you'll have a trace of what you did for other projects that were upgraded.
Tricky things like modifying your own modules "on place" can be risky cos you'll modify all your projects.
Yeah that would be a risk for a larger team with lots of projects (ie not us). I suppose education and branching would help. Does svn have any scope for read-only working dirs?
You should branch the module, modify one project to link to the new version, test it, change the link of all the others... The workflow should require a lot of discipline.
Doesn't any programming workflow? ;)
There is still too much manual work in programming ;) wasn't it all about automation?
what did you simplify in your svn layout? what is the exact workflow? including the launch of scripts?
We basically went from separate cvs/svn working dirs that were synced by scripts to a combined working dir for both systems.
Why 2 steps? tmp -> scp?
We do some cleanup in between to get rid of all the files that don't need to go to the production site - eg translations, scripts, text files etc. Not really required of course, we're just being picky.
Anyway...
This stuff isn't the be-all and end-all of setups, just something that works for us. Hopefully some of the ideas can be adapted to suit others :)
On Thu, 18 Oct 2007 10:12:35 +1300 Anton anton.list@gmail.com wrote:
Yeah that would be a risk for a larger team with lots of projects (ie not us). I suppose education and branching would help. Does svn have any scope for read-only working dirs?
You can make read-only dirs with apache/.htpaccess to different groups of people. I don't know if you can do it through svn client (I doubt it).
You should branch the module, modify one project to link to the new version, test it, change the link of all the others... The workflow should require a lot of discipline.
Doesn't any programming workflow? ;)
Not where the wine is good ;)
OK... if I'll ever need to make important changes to core or modules I'll know where to start from to deploy an infrastructure that will help me.
the trick for having dev/prod setting with multi-site is neat and easy to understand. Having time to learn how to use the svn hook system that would be another place where to look at.
thanks
I use it to manage different instances (eg dev, test, production) of the same site so that each instances unique settings.php stuff can be version controlled along with the other instances in the same codebase. That way I can check out the codebase onto any of our servers or virtual hosts and it will pick up the appropriate database connection and other settings.
Ok - this sounds vastly interesting to me. I would love to hear a more detailed explanation of how this might be set up. Can you upgrade a module on dev first and then port it to stage/prod? Can you use subdomains for this (mysite.com and www.mysite.com, dev.mysite.com, stage.mysite.com)? Is there a tutorial on drupal.org or would you be willing to provide more detailed information as to how you accomplish this?
I am currently running my dev environment on a completely separate server and that worked for this smaller project but I am about to embark on some bigger projects and this sounds like it will be the way to go.
.s
-
Anton -
Ivan Sergio Borgonovo -
sander-martijn