Hi Christopher,

First of all, I advise you to consider cross-posting the mail in the development list. While you need a straight answer, which is kind of a support request, your insights might make others re-think how Drupal's database is built, and how we can make it (even) easier to develop Drupal site over many years.

Secondly, I was wondering with yhager about having a BoF meeting in Szeged, since the situation you are experiencing is one of the most common issues with Drupal development. Since Configuration and Content are kept on the same DB, it is practically impossible to export only some of the data (for example only content, or only configuration).

We were wondering how come everyone is suffering, and no one speaks :) We finally let go of the idea.

However, if there is enough interest, maybe we can still organize a BoF in Szeged. And that's the second reason why I think you should cross-post in the dev list - maybe we'll see that there is enough man power to advance in this matter soon.

oh... and... sorry for the situation you're in, Unfortunately, I don't have a winning advice ;-)

Zohar Stolar Linnovate - Community Frameworks zohar@linnovate.net http://www.linnovate.net Tel. +972-77-5345-704 Cel. +972-52-8348-278 Fax. +972-72-2500-882 SkypeID: zstolar LinkedIn: http://www.linkedin.com/in/drupal

Christopher M. Jones wrote:

I'm looking for a decent approach to staging a drupal site. The production site will be a collaborative authoring project, with forums, blogs, and lots of media. The client will have access to this site, and will be maintaining some of the content. Other content will be maintained by us.

The development company that hosts this project prefers to make all of their changes, both to content and templates, in a testing environment. Once their client has approved the changes, they like to 'push' them to production.

However, while these changes are taking place, the client may be administering forums and writing blog posts in the -production- version.

To further complicate things, my company wants a three-stage cycle. They want a dev site, where they make changes for in-house review, which they then push to testing for client review before everything is pushed to production.

I'm unsure how to approach this. The site in question has always been static html created in Dreamweaver. At some point they started adding other things, so now there are two wordpress blogs, and two phpbb forums. The forums and blogs presently are excepted from the development cycle. They simply appear to be part of the site, because their templates have been designed so. But that means that we've got to propagate template changes across five templates. Things are breaking constantly, and this is why I piped up to them about drupal.

I've seen a lot of discussion about this topic, but I really need some hard answers. What should I do?

I've seen the Staging module for 6.x. Is it safe to use? If so, then we could use that for the database. Templates could be pushed with rsync or svn... whatever. But would this work two ways? Could we sync the dev / testing sites to the production site, then sync the other way? Would we need to?

Also, I envision using a multisite environment so that all sites share the same modules, media, etc, but use different templates.

I desperately need the detailed advice of someone with experience, here.

Hi Christopher,

There is no packaged answer to what you are looking for.

My forthcoming book, Leveraging Drupal (http://www.amazon.com/Leveraging-Drupal-Getting-Your-Right/dp/0470410876, not due out till December at the earliest), deals with this very issue, with a real life development and deployment example. But it is still a work in progress, and I am streamlining an approach that will allow for a Drupal 5.x to Drupal 6.x transition.

Basically some of the conclusions I'm coming to confirm methods I have been using with my clients, basically, making heavy use of an SVN repository for deployment of the ./sites directory, while checking out and updating Drupal core using CVS (simple command line one-liners).

Deploying changes to templates and custom modules is as easy as "svn commit" and "svn update" and even that can be automated. I also rely heavily on the new features of the drush and update status modules using the SVN plugin.

The difficulty resides in the configuration and content using the database at the same time in the different sites, so that programmers working with, say, Panels 2, on the dev and test sites would need to merge their database changes with the production database if the client is close to launching and is putting in content at the same time.

On this many have promised but few shared; perhaps because it is not so simple and has heavy dependencies on concrete situations.

Others seem to be actually finding some solutions and are willing to share them, notably Kathleen Murtagh in her blog article "Development to Production: A Database Merge Script" and similar. See http://ceardach.com/blog/2008/06/development-production-database-merge-scrip... . She has authored the database scripts module, which looks like it could be one of the promising building blocks for what you need, alongside CVS and SVN.

Victor Kane http://awebfactory.com.ar

On Thu, Aug 7, 2008 at 4:22 PM, Christopher M. Jones cjones@partialflow.com wrote:

I'm looking for a decent approach to staging a drupal site. The production site will be a collaborative authoring project, with forums, blogs, and lots of media. The client will have access to this site, and will be maintaining some of the content. Other content will be maintained by us.

The development company that hosts this project prefers to make all of their changes, both to content and templates, in a testing environment. Once their client has approved the changes, they like to 'push' them to production.

However, while these changes are taking place, the client may be administering forums and writing blog posts in the -production- version.

To further complicate things, my company wants a three-stage cycle. They want a dev site, where they make changes for in-house review, which they then push to testing for client review before everything is pushed to production.

I'm unsure how to approach this. The site in question has always been static html created in Dreamweaver. At some point they started adding other things, so now there are two wordpress blogs, and two phpbb forums. The forums and blogs presently are excepted from the development cycle. They simply appear to be part of the site, because their templates have been designed so. But that means that we've got to propagate template changes across five templates. Things are breaking constantly, and this is why I piped up to them about drupal.

I've seen a lot of discussion about this topic, but I really need some hard answers. What should I do?

I've seen the Staging module for 6.x. Is it safe to use? If so, then we could use that for the database. Templates could be pushed with rsync or svn... whatever. But would this work two ways? Could we sync the dev / testing sites to the production site, then sync the other way? Would we need to?

Also, I envision using a multisite environment so that all sites share the same modules, media, etc, but use different templates.

I desperately need the detailed advice of someone with experience, here.

[ Drupal support list | http://lists.drupal.org/ ]

Your development company is correct. There is no professional way to push changes to a live site directly. It is absolutely dangerous, a recipe for bringing down your site and/or for destroying live material. Or, as you complain, things will simply be breaking constantly.

There are many ways to separate the content (e.g., new posts to a blog) from the database code behind that content.

Any professional development company will do development as you describe it (sometimes there will be reasons for additional stages). "Dev" is where structural changes are done; "Test" or "QA" is where new editorial content is checked (out of the way of development), and where dev is finally tested before being pushed live. By using virtual servers, one can ensure that each environment is identical to the live environment (except for dev, which will often differ radically)

This is tricky, because many database-backed websites don't have good ways to promote just code changes (and drupal doesn't make it so easy), but it is critical that, at a minimum, those three functions are kept separate. In the case of static sites, we often used scripts based on rsync to move changes from test to production, never allowing humans to log into production. This is trickier when you have to update/change some, but not all database tables (which also require more than rsync to stay in sync).

(It is often the case and test and qa are also separate--one used to test code changes before production; the other for content changes. And, of course, if there are conflicting dev projects, there may be more dev instances, or sandboxes, to deal with that variety--but for most of us, that would be rare.

ari

On Thu, Aug 7, 2008 at 3:22 PM, Christopher M. Jones <cjones@partialflow.com

wrote:

I'm looking for a decent approach to staging a drupal site. The production site will be a collaborative authoring project, with forums, blogs, and lots of media. The client will have access to this site, and will be maintaining some of the content. Other content will be maintained by us.

The development company that hosts this project prefers to make all of their changes, both to content and templates, in a testing environment. Once their client has approved the changes, they like to 'push' them to production.

However, while these changes are taking place, the client may be administering forums and writing blog posts in the -production- version.

To further complicate things, my company wants a three-stage cycle. They want a dev site, where they make changes for in-house review, which they then push to testing for client review before everything is pushed to production.

I'm unsure how to approach this. The site in question has always been static html created in Dreamweaver. At some point they started adding other things, so now there are two wordpress blogs, and two phpbb forums. The forums and blogs presently are excepted from the development cycle. They simply appear to be part of the site, because their templates have been designed so. But that means that we've got to propagate template changes across five templates. Things are breaking constantly, and this is why I piped up to them about drupal.

I've seen a lot of discussion about this topic, but I really need some hard answers. What should I do?

I've seen the Staging module for 6.x. Is it safe to use? If so, then we could use that for the database. Templates could be pushed with rsync or svn... whatever. But would this work two ways? Could we sync the dev / testing sites to the production site, then sync the other way? Would we need to?

Also, I envision using a multisite environment so that all sites share the same modules, media, etc, but use different templates.

I desperately need the detailed advice of someone with experience, here.

[ Drupal support list | http://lists.drupal.org/ ]