[development] Solving the dev->staging->live problem

[Greg Dunlap]

When I created Deploy I had a set of goals in mind that I felt a solution
had to meet. I think (hope) that most people will agree that these goals
represent the ideal of what any staging and deployment system should
achieve. Some of this will repeat some of what has already been said
insightfully and thoughtfully above, but I kind of wanted it all put
together in one place.

I wanted to see a solution that focused on what can be done within Drupal,
without modifying core. That includes not modifying the default installs of
core tables (to reserve IDs for instance.) I see a lot of people focusing on
the database when talking about deployment which I think is the wrong way to
look at things. When you're moving a node, you should deal with it the same
way the rest of Drupal does. load it, save it. The only thing missing there
is how do you move it from one place to another. We should use that which
Drupal gives us.

Everything should be deployable - content, comments, taxonomy, users,
configuration, content types, the whole nine yards. When "things" are
deployed their references to other "things" should be preserved. It must
work bi-directionally. If I make a change to my Pathauto config, I should be
able to push it forward. Conversely if a user submits a new comment, that
comment should be able to be pushed back to my staging site (and it should
still be linked to the proper node.) Kathleen makes a good point that we
need to account for things that are different between servers by design.

Modules should be able to hook into the deployment process easily and with a
great deal of flexibility (this is the part I think my module currently gets
pretty right.) This framework could live in core or not, I really think it
doesn't matter in the end. As recently saw with D6 & Views, any module with
a sufficiently large enough install base is "core" in that people won't
build sites without them and won't build modules without integrating with
them. (Please note that I am not implying my module is at anywhere near the
level of Views.) Build a deployment system that solves everyone's problems
and it will get implemented throughout the spectrum.

Any system should also be flexible enough to meet the needs of all users -
that means there should be a point and click UI for non-developers to be
able to deploy things, but also the ability to move the information into
code for traditional deployment as Adrian wants. Things should be able to be
deployed out of cron or published through a timetable. Different sites have
different needs and with a flexible, sensible API this shouldn't be a
problem.

There is really only one thing standing in the way of all of this happening
and that is the fact that we cannot uniquely identify most things between
servers. I create a node on my dev server and it is given the ID of 10. It
goes through my editorial workflow until its done and I push it out. However
on my live site node 10 is a forum post that one of my users made. So my new
node becomes node 11. Now I make a change to this node on my dev site and
there's no way to push the update. The same goes for bringing forum node 10
back to my dev server. This is why all the id-specific hoop-jumping happens.
The only reason Deploy works at all is because I'm pushing around things
like CCK content types that can be uniquely identified through a text string
(the user-specified machine name.)

This whole problem would be much simpler if every "thing" had a GUID. Get
that in place, and building a system to do deployment in the way described
above actually becomes a realistic proposition. As Sam implied I've been
conceptualizing a solution based on this premise, but its been hard finding
time to really work on it. Hopefully soon.

Having made that very long, rambling post (sorry its been a long day and its
late) I think there's a lot of room for many solutions dependent on your
needs, and its unlikely that any "one size fits all" answer will be
forthcoming. Sites have different needs and different resources and
different workflows and thus its natural different solutions will develop.
People like Kathleen and Dave and the Bryght/Raincity folks and many others
have done great work in this sphere and I'm sure will continue to. The more
smart people are thinking about it and working on it, the more ideas and
knowledge will be built which can only lead to good things down the road.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.drupal.org/pipermail/development/attachments/20080810/889d9bac/attachment-0001.htm