best pratices, test environment, patch review: clues how to make easier to contribute
There are a lot of good tools for helping coders and reviewer. There should be a lot of experience in building up a dev environment *and* a patch review environment spread in the community but "hidden". If I had to put up a dev/review environment now I would: * install drupal HEAD * install coder devel modules(?) * fill drupal with some data using devel * backup * update from cvs * restore db connection setting * apply a patch - check if any schema change * test (benchmark?) * report * restore db Still this look very naive and I think there are a lot of tools that could help without getting into module frenziness and a lot of details that could streamline the process or setup that may hide some problems. There is not a compact knowledge base of several techniques that may make more profitable for people that already maintain drupal sites to contribute to core and contrib. I think that most developers are still applying common wisdom and their own hand made recipes. code coverage, deploy, drupal automated staging toolkit, drush, simpletest and auto test??? If tests are already automatically made on submitted patches, preparing a testing environment isn't so critical for a patch reviewer, but it could come handy for a developer. If I had to improve the above naive method I'd start from writing a script that given a patch url: * update from cvs * restore db connection setting * apply the patch Second thing I'd do would be to replicate the simpletest environment and start testing the patch. Then I'd choose a set of "important modules" and see if patches or HEAD breaks them. I think this looks a bit more tricky and may require some rcs wizardry especially if I've my modules to test too. Another thing I'd like to learn to do better is how to maintain modified drupal core, supposing I submitted patch waiting and I'd like to check if they still works when I update to HEAD/review a patch and mix svn (or any rcs of choice) with drupal cvs. A DVCS could come handy here... but... that's not the point here. The material about how to put up a dev/review environment is sparse. Something that may be interesting and I would like to read more about or see them grouped or add some glue so that it would be possible to guide people to a more rationalised dev/review environment: Using CVS to maintain a Drupal website http://drupal.org/node/93966 R. Keeping Your Local and Remote Sites Synchronized http://drupal.org/node/120617 Drupal development server configuration http://www.garfieldtech.com/blog/drupal-dev-server K. Moving Entire Drupal Site with Databases http://drupal.org/node/120627 Migrating Database Changes From Development to Live Websites http://drupal.org/node/199771 Drupal Performance Measurement & Benchmarking http://drupal.org/node/282862 Drupal with safe mode enabled and open basedir http://drupal.org/node/82223 More than one Drupal site on one machine http://drupal.org/node/274 Transforming a default table names installation into a prefix table names installation http://drupal.org/node/195758 How-To: Virtual Hosting with Drupal http://drupal.org/node/80472 -- Ivan Sergio Borgonovo http://www.webthatworks.it
Something I think should be included in patch review is also database load. Check any new/changed queries the patch might make and do an explain on it to see if that query is properly using indexes and not going to bring the database server to a crawl. I don't know if there is a doc page on this. If not, it might be worth adding one, giving a brief tutorial on using explain. On the benchmarking/testing, something I have found to be a nice addition to the toolbox is Siege. You can create a list of urls to different pages in your installation so you get more of a real world test (instead of benching a single page). When developing a site, I keep a url.txt file for that site just for this purpose. That way I can quickly run a benchmark. Siege - http://www.joedog.org/JoeDog/Siege (Also - if anyone has any other suggestions for benchmarking, I would love to hear it. I am always looking for new tools to use) Jamie Holly http://www.intoxination.net Skype:intoxination Phone: 1-513-252-2919 Ivan Sergio Borgonovo wrote:
There are a lot of good tools for helping coders and reviewer. There should be a lot of experience in building up a dev environment *and* a patch review environment spread in the community but "hidden".
If I had to put up a dev/review environment now I would: * install drupal HEAD * install coder devel modules(?) * fill drupal with some data using devel * backup
* update from cvs * restore db connection setting * apply a patch - check if any schema change * test (benchmark?) * report * restore db
Still this look very naive and I think there are a lot of tools that could help without getting into module frenziness and a lot of details that could streamline the process or setup that may hide some problems. There is not a compact knowledge base of several techniques that may make more profitable for people that already maintain drupal sites to contribute to core and contrib. I think that most developers are still applying common wisdom and their own hand made recipes.
code coverage, deploy, drupal automated staging toolkit, drush, simpletest and auto test???
If tests are already automatically made on submitted patches, preparing a testing environment isn't so critical for a patch reviewer, but it could come handy for a developer.
If I had to improve the above naive method I'd start from writing a script that given a patch url: * update from cvs * restore db connection setting * apply the patch
Second thing I'd do would be to replicate the simpletest environment and start testing the patch.
Then I'd choose a set of "important modules" and see if patches or HEAD breaks them. I think this looks a bit more tricky and may require some rcs wizardry especially if I've my modules to test too. Another thing I'd like to learn to do better is how to maintain modified drupal core, supposing I submitted patch waiting and I'd like to check if they still works when I update to HEAD/review a patch and mix svn (or any rcs of choice) with drupal cvs. A DVCS could come handy here... but... that's not the point here.
The material about how to put up a dev/review environment is sparse. Something that may be interesting and I would like to read more about or see them grouped or add some glue so that it would be possible to guide people to a more rationalised dev/review environment:
Using CVS to maintain a Drupal website http://drupal.org/node/93966
R. Keeping Your Local and Remote Sites Synchronized http://drupal.org/node/120617
Drupal development server configuration http://www.garfieldtech.com/blog/drupal-dev-server
K. Moving Entire Drupal Site with Databases http://drupal.org/node/120627
Migrating Database Changes From Development to Live Websites http://drupal.org/node/199771
Drupal Performance Measurement & Benchmarking http://drupal.org/node/282862
Drupal with safe mode enabled and open basedir http://drupal.org/node/82223
More than one Drupal site on one machine http://drupal.org/node/274
Transforming a default table names installation into a prefix table names installation http://drupal.org/node/195758
How-To: Virtual Hosting with Drupal http://drupal.org/node/80472
On Thu, Aug 14, 2008 at 10:53 PM, Ivan Sergio Borgonovo <mail@webthatworks.it> wrote:
There is not a compact knowledge base of several techniques that may make more profitable for people that already maintain drupal sites to contribute to core and contrib. I think that most developers are still applying common wisdom and their own hand made recipes.
Actually, there is.... http://drupal.org/patch/review and in more general there is the entire contributors guide handbook about this.
If tests are already automatically made on submitted patches, preparing a testing environment isn't so critical for a patch reviewer, but it could come handy for a developer.
One of the major reasons for push to get unit tests written is so that this basic type of review can be automated within the issue queue. You can already load these tests yourself and test them automatically.
The material about how to put up a dev/review environment is sparse.
again not correct..... Setting up a test environment to review patches: http://drupal.org/node/28245 However, I think that most of this problem is not really a lack of documentation for doing these reviews. Anyone that is sufficiently interested can find this stuff out or ask someone and learn very easily. I think more of this issue is community and infrastructure based. -Ryan
Ryan, thanks! I started another thread on this subject not knowing these pages exist. ~Jerad On Thu, Aug 14, 2008 at 9:30 AM, Ryan Cross <drupal@ryancross.com> wrote:
On Thu, Aug 14, 2008 at 10:53 PM, Ivan Sergio Borgonovo <mail@webthatworks.it> wrote:
There is not a compact knowledge base of several techniques that may make more profitable for people that already maintain drupal sites to contribute to core and contrib. I think that most developers are still applying common wisdom and their own hand made recipes.
Actually, there is.... http://drupal.org/patch/review
and in more general there is the entire contributors guide handbook about this.
If tests are already automatically made on submitted patches, preparing a testing environment isn't so critical for a patch reviewer, but it could come handy for a developer.
One of the major reasons for push to get unit tests written is so that this basic type of review can be automated within the issue queue. You can already load these tests yourself and test them automatically.
The material about how to put up a dev/review environment is sparse.
again not correct..... Setting up a test environment to review patches: http://drupal.org/node/28245
However, I think that most of this problem is not really a lack of documentation for doing these reviews. Anyone that is sufficiently interested can find this stuff out or ask someone and learn very easily. I think more of this issue is community and infrastructure based.
-Ryan
Ryan, has pointed out the main resources that we have for this that already exist. That isn't to say that things can't be improved. Catch *just* redid the patch/review page and PLEASE feel free to hop in and help fill out the issue for the new Testing section of the Getting Involved guide: http://drupal.org/node/240216. For more background on that book and where the Testing section will fit, read the front page post re: docs http://drupal.org/node/293455. Thanks Addi On Aug 14, 2008, at 9:30 AM, Ryan Cross wrote:
On Thu, Aug 14, 2008 at 10:53 PM, Ivan Sergio Borgonovo <mail@webthatworks.it> wrote:
There is not a compact knowledge base of several techniques that may make more profitable for people that already maintain drupal sites to contribute to core and contrib. I think that most developers are still applying common wisdom and their own hand made recipes.
Actually, there is.... http://drupal.org/patch/review
and in more general there is the entire contributors guide handbook about this.
If tests are already automatically made on submitted patches, preparing a testing environment isn't so critical for a patch reviewer, but it could come handy for a developer.
One of the major reasons for push to get unit tests written is so that this basic type of review can be automated within the issue queue. You can already load these tests yourself and test them automatically.
The material about how to put up a dev/review environment is sparse.
again not correct..... Setting up a test environment to review patches: http://drupal.org/node/28245
However, I think that most of this problem is not really a lack of documentation for doing these reviews. Anyone that is sufficiently interested can find this stuff out or ask someone and learn very easily. I think more of this issue is community and infrastructure based.
-Ryan
On Thu, 14 Aug 2008 23:30:50 +1000 "Ryan Cross" <drupal@ryancross.com> wrote:
On Thu, Aug 14, 2008 at 10:53 PM, Ivan Sergio Borgonovo <mail@webthatworks.it> wrote:
There is not a compact knowledge base of several techniques that may make more profitable for people that already maintain drupal sites to contribute to core and contrib. I think that most developers are still applying common wisdom and their own hand made recipes.
Actually, there is.... http://drupal.org/patch/review
again not correct..... Setting up a test environment to review patches: http://drupal.org/node/28245
The only major thing I see that differ from what I consider a naive approach is to name "label" the copy with the issue #. That setup is not going to test if everything keep working on a multi-site setup or with some prefixed tables. I'd consider interesting to include in the test some popular modules... or include some modules that just "test" a set of core API. It would be nice to share some more wisdom like Larry did about development environment. I think teaching how people can manage and integrate their everyday job and patch review/patch contribution could help a lot. eg. there are patches that are still waiting to be applied but that are required in certain environment + people maintaining their own modules. So the process to test a patch for some people may become: * keep a set of internal patch to Drupal core. * install Drupal along with some "interesting" modules as multisite + db prefix + some data produced by devel module * get Drupal HEAD * apply internal patches to core * apply issue queue patch * restore "clean" test db Now I'll gain another bit of unpopularity. If I use Drupal, I've to patch core, my patches are seldom committed but I've no easy way to check if other patches play nice with mine... I'd prefer the status quo. Reviewing doesn't look as an interesting activity... my focus will be on "when do my patch get into core" and "how do I handle next core/module update?". If on the other side reviewing patches is part of my everyday job (eg. I review changes from colleagues in my drupal shop already) a) I'll see more patches approved including mine b) I won't suffer the "task switch" effect Kathleen was mentioning c) I'll work in a uniform environment when developing my modules/core patches and an uniform environment similar to the one used by other developers making communication easier. I'm not saying someone has to be blamed. I just would like to see more discussion, learn more etc... I think there is still a lot of hidden wisdom about best practices and coming up with some sort of standardisation may lower the learning curve and provide a common dev environment that could make testing, applying patches and communication among dev easier. As Kathleen and Chris pointed out we miss a common pattern for "test patch files" in a more complex development environment (think about Drupal shops). I'm not sure if this is lack of infrastructure as you pointed out or lack of common best practices that are suited for more complex environment than just a plain Drupal install and a patch at a time. Or maybe I'm just too optimist and I believe that there should be a way to make all this a bit easier. Maybe all this fuss will help me to accept it's not going to be easy, accept that in certain environment a patch review is going to cost more than 20min and concentrate on work that can actually be done in spite of wasting time trying to turn lead into gold. I just would like to know what others think and how they manage their patch/review env when it get more complex. BTW I get an access denied here http://drupal.org/node/283840 BTW2 among the list of interesting documents I forgot to add other than /review there should be also Drupal coding standards. thanks -- Ivan Sergio Borgonovo http://www.webthatworks.it
Ivan Sergio Borgonovo wrote:
That setup is not going to test if everything keep working on a multi-site setup or with some prefixed tables.
SimpleTest uses prefixed tables, so that point is moot. The number of patches which are likely to affect multisite installs are very, very little, maybe 1/1000.
I'd consider interesting to include in the test some popular modules... or include some modules that just "test" a set of core API.
Again, SimpleTest - have you actually downloaded Drupal 7 and tried it out? As to popular modules, when you find a 7.x version of Views and CCK, please let me know. If you fancy porting some modules to 7.x early and often to help test new APIs I'm sure maintainers of those modules will welcome patches.
BTW I get an access denied here http://drupal.org/node/283840
That page is now merged into http://drupal.org/patch/review Nat
If I use Drupal, I've to patch core...
I think that is probably the root of most of your problems
I just would like to know what others think and how they manage their patch/review env when it get more complex.
The key with reviewing patches is to *not* use a complex environment. You have to test and review patches against HEAD. If you want to further test them against your customized version of drupal, then that is a separate task and isn't relevant to getting that patch into core. -Ryan
participants (6)
-
Addison Berry -
Ivan Sergio Borgonovo -
Jamie Holly -
Jerad Bitner -
Nathaniel Catchpole -
Ryan Cross