[development] Drupal Core Test Plan

[Angela Byron]

One of my specialties (or at least something I get roped into doing  
fairly often ;)) is testing things with a fine-toothed comb. I have  
no formal background in testing methodologies or anything like that  
at all, but in my experience proper testing involves both:

a) Figuring out all the various bits of functionality each sub-set of  
Drupal offers and making sure they all work.
b) Intentionally throwing invalid stuff at the system or using it in  
unexpected ways to make sure it recovers.

Over last weekend, chx and I were tag-teaming the forthcoming D6  
issue queue; I was going through all aspects of core in detail  
identifying critical issues, chx was rolling patches for them, and I  
would then review/RTBC the patches he came up with. About halfway  
through, I decided it might be useful to document all the stuff that  
I was testing, and came up with this document:

http://groups.drupal.org/node/5974

It's pretty rough at this point, and I was hoping to get much further  
with it before I posted it to the list, but I figured better early  
than never. ;) It's a wiki page, and I hope that others will  
contribute to extending the list to cover all of core. As new bugs  
come up, we can add tests that cover them to the list.

Among the things we could do with this:

- Use it purely as a means of cataloguing all of the bugs that have  
broken core in one way or another. Before a release, we "spot check"  
a few of these trouble areas and make sure they're good to go.

- Before any release, we split up the various sections among  
individuals, so that each person can take a component (node system,  
user system, etc.) and make sure all of the tests there pass. This  
way we have no more of this "one person trying to test all of core by  
herself" business again. ;)

- We can also use this as a check-list for our library of SimpleTest  
automated tests (or Selenium or whatever). While it takes awhile to  
write all of these, once they were written "in theory" no human would  
ever need to test those aspects again.

If anyone can suggest improvements or wants to help fill out this  
list, by all means do! This is one of those tasks that literally  
anyone can help with, regardless of your Drupal experience, and it's  
a great way to get into that "critical eye" mindset that can help  
make you a good tester.

Let's kick those critical bugs to the curb! :)
-Angie