Larry Garfield wrote:
webchick lamented the lack of "embracing testing" and that we have only a "testing brigade". I'm actually not surprised at that. That's known as a "QA Team" in many circles, and is a very good thing. A semi-dedicated QA team that beats people over the head is a more efficient use of resources than trying to get everyone to be everything for every patch. My hats off to the "testing brigade" for their ongoing work, and may they continue to thwap the rest of us when appropriate. (Incidentally, is it my imagination or does the Testing Brigade consist mostly of former GHOP students and mentors?) Their work is made even harder by the time needed to basically rewrite our own testing framework, since we're apparently not sticking with any existing testing framework (which seems inline with Drupal's usual policy...)
A couple things: 1. I wasn't lamenting anything. I was simply stating the reality. The reality is that the community hasn't remotely "embraced testing," and a huge part of that is because the testing brigade has had its hands more than full hammering out larger over-arching issues with the testing framework and existing tests, which I proceeded to document in the rest of my reply. 2. I'm fine with this reality, and at the present point in time it would be silly to expect anything else. If we had tried to get 800+ developers to "embrace testing" back in March, it would've been a complete CF. I'm saying that NOW, we're in a much better position to have a larger body of developers working on testing, thanks to the hard work of the testing brigade over the past several months. 3. We /are/ going to need 800+ people fluent in writing tests by the time the D7 release is upon us. 15-20 people simply cannot provide 100% (or even 10%) test coverage on their own, nor should we want that to happen. The testing brigade should be focusing on doing "test reviews," mentoring developers on how to write good tests and answering their questions, checking for thorny areas which need more test coverage, fix issues in the testing framework that are beyond regular developers' expertise, etc. 4. Yes, there's a strong GHOP tie-in. I think that's because most of the people who helped out with and participated in GHOP are the same people who generally help out with and participate in any community initiative. And the same people who didn't... well... ;) 5. The testing framework re-write I was originally very opposed to. But the fact is that SimpleTest had all manner of problems running with our code, and going in there and monkeying with the code to fix it was a huge time-sink every single time. The code that's there now is at least understandable by more than 3 people, it's fully documented, and it also takes far less time to run the full test suite. In general, I'd say it's a huge win. 6. I agree about testable APIs being another huge component of this. The solution of course is to grow the size of the testing brigade, and the size of the body of people who know how to write tests. If this area was more well-covered, developers who'd normally be putting efforts into API re-writes could resume doing so. As it is currently, however, everyone in the testing brigade has more than enough to do on that single area of focus. -Angie