Dries Buytaert wrote:
I do think we're doing great work on testing. I've been committing testing related patches on a (almost) daily basis. That's pretty sweet.
Like catch and others, I've been dumping basically all the time I have available to work on D7 on testing stuff, in the hopes that it will help extend the code freeze out to when I will have much MORE time to dump into D7 in the fall. ;)
That said, we aren't able to measure our test coverage yet. In other words, it's really hard to tell how well we actually do. Any updates on that?
I think that our current test coverage by conventional tools is going to be close to 0%. It's also clear that the community has not in fact "embraced testing." Instead, a hardcore group of around 15-20 people (the "testing brigade" ;)) have, and are driving this effort forward on behalf of the rest of the development team. I also think that both of these situations are okay right now. Because what the "testing brigade" has been focusing on is: 1. Improvement of testing tools. For example, that Batch API patch that was just committed I think is the *key* turning point that will make testing something that can be done by normal humans rather than just the "testing brigade." 2. Fixing of existing tests. Back when I wrote http://webchick.net/itch-of-the-week/fix-testing-crisis, we were in pretty sorry shape in this regard. I remember growing the critical bug queue by at least 20 in one night documenting all the tests that didn't pass. Now, we're down to 3 failing tests as of this morning. 3. Improving coverage of tests that run through the end-user experience via the browser. This helps save our reviewers from getting carpal tunnel clicking on forms to ensure that the basic system is running properly while they're testing a patch. 4. Developing testing guidelines, best practices. We're not totally there yet, but a fairly large amount of time has been spent on things like documentation, clean-up of tests so that they all follow basically the same conventions, etc. This type of work is important to getting new developers on the testing train. However, all of this "clean-up" work has come at the expense of dramatically increasing our testing coverage. But I actually think that it would've been way too premature to dump a bunch of time into increasing our test coverage while the above 4 points were outstanding. My goal is for the "Awesome Testing Party" at Drupalcon Szeged (if it gets accepted) to help a lot in this regard. -Angie