The Drupal SimpleTest module has progressed far beyond the original
SimpleTest we started with - we now have a more powerful, although
Drupal-specific testing framework which still retains the somewhat
misleading name "SimpleTest" (there's an issue somewhere to rename
SimpleTest to testing). We currently do have both functional and unit
tests working under the current framework - although javascript tests
currently remain impossible, although there are several issues
addressing that point (feel free to help!).
I think what you're saying confirms that while 'far-beyond the original simpletest' - the current testing framework in Drupal still has this in common with the standard version of Simpletest -- it's primarily focused on unit/functional testing - not acceptance testing, which is a whole different focus. ( http://en.wikipedia.org/wiki/Acceptance_testing )
To help make what I'm talking about more understandable, here's a list of what selenium (for instance) offers that simpletest (and the Drupal implementation) does not:
a) a top-rated browser plugin which offers an interface for scripting, exporting, and running tests specifically aimed at the browser-based aspect of a web application (again *not* unit/functional testing) (screenshots and explanation
http://www.spreadfirefox.com/node/980 )
b) an additional tool, Selenium RC, which includes functionlity to
automatically launch a variety of browsers and step through the tests in each browser - as an end-user would do
c) testing of ajax/js ui elements
But basically, our Drupal-specific extension of SimpleTest meets our
needs for functional testing and unit testing so well, I don't think
there's really any point to looking into other harder-to-install
automated testing tools - they'll never be used by a sizable percentage
of the Drupal community, whereas the SimpleTest module for Drupal
requires very little in order to function properly. The current
framework covers the vast majority of our testing needs; the tests are
lagging far behind the testing framework here. We don't really need to
rethink our framework or consider other frameworks until we've reached
the limit of what the current framework can do. JavaScript tests would
certainly be nice, but should we really worry ourselves with them when
much important core php code remains entirely untested?
Again, acceptance testing/selenium does not just = adding-javascript support to unit tests - it equals a whole other kind of testing - one I believe that a very sizable percentage of the Drupal community is interested in I'm sure - testing from the end-user/client perspective. So this would not be a matter of rethinking the existing framework - it's would be adding on to it and/or getting a way to more easily get selenium working with Drupal so that Drupal developers can include accetance test within their workflow.
For anyone who doesn't have the bandwidth or the interest to work on finding solutions to better acceptance testing within Drupal I can understand - but I just wanted to point out the difference between what is in Drupal now and what I'm talking about. They are different and it's not just about what's in Drupal 'missing javascript support'.
- Caleb