[development] Drupal and acceptance testing

[Caleb Gilbert]

The Drupal SimpleTest module has progressed far beyond the original
SimpleTest we started with - we now have a more powerful, although
Drupal-specific testing framework which still retains the somewhat
misleading name "SimpleTest" (there's an issue somewhere to rename
SimpleTest to testing). We currently do have both functional and unit
tests working under the current framework - although javascript tests
currently remain impossible, although there are several issues
addressing that point (feel free to help!).

I think what you're saying confirms that while 'far-beyond the original
simpletest'  - the current testing framework in Drupal still has this in
common with the standard version of Simpletest -- it's primarily focused on
unit/functional testing - not acceptance testing, which is a whole different
focus. ( http://en.wikipedia.org/wiki/Acceptance_testing )

To help make what I'm talking about more understandable, here's a list of
what selenium (for instance) offers that simpletest (and the Drupal
implementation) does not:
a) a top-rated browser plugin which offers an interface for scripting,
exporting, and running tests specifically aimed at the browser-based aspect
of a web application (again *not* unit/functional testing) (screenshots and
explanation http://www.spreadfirefox.com/node/980 )
b) an additional tool, Selenium RC, which includes functionlity to
automatically launch a variety of browsers and step through the tests in
each browser - as an end-user would do
c) testing of ajax/js ui elements


But basically, our Drupal-specific extension of SimpleTest meets our
needs for functional testing and unit testing so well, I don't think
there's really any point to looking into other harder-to-install
automated testing tools - they'll never be used by a sizable percentage
of the Drupal community, whereas the SimpleTest module for Drupal
requires very little in order to function properly. The current
framework covers the vast majority of our testing needs; the tests are
lagging far behind the testing framework here. We don't really need to
rethink our framework or consider other frameworks until we've reached
the limit of what the current framework can do. JavaScript tests would
certainly be nice, but should we really worry ourselves with them when
much important core php code remains entirely untested?

Again, acceptance testing/selenium does not just = adding-javascript support
to unit tests - it equals a whole other kind of testing - one I believe that
a very sizable percentage of the Drupal community is interested in I'm sure
- testing from the end-user/client perspective. So this would not be a
matter of rethinking the existing framework - it's would be adding on to it
and/or getting a way to more easily get selenium working with Drupal so that
Drupal developers can include accetance test within their workflow.

For anyone who doesn't have the bandwidth or the interest to work on finding
solutions to better acceptance testing within Drupal I can understand - but
I just wanted to point out the difference between what is in Drupal now and
what I'm talking about. They are different and it's not just about what's in
Drupal 'missing javascript support'.

- Caleb
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.drupal.org/pipermail/development/attachments/20080821/ac60c3ca/attachment.htm