Ok, let's completely abandon the idea of automatically unit testing patches. Instead let's have a block on http://drupal.org/project/ issues that has a file upload. The file upload connects to a server that is available to unit test the patch.
It spits back a test result number with the time the test was run and the results of the test.
Instead of spending all this effort automating some thing simple we focus on making it a valuable service to the developer community. The person who posts the patch can just provide a link to the unit test result.
Keep It Simple S******
Kieran
On the first view your approach has one advantage: Developers can post patches before they "release" them to the issue tracker. Well but this is actually even more easy to do by just locally installing the simpletest framework. So you can test your modifications even before generating the patch! While the narrowed approach would still be almost just as useful I worry it will not be used that much. It simply is more work to: - post patch to test server - wait for result (automated testing does take significant time) - check result - post patch to the issue tracker including a link in contrast to: - post patch (that you think is good) to issue tracker - check result once its out (just while checking for other human replies) I am not suspecting the devel community to be lazy but I know from myself that I would only do this with the bigger patches. Testing generally is a boring thing, thats why we want to automate it (which in contrast is way more interesting). This may lead to some smaller patches going through untested and thus possibly breaking the system by making a fresh HEAD fail the tests, this is a very bad situation for future testing. That is why I believe the forced test process is way more valuable. The amount of work needed for the reduced solution is less, no doubt. - Thomas