Hi Nat,
That's not what it says, or at least not what's happening. It says that if null, it receives the default permissions stated in _drupalCreateRole() (which is array('access comments', 'access content', 'post comments', 'post comments without approval') according to
http://api.drupal.org/api/file/modules/simpletest/drupal_web_test_case.php/7/source). But, this does not influence the permissions every user receives by the hard coded Authenticated User role. Not directly, but by association/inheritance.
This behavior is expected, because it's the Drupal way of permission inheritance. But to test the behavior I still need to find a way to alter the permissions not my newly created user+role, but of the global Authenticated User Role. You can try it by removing 'post comments' from row 27 of comment.test. You'd expect al sorts of failures, because you just made web_user not able to create comments, but everything passes effortlessly.