I've done some further testing on this. Apache/2.0.52 (Win32) mod_perl/1.99_12 Perl/v5.8.2 PHP/4.3.4, Turck's mmCache running on Win XP SP2
Start with zlib.output_compression = off in php.ini Drupal cache off Logout Firefox and IE6 work fine.
Now turn Drupal cache on and log out All pages display in both firefox and IE6. Pages can be seen to be cached in the dbms. Using the firefox live http headers extension you can see either the 304 unmodified or Content-Encoding: gzip headers. What may be missing is the Vary: Accept-Encoding header
Now switch zlib.output_compression = on in php.ini and restart apache new pages display fine in firefox and IE6. Cached pages display as binary in firefox and throw an error in IE6 "unknown content type". New pages are served with no gzip. Cached pages are either 304 or show Content-Encoding: gzip.
Now get back to a page that is fresh so you can login. login and turn Drupal cache off. All pages now display fine in both Firefox and IE6. Returned headers show Content-Encoding: gzip Vary: Accept-Encoding
As a final test I went into bootstrap.inc and in drupal_page_header() I commented out the line header('Content-Encoding: gzip'); and changed the browser gzip check so it always inflated the cached data and everything worked fine with cache enabled and php set to compress.
So it looks pretty clear to me that Drupal should not be attempting to gzip encode the returned cache page if php or apache is also gzip encoding everything. I don't really see the benefit of storing the cached data gzip encoded as even on vast site, there's not going to be that much html.
The short term solution is just to always inflate the cached page before the print statement. The longer term solution is to provide an admin switch so that people without access to php.ini, .htaccess and so on, can choose.
And/or test for zlib_get_coding_type()=='gzip' to see whether it's turned on and take appropriate action. However this last will not find mod_gzip if that's installed in apache. Here's one way of doing it. Around line 262 of bootstrap.inc
// Determine if the browser accepts gzipped data. if ( zlib_get_coding_type() == 'gzip' || (@strpos($_SERVER['HTTP_ACCEPT_ENCODING'], 'gzip') === false && function_exists('gzencode')) ) {