A better solution would be to make the .desc files optional (i.e. to be encouraged for use by all contrib modules), and to keep the 'admin/modules#description' but in hook_help for core modules. When loading the admin/modules page, Drupal can first look for a .desc file for each module, and if one is not found, then load the whole module file and call hook_help.
+1 This a path of least resistance appraoch. In the future, we could change the .desc files to .info, and have a lot more stuff in there (description, author, version, dependancies, ...etc.) Right now, a .desc scheme for contrib will solve the memory issue and avoid the need to load the module. It can even be scripted to generate the .desc from the existing _help now to give everyone a headstart.