the $base_url variable is "www.mydomain.com/mysite"
If I then modify base url in the theme and phptemplate as suggested, and put the path to an image in my theme as <a href="files/pictures/whatever.jpg"> then it does not obey the base url, and tries to read the file from www.mydomain.com/files/pictures/...
Then you're doing it wrong. If $base_url is www.mydomain.com/mysite, then adding the following to your theme would be replicating the behavior of Drupal's past:
<?php global $base_url; ?> <base href="<?php print $base_url; ?>" />
That would cause links that look like files/pictures/whatever.jpg to be treated as www.mydomain.com/mysite/files/pictures/whatever.jpg. Have you been doing something different?