I know that the file system is very hard to move from the root of drupal once your site is mature, but I have tried it anyway (not on my live site, but on a clone of it). I have tried to read also as much as I could about this both in the drupal.org search and doing a general google search.
So I'm telling the list what I did and what didn't work, just in case anybody can give me any hints to understand a little bit better how things work with the file system.
My site is Drupal 5.7. I have full access to the server, database, you name it (it must be almost impossible to do this without full access). Server is linux with standard tools.
My previous setup was file system set to "private" but located in the root of drupal, in a "files" directory.
So I created a directory out of the webserver root and put my "files" directory there, with write permissions and owned by apache owner and group. Then I changed the path in admin/settings/file-system to the new path, and erased my old "files" directory at the root.
I use fckeditor to upload files and images to nodes, so these get an initial path of "/files/file" and "/files/image". Now these paths have to be moved to "/system/files/file" and "/system/files/image" once the files are out of the root. I changed this searching for all tables which contained these strings with the "search" feature of phpmyadmin, and then substituting the strings with the mysql command line (UPDATE table SET column = REPLACE(filepath,"/files/file", "/system/files/file");) I tried this of course on a sandbox install and database, and will get all the backups if I try to mess around with my actual database. There are not so many tables that contain references to files, so this can be done in about half an hour.
In any case, after adjusting a couple more things related to fckeditor seeing the new file system, everything seemed to work until I logged out. The anonymous user was unable to access the files in the new file system. Also strangely some roles could access them but some others could not, and in particular I have two roles which look very similar in terms of permissions but one can access it and the other not. I also tried giving the remaining permissions to the role whith no access, but no result, no matter what I tried.
So now I have a lot of questions: 1) is the "private" file system outside of the root always inaccessible for anonymous users? 2) Which is the permission that gives access to this type of private file system? 3) What is the effect of having "private" chosen, but the file system in the root? This seems to give access to everybody to the files and images that are referred in nodes, despite the fact of setting it as "private". 4) And how come I have such a disparate behavior of different roles, what determines which ones can access the outside private file system?
Thanks for any hints on this.
Walter,
I would love to hear more form others as I have a number of sites set to private where the folder is above the root web and I need to convert back to public with files in sites/default/files.
Anyway, I do know that it is a good idea to make sure the files folder is about the root web our out of there when using private files setting. If you don't, someone who knows the correct file name can go directly to the file. If a private file is called via a drupal node, drupal will only stream the file out to the client if they have the proper permissions.
Thanks! Mark
On Mar 19, 2008, at 8:08 AM, Walter Garcia-Fontes wrote:
I know that the file system is very hard to move from the root of drupal once your site is mature, but I have tried it anyway (not on my live site, but on a clone of it). I have tried to read also as much as I could about this both in the drupal.org search and doing a general google search.
So I'm telling the list what I did and what didn't work, just in case anybody can give me any hints to understand a little bit better how things work with the file system.
My site is Drupal 5.7. I have full access to the server, database, you name it (it must be almost impossible to do this without full access). Server is linux with standard tools.
My previous setup was file system set to "private" but located in the root of drupal, in a "files" directory.
So I created a directory out of the webserver root and put my "files" directory there, with write permissions and owned by apache owner and group. Then I changed the path in admin/settings/file-system to the new path, and erased my old "files" directory at the root.
I use fckeditor to upload files and images to nodes, so these get an initial path of "/files/file" and "/files/image". Now these paths have to be moved to "/system/files/file" and "/system/files/image" once the files are out of the root. I changed this searching for all tables which contained these strings with the "search" feature of phpmyadmin, and then substituting the strings with the mysql command line (UPDATE table SET column = REPLACE(filepath,"/files/file", "/system/files/ file");) I tried this of course on a sandbox install and database, and will get all the backups if I try to mess around with my actual database. There are not so many tables that contain references to files, so this can be done in about half an hour.
In any case, after adjusting a couple more things related to fckeditor seeing the new file system, everything seemed to work until I logged out. The anonymous user was unable to access the files in the new file system. Also strangely some roles could access them but some others could not, and in particular I have two roles which look very similar in terms of permissions but one can access it and the other not. I also tried giving the remaining permissions to the role whith no access, but no result, no matter what I tried.
So now I have a lot of questions: 1) is the "private" file system outside of the root always inaccessible for anonymous users? 2) Which is the permission that gives access to this type of private file system? 3) What is the effect of having "private" chosen, but the file system in the root? This seems to give access to everybody to the files and images that are referred in nodes, despite the fact of setting it as "private". 4) And how come I have such a disparate behavior of different roles, what determines which ones can access the outside private file system?
Thanks for any hints on this.
-- Walter Garcia-Fontes Barcelona
-- [ Drupal support list | http://lists.drupal.org/ ]
On Wed, 19 Mar 2008 09:21:43 -0400 Mark Shropshire mdshrops@shropnet.com wrote:
Walter,
I would love to hear more form others as I have a number of sites set to private where the folder is above the root web and I need to convert back to public with files in sites/default/files.
Anyway, I do know that it is a good idea to make sure the files folder is about the root web our out of there when using private files setting. If you don't, someone who knows the correct file name can go directly to the file. If a private file is called via a drupal node, drupal will only stream the file out to the client if they have the proper permissions.
It is not "the solution" but it works. You just have to add an .htaccess that deny all direct access.
Good point Ivan.. Thanks!
Do you happen to know what to put into .htaccess to make this happen so I can keep in my notes?
Also.. I think there was a summer of code project to allow private and public files simultaneously. I haven't tried the module, but that would be nice so you don't have to stream every image just because you need a few files protected.
On Mar 19, 2008, at 9:36 AM, Ivan Sergio Borgonovo wrote:
On Wed, 19 Mar 2008 09:21:43 -0400 Mark Shropshire mdshrops@shropnet.com wrote:
Walter,
I would love to hear more form others as I have a number of sites set to private where the folder is above the root web and I need to convert back to public with files in sites/default/files.
Anyway, I do know that it is a good idea to make sure the files folder is about the root web our out of there when using private files setting. If you don't, someone who knows the correct file name can go directly to the file. If a private file is called via a drupal node, drupal will only stream the file out to the client if they have the proper permissions.
It is not "the solution" but it works. You just have to add an .htaccess that deny all direct access.
-- Ivan Sergio Borgonovo http://www.webthatworks.it
-- [ Drupal support list | http://lists.drupal.org/ ]
On Wed, 2008-03-19 at 09:38 -0400, Mark Shropshire wrote:
Good point Ivan.. Thanks!
Do you happen to know what to put into .htaccess to make this happen so I can keep in my notes?
Also.. I think there was a summer of code project to allow private and public files simultaneously. I haven't tried the module, but that would be nice so you don't have to stream every image just because you need a few files protected.
Just as a datapoint, I have been very disappointed with drupal's performance until I got rid of that private filesystem thing. Filtered access through drupal (5.x) and direct access (i.e. just apache) are like day and night in terms of speed. Previously I could watch my acidfree galleries display one thumbnail per second, now the whole page displays instantly.
I understand the /system/ dir is needed for access control, but the way it's implemented looks suboptimal to me. Not that I have a better idea right now.
Xav
On 20/03/2008, Xavier Bestel xavier.bestel@free.fr wrote:
Just as a datapoint, I have been very disappointed with drupal's performance until I got rid of that private filesystem thing. Filtered access through drupal (5.x) and direct access (i.e. just apache) are like day and night in terms of speed. Previously I could watch my acidfree galleries display one thumbnail per second, now the whole page displays instantly.
Yeah it isn't a good solution for popular pages that require lots of images downloaded to display - as each image is a whole new Drupal bootstrap. Private downloads turns a gallery listing that was 1 Drupal bootstrap and 20 Apache downloads into 21 Drupal bootstraps which is a whole lot more server load (up to 20x more).
But it is good for standalone downloads (eg not on-page images) that don't get downloaded that often but still need protection. It doesn't add much extra load in that scenario.
I understand the /system/ dir is needed for access control, but the way it's implemented looks suboptimal to me. Not that I have a better idea right now.
There isn't much that can be done about that. If Drupal needs to decide whether or not to allow access to the file, Drupal needs to be bootstrapped.
One (security through obscurity) possibility that might work for some sites: Your public downloads could be protected by Apache checking for "authorised" referer headers. If the referer header contains url(s) that only authorised Drupal uses can get to, you could assume they were authorised to download the files.
An unauthorised visitor would have to forge those headers to get around that. Although it is only "Security through obscurity" that should be enough of a barrier for most sites/webmasters to be happy with.
On Wed, 19 Mar 2008 09:38:38 -0400 Mark Shropshire mdshrops@shropnet.com wrote:
Good point Ivan.. Thanks!
Do you happen to know what to put into .htaccess to make this happen so I can keep in my notes?
order allow,deny deny from all
but then everything you put in files will be "private". That means that if you're publishing articles that contains pics, those pics will have to be served by PHP and by default there no way to allow access to some rule etc... You could surely place stuff through ftp in a different dir etc... but it is a bit of pain.
You could: files (no .htaccess limits) files/private (.htaccess) + ftp
you could write a "smarter" .htaccess that limit direct access to files with a certain pattern and instruct your users to upload node attachment following that rule. It would still be a PITA even if your users are willing to be instructed ;)
There are modules that let you have private/public access: filemanager and private_upload I ended up coding my own solution due to my limited needs so I can't comment if the above may be up to the task, but they seemed the most promising.
Maybe fckeditor (and others) can upload files in subdirs of files/. Be warned that such kind of toys may let people save files *below* your "root"... I think fckeditor has a strong enough reputation to avoid the problem... but I'd check the configuration of such kind of beasts before making any serious plan.
Ivan,
Thanks much. I will take your advise and think on it.
Mark
On Mar 19, 2008, at 10:57 AM, Ivan Sergio Borgonovo wrote:
On Wed, 19 Mar 2008 09:38:38 -0400 Mark Shropshire mdshrops@shropnet.com wrote:
Good point Ivan.. Thanks!
Do you happen to know what to put into .htaccess to make this happen so I can keep in my notes?
order allow,deny deny from all
but then everything you put in files will be "private". That means that if you're publishing articles that contains pics, those pics will have to be served by PHP and by default there no way to allow access to some rule etc... You could surely place stuff through ftp in a different dir etc... but it is a bit of pain.
You could: files (no .htaccess limits) files/private (.htaccess)
ftp
you could write a "smarter" .htaccess that limit direct access to files with a certain pattern and instruct your users to upload node attachment following that rule. It would still be a PITA even if your users are willing to be instructed ;)
There are modules that let you have private/public access: filemanager and private_upload I ended up coding my own solution due to my limited needs so I can't comment if the above may be up to the task, but they seemed the most promising.
Maybe fckeditor (and others) can upload files in subdirs of files/. Be warned that such kind of toys may let people save files *below* your "root"... I think fckeditor has a strong enough reputation to avoid the problem... but I'd check the configuration of such kind of beasts before making any serious plan.
-- Ivan Sergio Borgonovo http://www.webthatworks.it
-- [ Drupal support list | http://lists.drupal.org/ ]
-
Anton -
Ivan Sergio Borgonovo -
Mark Shropshire -
Walter Garcia-Fontes -
Xavier Bestel