Files table includes file_directory_path
Date: Wed, 15 Jul 2009 17:09:21 +0200 From: Clemens Tolboom <clemens@build2be.nl> Subject: [development] Why is the Drupal File System relative to website root and not relative to admin/settings/file-system path?
Having a development test acceptance and production environment it's a drag when moving the production database and files to an acceptance or development environment.
One has to update the files database table and more tables or do trickery with symlinks to get the site running clean.
If we could make the filepath in the files table relative to the /admin/settings/file-system path the real files are relocatable without pain. Just do a db load and a rsync to get the files and change the dmin/settings/file-system path.
What do you think? Did I miss something important?
There is a featurebug in Drupal core that has been around since Drupal 5 or before, where the file API allows modules to save and retrieve files using a path relative to either the Drupal root or variable_get('file_directory_path', ...). A number of people have tried to fix this (including myself) in Drupal core, but it is a rather overwhelming issue since so much code in core and contrib now expects not only the file API, but also the files table to include variable_get('file_directory_path', ...). Attempts to fix it in the past sometimes gets distracted by related issues such as sites/sites.php and default files_directory_path, which don't actually solve the featurebug. At CivicActions we have a tool (pushdb --xFix) that fixes the paths in the files directory and elsewhere in the db, which we run when copying an instance of the site to a staging environment. However this approach is becoming unsustainable and we are considering using a separate instances of Drupal core in each and every staging environment so that they all use sites/default. What this means is that much of the time this featurebug is such a PITA that Drupal's multisite features don't work for staging environments. In my own sandbox I don't use multisite for staging environments at all, because of this issue, Do others?
Message: 6 Date: Wed, 15 Jul 2009 19:02:38 +0300 From: ufuk bayburt <ufukbay@gmail.com> Subject: Re: [development] Why is the Drupal File System relative to website root and not relative to admin/settings/file-system path?
That's the cleanest fix for this bug I've seen yet, but still seems to have wider ramifications than that patch fixes, and that's just Drupal core. I think it also needs an update handler that fixes paths in the files table for Drupal 6 sites upgrading to Drupal 7.
Date: Wed, 15 Jul 2009 11:08:44 -0500 From: James Gilliland <neclimdul@gmail.com> Subject: Re: [development] Why is the Drupal File System relative to website root and not relative to admin/settings/file-system path?
Set up your production filesystem path first and set that before developing your site?
You misunderstood the initial post. The poster is asking about the case where he/she creates a staging environment from the live environment, not the other way around. However you've raised another issue derived from this featurebug. If you are developing a site on a domain which will not be the final domain name (e.g. because the site you're developing is replacing a non-drupal site), and your conf_path() is not sites/default, then you need to set your files_directory_path beforehand and/or create a system of symlinks to workaround the featurebug (or be screwed if your on windows). The foresight usually come with the experience of having been burned by this featurebug – probably when you deployed a site and all the files broke a previous time. This hurts the developer experience.
Date: Wed, 15 Jul 2009 12:39:17 -0400 From: Moshe Weitzman <weitzman@tejasa.com> Subject: Re: [development] Why is the Drupal File System relative to website root and not relative to admin/settings/file-system path?
fixed in d7 with sites/sites.php. you can map one path to another so the that two environments can be made to look the same. see http://api.drupal.org/api/function/conf_path/7.
of course d6 still needs some tricks.
While that's a great feature it doesn't fix this featurebug, it just provides another workaround for when it breaks and for those with foresight. Bevan/
On Wed, Jul 15, 2009 at 5:54 PM, Bevan Rudge <bevan@civicactions.com> wrote:
At CivicActions we have a tool (pushdb --xFix) that fixes the paths in the files directory and elsewhere in the db, which we run when copying an instance of the site to a staging environment. However this approach is becoming unsustainable and we are considering using a separate instances of Drupal core in each and every staging environment so that they all use sites/default.
What this means is that much of the time this featurebug is such a PITA that Drupal's multisite features don't work for staging environments. In my own sandbox I don't use multisite for staging environments at all, because of this issue, Do others?
I don't use multisite for managing dev, staging and production environments. In my workflow, it would be *more* complicated to use this method. I put the entirety of Drupal core into version control and deploy working spaces straight from version control. This makes it much easier to control exactly what and when code is pushed to production, and enable the ability navigate through the history to find sources of bugs. The only time I use multisite is for actual separate, yet integrated websites. The most common use for me are multiple websites that share tables, like the user-related tables.
Storing a file's path which may change in the future inside the database is a bug no matter what the use case is. I had once developed a site and then decided to move the files from sites/default/ to sites/<domain-name>/ in order to create a new site using the same installation and was very surprised at seeing this bug. The path is stored in the files and users table (for profile pics) I believe. On Thu, Jul 16, 2009 at 1:15 AM, Kathleen Murtagh <kathleen@ceardach.com>wrote:
On Wed, Jul 15, 2009 at 5:54 PM, Bevan Rudge <bevan@civicactions.com>wrote:
At CivicActions we have a tool (pushdb --xFix) that fixes the paths in the files directory and elsewhere in the db, which we run when copying an instance of the site to a staging environment. However this approach is becoming unsustainable and we are considering using a separate instances of Drupal core in each and every staging environment so that they all use sites/default.
What this means is that much of the time this featurebug is such a PITA that Drupal's multisite features don't work for staging environments. In my own sandbox I don't use multisite for staging environments at all, because of this issue, Do others?
I don't use multisite for managing dev, staging and production environments. In my workflow, it would be *more* complicated to use this method. I put the entirety of Drupal core into version control and deploy working spaces straight from version control. This makes it much easier to control exactly what and when code is pushed to production, and enable the ability navigate through the history to find sources of bugs.
The only time I use multisite is for actual separate, yet integrated websites. The most common use for me are multiple websites that share tables, like the user-related tables.
-- Ashraf Amayreh http://aamayreh.org
As others said, you either use symlinks (which forces you to have two directories per site), or the new sites.php feature of Drupal 7. Using that, you can have a contrived name for each site (even site1, site2, or an md5 hash for each site), and redirect the site in it. The trick is to not use sites/default for each site from now on, and only use a unique identifier. That identifier can be the same when you develop the site, and remains the same when you deploy the site. On Thu, Jul 16, 2009 at 9:50 AM, Ashraf Amayreh <mistknight@gmail.com>wrote:
Storing a file's path which may change in the future inside the database is a bug no matter what the use case is.
I had once developed a site and then decided to move the files from sites/default/ to sites/<domain-name>/ in order to create a new site using the same installation and was very surprised at seeing this bug. The path is stored in the files and users table (for profile pics) I believe.
On Thu, Jul 16, 2009 at 1:15 AM, Kathleen Murtagh <kathleen@ceardach.com>wrote:
On Wed, Jul 15, 2009 at 5:54 PM, Bevan Rudge <bevan@civicactions.com>wrote:
At CivicActions we have a tool (pushdb --xFix) that fixes the paths in the files directory and elsewhere in the db, which we run when copying an instance of the site to a staging environment. However this approach is becoming unsustainable and we are considering using a separate instances of Drupal core in each and every staging environment so that they all use sites/default.
What this means is that much of the time this featurebug is such a PITA that Drupal's multisite features don't work for staging environments. In my own sandbox I don't use multisite for staging environments at all, because of this issue, Do others?
I don't use multisite for managing dev, staging and production environments. In my workflow, it would be *more* complicated to use this method. I put the entirety of Drupal core into version control and deploy working spaces straight from version control. This makes it much easier to control exactly what and when code is pushed to production, and enable the ability navigate through the history to find sources of bugs.
The only time I use multisite is for actual separate, yet integrated websites. The most common use for me are multiple websites that share tables, like the user-related tables.
-- Ashraf Amayreh http://aamayreh.org
-- Khalid M. Baheyeldin 2bits.com, Inc. http://2bits.com Drupal optimization, development, customization and consulting. Simplicity is prerequisite for reliability. -- Edsger W.Dijkstra Simplicity is the ultimate sophistication. -- Leonardo da Vinci
sites.php was actually added specifically for this sort of issue, because the sites/ directory structure was too brittle. Of course, for the files directories in particular I have long since dropped using sites/<sitename>/files in favor of files/<sitekey>, which sidesteps the issue entirely. That doesn't need to change no matter what server the site moves to. Khalid Baheyeldin wrote:
As others said, you either use symlinks (which forces you to have two directories per site), or the new sites.php feature of Drupal 7.
Using that, you can have a contrived name for each site (even site1, site2, or an md5 hash for each site), and redirect the site in it.
The trick is to not use sites/default for each site from now on, and only use a unique identifier. That identifier can be the same when you develop the site, and remains the same when you deploy the site.
On Thu, Jul 16, 2009 at 9:50 AM, Ashraf Amayreh <mistknight@gmail.com <mailto:mistknight@gmail.com>> wrote:
Storing a file's path which may change in the future inside the database is a bug no matter what the use case is.
I had once developed a site and then decided to move the files from sites/default/ to sites/<domain-name>/ in order to create a new site using the same installation and was very surprised at seeing this bug. The path is stored in the files and users table (for profile pics) I believe.
On Thu, Jul 16, 2009 at 1:15 AM, Kathleen Murtagh <kathleen@ceardach.com <mailto:kathleen@ceardach.com>> wrote:
On Wed, Jul 15, 2009 at 5:54 PM, Bevan Rudge <bevan@civicactions.com <mailto:bevan@civicactions.com>> wrote:
At CivicActions we have a tool (pushdb --xFix) that fixes the paths in the files directory and elsewhere in the db, which we run when copying an instance of the site to a staging environment. However this approach is becoming unsustainable and we are considering using a separate instances of Drupal core in each and every staging environment so that they all use sites/default.
What this means is that much of the time this featurebug is such a PITA that Drupal's multisite features don't work for staging environments. In my own sandbox I don't use multisite for staging environments at all, because of this issue, Do others?
I don't use multisite for managing dev, staging and production environments. In my workflow, it would be *more* complicated to use this method. I put the entirety of Drupal core into version control and deploy working spaces straight from version control. This makes it much easier to control exactly what and when code is pushed to production, and enable the ability navigate through the history to find sources of bugs.
The only time I use multisite is for actual separate, yet integrated websites. The most common use for me are multiple websites that share tables, like the user-related tables.
-- Ashraf Amayreh http://aamayreh.org
-- Khalid M. Baheyeldin 2bits.com <http://2bits.com>, Inc. http://2bits.com Drupal optimization, development, customization and consulting. Simplicity is prerequisite for reliability. -- Edsger W.Dijkstra Simplicity is the ultimate sophistication. -- Leonardo da Vinci
On Thu, 2009-07-16 at 10:18 -0500, larry@garfieldtech.com wrote:
sites.php was actually added specifically for this sort of issue, because the sites/ directory structure was too brittle.
Of course, for the files directories in particular I have long since dropped using sites/<sitename>/files in favor of files/<sitekey>, which sidesteps the issue entirely. That doesn't need to change no matter what server the site moves to.
What is your sitekey structure in case of a multi-site environement and what when doing a staging scenario ie move production to a demo of regression environment? (With D5 multisite I moved from the files/<sitename> to sites/<sitename>/files to get rid of painfull all at once D5 updates.)
Clemens Tolboom wrote:
What is your sitekey structure in case of a multi-site environement I do the same as Crell. It can be anything that makes sense to you.
For example, on my Drupal documentation site instead of sites/nanwich.info/files, I use files/docs. -- Nancy E. Wichmann, PMP Injustice anywhere is a threat to justice everywhere. -- Dr. Martin L. King, Jr
On Friday 17 July 2009 2:44:30 pm Clemens Tolboom wrote:
On Thu, 2009-07-16 at 10:18 -0500, larry@garfieldtech.com wrote:
sites.php was actually added specifically for this sort of issue, because the sites/ directory structure was too brittle.
Of course, for the files directories in particular I have long since dropped using sites/<sitename>/files in favor of files/<sitekey>, which sidesteps the issue entirely. That doesn't need to change no matter what server the site moves to.
What is your sitekey structure in case of a multi-site environement and what when doing a staging scenario ie move production to a demo of regression environment?
(With D5 multisite I moved from the files/<sitename> to sites/<sitename>/files to get rid of painfull all at once D5 updates.)
Anything recognizable to me will work. For example, on a D5 multi-site for a Chicago Pre-School program (http://www.virtualk.org/ and http://www.virtualpre-k.org) we used "vk" and "vpk" subdirectories. On another install where we're rolling out 40-50 micro sites all as third-level or fourth-level domains, we just use that part of the domain. (So foo.example.com and bar.example.com become files/foo and files/bar.) Drupal doesn't care what your files directory is or if it maps to a domain name; it just cares that you don't change it part-way through the site's life time. That's when stuff gets weird. -- Larry Garfield larry@garfieldtech.com
Quoting Clemens Tolboom <clemens@build2be.nl>:
On Thu, 2009-07-16 at 10:18 -0500, larry@garfieldtech.com wrote:
sites.php was actually added specifically for this sort of issue, because the sites/ directory structure was too brittle.
Of course, for the files directories in particular I have long since dropped using sites/<sitename>/files in favor of files/<sitekey>, which sidesteps the issue entirely. That doesn't need to change no matter what server the site moves to.
What is your sitekey structure in case of a multi-site environement and what when doing a staging scenario ie move production to a demo of regression environment?
(With D5 multisite I moved from the files/<sitename> to sites/<sitename>/files to get rid of painfull all at once D5 updates.)
Thank you for this rebuttal of Larry's suggestion. Larry, with your layout of files/<sitekey> how do you upgrade one site at a time? -- Earnie -- http://r-feed.com/ -- http://for-my-kids.com/ -- http://www.4offer.biz/ -- http://give-me-an-offer.com/
Earnie Boyd wrote:
Quoting Clemens Tolboom <clemens@build2be.nl>:
On Thu, 2009-07-16 at 10:18 -0500, larry@garfieldtech.com wrote:
sites.php was actually added specifically for this sort of issue, because the sites/ directory structure was too brittle.
Of course, for the files directories in particular I have long since dropped using sites/<sitename>/files in favor of files/<sitekey>, which sidesteps the issue entirely. That doesn't need to change no matter what server the site moves to.
What is your sitekey structure in case of a multi-site environement and what when doing a staging scenario ie move production to a demo of regression environment?
(With D5 multisite I moved from the files/<sitename> to sites/<sitename>/files to get rid of painfull all at once D5 updates.)
Thank you for this rebuttal of Larry's suggestion. Larry, with your layout of files/<sitekey> how do you upgrade one site at a time?
How do you upgrade one site at a time on any other multi-site? Yep, same way. As I said, Drupal doesn't care where the files directory is, as long as it doesn't change part way through the site's life time. --Larry Garfield
Quoting "larry@garfieldtech.com" <larry@garfieldtech.com>:
Earnie Boyd wrote:
Quoting Clemens Tolboom <clemens@build2be.nl>:
On Thu, 2009-07-16 at 10:18 -0500, larry@garfieldtech.com wrote:
sites.php was actually added specifically for this sort of issue, because the sites/ directory structure was too brittle.
Of course, for the files directories in particular I have long since dropped using sites/<sitename>/files in favor of files/<sitekey>, which sidesteps the issue entirely. That doesn't need to change no matter what server the site moves to.
What is your sitekey structure in case of a multi-site environement and what when doing a staging scenario ie move production to a demo of regression environment?
(With D5 multisite I moved from the files/<sitename> to sites/<sitename>/files to get rid of painfull all at once D5 updates.)
Thank you for this rebuttal of Larry's suggestion. Larry, with your layout of files/<sitekey> how do you upgrade one site at a time?
How do you upgrade one site at a time on any other multi-site? Yep, same way.
As I said, Drupal doesn't care where the files directory is, as long as it doesn't change part way through the site's life time.
--Larry Garfield
Cool, so it doesn't matter where you put the files directory as long as you have the files separated by a site key. Makes sense to me. -- Earnie
participants (9)
-
Ashraf Amayreh -
Bevan Rudge -
Clemens Tolboom -
Earnie Boyd -
Kathleen Murtagh -
Khalid Baheyeldin -
Larry Garfield -
larry＠garfieldtech.com
-
nan_wich＠bellsouth.net