files owned by uid, patch review request.
I've got one patch I've been working on for a while. I re-rolled it a couple days ago. I'd appreciate getting it reviewed from a few people. So far in my tests it works flawlessly, I'd really like to see this in D6. The idea to to associate files in the files table to a user id instead of a node id. Upload, Filefield, Imagefield, etc will still be able to associate files to nodes. This particular patch updates Upload to use the file_revisions table to maintain the node>-<file associations, maybe it should become the upload_node table. QUICK OVERVIEW: Problems this patch solves: - any thing related to displaying files on node previews. - hanging files in drupal's temp dir. - centralizes some security features that are in upload to file.inc Problems this patch creates: - no ui to access orphaned files in files table. Features this patch adds: - global quotas and extension limits on uploads per role. - save files with out a node via file_save_data. Possibilities this patch creates: - associating existing files to nodes. - advanced file browsers - extending db handling to file_move, file_copy, file_delete. - extended file functions would be an ideal spot to implement a hook_file VERBOSE OVERVIEW: This patch moves file upload limits and validation out of upload.module and to Drupal Core. This allows administrators to set allowed extensions per role, and quotas that will be enforced for any module using core's file_save_upload function. It also moves upload_munge_filename into file.inc so other modules handling files do no have to reimplement it or depend on upload.module. Database handling has worked it's way into file_save_upload. When you call file_save_upload('upload'), It validates the file by testing quotas and allowed extensions, munges the file name to prevent the mime based exploits, saves the file to its destination, and adds a record to the files table. This patch also completely removes file_check_upload, since it is fully merged with file_save_upload now.. This is a stepping stone to including more database interaction in file.inc. In a better world you would file_copy($file_object, $destination), instead of if (file_copy($file_object->filepath, destination)) { db_query(INSERT ...) }. This doesn't preclude keeping _file_copy($filepath, $destpath) which will probably become a helper function of file_move. It does smell of an end to the mixed parameter file functions that accept both objects and paths and do quite the internal dance to support it. File functions that interact with the database would be a great place to implement life cycle hooks for files, since you have the file object as a container for metadata, which isn't always the case with the file functions in their current state. Fid is a better key than md5_file($path) anyways. This patch introduces one major issue, which is dealing with unassociated files. There is currently no UI for navigating the files table. I'm not sure if I have the time to build a core worthy browser before the freeze. .darrel. ps. ted & nate, if you guys still have time I'd like to actually get the file browser at least started the 3rd week of may.
I guess it would help if I included a link to the patch, http://drupal.org/node/115267 .darrel.
I wonder what the user-file association would mean for a site which hires/fires editors and still owns/reuses their files. Or in a site like drupal.org with patches as attached files? I understand that the orphaned user-files would pile up in a default user account, but if that is very likely to happen for a lot of files, then user ownership of files would seem accidental. Perhaps files should be able to stand on their own, with options to associate then with nodes or users? On 5/5/07, Darrel O'Pry <dopry@thing.net> wrote:
I've got one patch I've been working on for a while. I re-rolled it a couple days ago. I'd appreciate getting it reviewed from a few people. So far in my tests it works flawlessly, I'd really like to see this in D6.
The idea to to associate files in the files table to a user id instead of a node id. Upload, Filefield, Imagefield, etc will still be able to associate files to nodes. This particular patch updates Upload to use the file_revisions table to maintain the node>-<file associations, maybe it should become the upload_node table.
QUICK OVERVIEW:
Problems this patch solves: - any thing related to displaying files on node previews. - hanging files in drupal's temp dir. - centralizes some security features that are in upload to file.inc
Problems this patch creates: - no ui to access orphaned files in files table.
Features this patch adds: - global quotas and extension limits on uploads per role. - save files with out a node via file_save_data.
Possibilities this patch creates: - associating existing files to nodes. - advanced file browsers - extending db handling to file_move, file_copy, file_delete. - extended file functions would be an ideal spot to implement a hook_file
VERBOSE OVERVIEW:
This patch moves file upload limits and validation out of upload.module and to Drupal Core. This allows administrators to set allowed extensions per role, and quotas that will be enforced for any module using core's file_save_upload function.
It also moves upload_munge_filename into file.inc so other modules handling files do no have to reimplement it or depend on upload.module.
Database handling has worked it's way into file_save_upload. When you call file_save_upload('upload'), It validates the file by testing quotas and allowed extensions, munges the file name to prevent the mime based exploits, saves the file to its destination, and adds a record to the files table.
This patch also completely removes file_check_upload, since it is fully merged with file_save_upload now..
This is a stepping stone to including more database interaction in file.inc. In a better world you would file_copy($file_object, $destination), instead of if (file_copy($file_object->filepath, destination)) { db_query(INSERT ...) }. This doesn't preclude keeping _file_copy($filepath, $destpath) which will probably become a helper function of file_move. It does smell of an end to the mixed parameter file functions that accept both objects and paths and do quite the internal dance to support it.
File functions that interact with the database would be a great place to implement life cycle hooks for files, since you have the file object as a container for metadata, which isn't always the case with the file functions in their current state. Fid is a better key than md5_file($path) anyways.
This patch introduces one major issue, which is dealing with unassociated files. There is currently no UI for navigating the files table. I'm not sure if I have the time to build a core worthy browser before the freeze.
.darrel.
ps. ted & nate, if you guys still have time I'd like to actually get the file browser at least started the 3rd week of may.
On Sat, 2007-05-05 at 06:28 +0300, Cog Rusty wrote:
I wonder what the user-file association would mean for a site which hires/fires editors and still owns/reuses their files. Or in a site like drupal.org with patches as attached files?
files won't be deleted if users are deleted. for ownership we can assume uid 1 safely if user.uid doesn't exist.
I understand that the orphaned user-files would pile up in a default user account, but if that is very likely to happen for a lot of files, then user ownership of files would seem accidental.
huh? pile up in a user account? accidental? user_access always returns true when uid=1... that isn't accidental. This patch cares very little for where files pile up when a user is deleted. Right now we don't really care what happens to files if their node gets deleted and they don't, but we also don't have any safe way of sharing files between nodes.
Perhaps files should be able to stand on their own, with options to associate then with nodes or users?
Then who can update, delete, and alter files? How do you control access to files? Drupal's permission system is user based, hence user_access. I've thought about this issue a lot. I've explored it and alternatives quite a bit over the last year. It's the easiest way I've found to implement files in Drupal without depending on nodes. It should be a fairly familiar model, most filesystem systems implement user ownership of files and grant the system super user an all access pass. The patch for changing default file relationships from nodes to users solves many issues without introducing excessive permissions related issues. It's a relatively simple change that doesn't introduce any new tables, and only make a minor change to our existing data model. The real point of this patch is remove all the hackage associated with file previews... Have you ever looked at the code that handles displaying file previews? Basically today, we have to save the file to the session, create a fake menu entry for the file at its final destination, and serve the file preview through drupal. It's kind of a pita, resource hog, and in makes file previews dependent on clean urls. Search through the issue queue for file preview related bug reports. Look at the crazy code that does file path replacement for node bodies during previews. Look at inline and how it manages to work with previews... It was the most difficult part for me to figure out working with drupal's files. I want to eliminate that pain from every other developer's life. As an aside this patch also centralizes validation handling from upload.module to file.inc, so we do gains centralized quota's and file extension validation as boon. Resolving the hanging temp file issues for uploads for this patch also resolves the long standing issues with garbage collection in drupal 4.7 and 5. The patch has been updated with feedback from unconed(steven wittens) and quicksketch(nate huang). I was able to resolve several issues with the patch. It's miles from where it was yesterday, and awaiting a fresh thrashing and some kind pgsql maven to write a pgsql update path. http://drupal.org/node/115267 .darrel.
I am listening. I do know that you have put a lot of work into this in the past year, I just wanted to make sure that some currently available common use cases have been taken into account. On 5/5/07, Darrel O'Pry <dopry@thing.net> wrote:
On Sat, 2007-05-05 at 06:28 +0300, Cog Rusty wrote:
I wonder what the user-file association would mean for a site which hires/fires editors and still owns/reuses their files. Or in a site like drupal.org with patches as attached files?
files won't be deleted if users are deleted. for ownership we can assume uid 1 safely if user.uid doesn't exist.
I understand that the orphaned user-files would pile up in a default user account, but if that is very likely to happen for a lot of files, then user ownership of files would seem accidental.
huh? pile up in a user account? accidental? user_access always returns true when uid=1... that isn't accidental.
I think I understand what you mean. By "accidental" I meant that if (under some common scenarios) too many files are assigned to account #1 and the user-owned files are few, then user-ownership of files would seem unimportant for basing the system on it. But I can see that account #1 can support permissions. I have one more more question (below) about whether account #1 is a good way to do that.
This patch cares very little for where files pile up when a user is deleted. Right now we don't really care what happens to files if their node gets deleted and they don't, but we also don't have any safe way of sharing files between nodes.
Perhaps files should be able to stand on their own, with options to associate then with nodes or users?
Then who can update, delete, and alter files? How do you control access to files? Drupal's permission system is user based, hence user_access.
Another questions is (a) who can update, delete, and alter orphaned files owned by user #1 (the "super-admin") and (b) whether there can be any granularity for securing files "really" owned by user #1.
I've thought about this issue a lot. I've explored it and alternatives quite a bit over the last year. It's the easiest way I've found to implement files in Drupal without depending on nodes. It should be a fairly familiar model, most filesystem systems implement user ownership of files and grant the system super user an all access pass.
In a Unix-like filesystem, the concept of "user group" permissions seems essential for access to files owned by other users or by user #1. Does the plan include anything similar? Or isn't this really essential?
The patch for changing default file relationships from nodes to users solves many issues without introducing excessive permissions related issues.
It's a relatively simple change that doesn't introduce any new tables, and only make a minor change to our existing data model.
The real point of this patch is remove all the hackage associated with file previews... Have you ever looked at the code that handles displaying file previews? Basically today, we have to save the file to the session, create a fake menu entry for the file at its final destination, and serve the file preview through drupal. It's kind of a pita, resource hog, and in makes file previews dependent on clean urls.
Search through the issue queue for file preview related bug reports.
Look at the crazy code that does file path replacement for node bodies during previews. Look at inline and how it manages to work with previews...
It was the most difficult part for me to figure out working with drupal's files. I want to eliminate that pain from every other developer's life.
As an aside this patch also centralizes validation handling from upload.module to file.inc, so we do gains centralized quota's and file extension validation as boon.
Resolving the hanging temp file issues for uploads for this patch also resolves the long standing issues with garbage collection in drupal 4.7 and 5.
The patch has been updated with feedback from unconed(steven wittens) and quicksketch(nate huang). I was able to resolve several issues with the patch. It's miles from where it was yesterday, and awaiting a fresh thrashing and some kind pgsql maven to write a pgsql update path.
.darrel.
+1 this in core. I maintain IMCE and it uses folder based quotation for users. It scans physical directories to get the file list. With this patch in core one can easily get the files of a user from DB and calculate quota. It would probably be faster than IMCE's current method. (or wouldn't it?) For a well organized file system directory, what completes this patch and what IMCE currently implements partially and will probably improve, is user(role)-folder associations. Folders are assigned to user roles, each having its own permission for browsing, uploading, and other operations like deleting, renaming etc. Folder names may contain tokens like %uid and %uname to allow more specialized/personalized directory structure. And a wildcard * can be used for specifying subfolders. an example for an admin role (full access to file directory path): folder1: * permissions: +upload, +browse(+all files, -own files), +delete(+all, -own) an example for an editor role (a personal folder to upload + browsing access to other editors' folders): folder1: editors/%uname permissions: +upload, +browse(+all, -own), +delete(+all, -own) folder2: editors/* permissions: -upload, +browse(+all, -own), -delete an example for auth. user role(upload to a shared folder, browse and delete own files) folder1: userfiles permissions: +upload, +browse(-all, +own), +delete(-all, +own) Note: For limitations/permissions on quantitative values(e.g. user quota), role precedence setting becomes a necessity when a user has multiple roles. Will roles in D6 have weight attribute? Was this discussed anywhere? Then comes the file browser. It will probably have a UI like ftp clients. Users will only be able to see the folders to which they have browsing access. To list files in a folder faster, it would be better to store filepath as separated into directory name(relative to file directory path) and the filename(as it is now). I think queries like "SELECT ... WHERE dirname='foo'" will be faster than queries like "SELECT ... WHERE filepath LIKE 'foo/'". Besides that, it may end up where we maintain a folders table. This will be necessary to get subfolders of a folder without a physical folder scan or without a query that includes a regular expression. Think about getting subfolders of the file sytem directory by just using the current files table. Actually this resembles the taxonomy system where taxonomy terms are folders and nodes are files. We should implement a similar but simpler system in order to make the file system browsable by just querying the DB. To conclude, i liked this patch and thank Darrel for all efforts. (I would really want to test the patch througly and see the result of porting IMCE to the new system before code freeze. I don't know if i will have time but thanks to Nathan :) i'm not alone anymore in this project.)
On Mon, 2007-05-07 at 02:17 +0300, ufuk bayburt wrote:
+1 this in core. I maintain IMCE and it uses folder based quotation for users. It scans physical directories to get the file list. With this patch in core one can easily get the files of a user from DB and calculate quota. It would probably be faster than IMCE's current method. (or wouldn't it?)
For a well organized file system directory, what completes this patch and what IMCE currently implements partially and will probably improve, is user(role)-folder associations.
Folders are assigned to user roles, each having its own permission for browsing, uploading, and other operations like deleting, renaming etc. Folder names may contain tokens like %uid and %uname to allow more specialized/personalized directory structure. And a wildcard * can be used for specifying subfolders.
Aye I'd really like to see token support in core, so directory hashing methods for files can be user defined. It is a common problem for large Drupal sites to have too many files in a single directory. Some filesystems just do not handle 10K's of files in directories very well.
an example for an admin role (full access to file directory path): folder1: * permissions: +upload, +browse(+all files, -own files), +delete(+all, -own)
an example for an editor role (a personal folder to upload + browsing access to other editors' folders): folder1: editors/%uname permissions: +upload, +browse(+all, -own), +delete(+all, -own) folder2: editors/* permissions: -upload, +browse(+all, -own), -delete
an example for auth. user role(upload to a shared folder, browse and delete own files) folder1: userfiles permissions: +upload, +browse(-all, +own), +delete(-all, +own)
Note: For limitations/permissions on quantitative values(e.g. user quota), role precedence setting becomes a necessity when a user has multiple roles. Will roles in D6 have weight attribute? Was this discussed anywhere?
This is a really cool concept, I'd like to find a way to have more user control over the permissions, but not something I want to bring home in this patch. I may have to play with IMCE to see the perms in action. I have also been thinking about using the status field and having a FILE_STATUS_HIDDEN that tells other modules to ignore files unless they happen to be responsible for the files.
Then comes the file browser. It will probably have a UI like ftp clients. Users will only be able to see the folders to which they have browsing access.
To list files in a folder faster, it would be better to store filepath as separated into directory name(relative to file directory path) and the filename(as it is now). I think queries like "SELECT ... WHERE dirname='foo'" will be faster than queries like "SELECT ... WHERE filepath LIKE 'foo/'".
Besides that, it may end up where we maintain a folders table. This will be necessary to get subfolders of a folder without a physical folder scan or without a query that includes a regular expression. Think about getting subfolders of the file sytem directory by just using the current files table.
My opinion on this is actually to just use a standard file hash on the server file system, and hide the details of the real system path from the end user, and build hierarchies using meta data to browser files. Most modern music plays provide and interface like this, and I personally find it far more convenient for managing a large number of media files.
Actually this resembles the taxonomy system where taxonomy terms are folders and nodes are files. We should implement a similar but simpler system in order to make the file system browsable by just querying the DB.
To conclude, i liked this patch and thank Darrel for all efforts.
(I would really want to test the patch througly and see the result of porting IMCE to the new system before code freeze. I don't know if i will have time but thanks to Nathan :) i'm not alone anymore in this project.)
Sweet.. .darrel.
On 5/6/07, Darrel O'Pry <dopry@thing.net> wrote:
I have also been thinking about using the status field and having a FILE_STATUS_HIDDEN that tells other modules to ignore files unless they happen to be responsible for the files.
Before I'd seen what you were up to, my goal was to eventually allow contrib modules to use the upload module's file uploading but keep upload from displaying them as attachments. The idea is to add a field to the files table so that modules can assert ownership over a file. Please take a look at this issue: http://drupal.org/node/33482 At this point I'm going to try to bring my ideas into line with your framework. andrew
On Sat, 2007-05-05 at 17:25 +0300, Cog Rusty wrote:
I am listening. I do know that you have put a lot of work into this in the past year, I just wanted to make sure that some currently available common use cases have been taken into account.
On 5/5/07, Darrel O'Pry <dopry@thing.net> wrote:
On Sat, 2007-05-05 at 06:28 +0300, Cog Rusty wrote:
I wonder what the user-file association would mean for a site which hires/fires editors and still owns/reuses their files. Or in a site like drupal.org with patches as attached files?
files won't be deleted if users are deleted. for ownership we can assume uid 1 safely if user.uid doesn't exist.
I understand that the orphaned user-files would pile up in a default user account, but if that is very likely to happen for a lot of files, then user ownership of files would seem accidental.
huh? pile up in a user account? accidental? user_access always returns true when uid=1... that isn't accidental.
I think I understand what you mean. By "accidental" I meant that if (under some common scenarios) too many files are assigned to account #1 and the user-owned files are few, then user-ownership of files would seem unimportant for basing the system on it. But I can see that account #1 can support permissions. I have one more more question (below) about whether account #1 is a good way to do that.
What common scenarios? What do you mean too many files assigned to uid 1?
This patch cares very little for where files pile up when a user is deleted. Right now we don't really care what happens to files if their node gets deleted and they don't, but we also don't have any safe way of sharing files between nodes.
Perhaps files should be able to stand on their own, with options to associate then with nodes or users?
Then who can update, delete, and alter files? How do you control access to files? Drupal's permission system is user based, hence user_access.
Another questions is (a) who can update, delete, and alter orphaned files owned by user #1 (the "super-admin") and (b) whether there can be any granularity for securing files "really" owned by user #1.
The answer to your question is undefined. Currently, you cannot update, delete, or alter files period. Permissions for relating files to nodes remain unaffected. Permissions for uploading files are unaffected. Yes there can be granular access control to files, but that is a feature not included in this patch.
I've thought about this issue a lot. I've explored it and alternatives quite a bit over the last year. It's the easiest way I've found to implement files in Drupal without depending on nodes. It should be a fairly familiar model, most filesystem systems implement user ownership of files and grant the system super user an all access pass.
In a Unix-like filesystem, the concept of "user group" permissions seems essential for access to files owned by other users or by user #1. Does the plan include anything similar? Or isn't this really essential?
Groups are fun and important, but are out of scope for this change. I could imaging a basic administer files permission that would create a super user group. Much like administer content.
On 07 May 2007, at 01:38, Darrel O'Pry wrote:
Yes there can be granular access control to files, but that is a feature not included in this patch.
Groups are fun and important, but are out of scope for this change. I could imaging a basic administer files permission that would create a super user group. Much like administer content.
I don't think Cog is requesting that this would be part of this patch. I think he just wants to understand how this patch affects feature requests like this. As in; does this patch makes it easier or more difficult to implement some of the common feature requests related to file handling. What if I want the files to get deleted when the node is deleted? Would they still be deleted with the proposed patch? -- Dries Buytaert :: http://www.buytaert.net/
On Mon, 2007-05-07 at 13:00 +0200, Dries Buytaert wrote:
On 07 May 2007, at 01:38, Darrel O'Pry wrote:
Yes there can be granular access control to files, but that is a feature not included in this patch.
Groups are fun and important, but are out of scope for this change. I could imaging a basic administer files permission that would create a super user group. Much like administer content.
I don't think Cog is requesting that this would be part of this patch. I think he just wants to understand how this patch affects feature requests like this. As in; does this patch makes it easier or more difficult to implement some of the common feature requests related to file handling.
Yes I think it could be implemented, and I think it would make it easier. While nodes did provide a convenient sort of grouping, we don't have a group access control model in Drupal core. We use a role based permission system currently. With the exception of organic groups. I'm no expert on node access and admittedly barely grok it some days, but further abstracting the existing node access system might help with the model. I've conjectured that adding and `object type` column, where an object could be node, comment, file, to the node access table would give us pluggable permission for everything. I don't really knwo if its feasible or how to head in that direction with node access. The nuances elude me.
What if I want the files to get deleted when the node is deleted? Would they still be deleted with the proposed patch?
No. The file remains, but is disassociated from the node. There is no way to guarantee the file is not in use elsewhere on your site, or linked from an external site, so the file remains. It can be modified to do so, but being able to do it sanely will require the addition on drewish's modules column, or a hook delete for reference counting to make sure files are not in use elsewhere.
On 07 May 2007, at 21:13, Darrel O'Pry wrote:
I don't think Cog is requesting that this would be part of this patch. I think he just wants to understand how this patch affects feature requests like this. As in; does this patch makes it easier or more difficult to implement some of the common feature requests related to file handling.
Yes I think it could be implemented, and I think it would make it easier. While nodes did provide a convenient sort of grouping, we don't have a group access control model in Drupal core. We use a role based permission system currently. With the exception of organic groups.
Thanks for the clarification.
What if I want the files to get deleted when the node is deleted? Would they still be deleted with the proposed patch?
No. The file remains, but is disassociated from the node. There is no way to guarantee the file is not in use elsewhere on your site, or linked from an external site, so the file remains.
It can be modified to do so, but being able to do it sanely will require the addition on drewish's modules column, or a hook delete for reference counting to make sure files are not in use elsewhere.
While I like the direction of your patch (I'm in favor), I think this might warrant a bit more thought and discussion. The current model is that when you delete a node, the files associated with that node are also deleted -- both from the database and from disk. With this proposed patch, the files would remain on disk, and you'd have no way to delete them -- nor from the database, nor from disk. (Unless maybe you use a contrib module). How do you all feel about this? Does this drawback (if considered a drawback) outweight the advantages of this patch? -- Dries Buytaert :: http://www.buytaert.net/
Dries Buytaert wrote:
The current model is that when you delete a node, the files associated with that node are also deleted -- both from the database and from disk. With this proposed patch, the files would remain on disk, and you'd have no way to delete them -- nor from the database, nor from disk. (Unless maybe you use a contrib module).
How do you all feel about this? Does this drawback (if considered a drawback) outweight the advantages of this patch?
Not killing the file or the db record of the file on node delete is a huge plus in my books. It means that it could be possible to re-attached to a different (or multiple) nodes without the user having to upload again. Upload once - use often. There would be no UI (yet) - but contrib modules could fill the void. andre
Op 8-mei-2007, om 8:27 heeft Dries Buytaert het volgende geschreven:
On 07 May 2007, at 21:13, Darrel O'Pry wrote:
What if I want the files to get deleted when the node is deleted? Would they still be deleted with the proposed patch?
No. The file remains, but is disassociated from the node. There is no way to guarantee the file is not in use elsewhere on your site, or linked from an external site, so the file remains.
It can be modified to do so, but being able to do it sanely will require the addition on drewish's modules column, or a hook delete for reference counting to make sure files are not in use elsewhere.
While I like the direction of your patch (I'm in favor), I think this might warrant a bit more thought and discussion.
The current model is that when you delete a node, the files associated with that node are also deleted -- both from the database and from disk. With this proposed patch, the files would remain on disk, and you'd have no way to delete them -- nor from the database, nor from disk. (Unless maybe you use a contrib module).
How do you all feel about this? Does this drawback (if considered a drawback) outweight the advantages of this patch?
-- Dries Buytaert :: http://www.buytaert.net/
IMO this is the behaviour people are looking for. If you did upload a file/image/whatever, it should be usable in every post/item you write, selectable with some sort of easy-to-use filebrowser.. I'm not sure how (yet?), but I think it would be possible and _very_ usable to determine which files are attached to which posts, and the other way around. If we find a mechanism to accomplish something like this, chances are smaller that files are being removed/deleted which are still in use, or that orphan files are eating your diskpace. I'll definatly check/test/review doprys patch out later today.. For anyone else who would like to see file handling improved, check the following patch: http://drupal.org/node/115267 - "Simplify File Uploads, Centralize File Validation and Quotas, Fix File Previews...." Steef
----- Original Message ----- From: "Stefan Nagtegaal" <development@robuustdesign.nl> To: <development@drupal.org> Sent: Tuesday, May 08, 2007 2:50 AM Subject: Re: [development] files owned by uid, patch review request.
IMO this is the behaviour people are looking for. If you did upload a file/image/whatever, it should be usable in every post/item you write, selectable with some sort of easy-to-use filebrowser..
I'm not sure how (yet?), but I think it would be possible and _very_ usable to determine which files are attached to which posts, and the other way around. If we find a mechanism to accomplish something like this, chances are smaller that files are being removed/deleted which are still in use, or that orphan files are eating your diskpace.
I don't know about the intricacies of the coding, but I have some thoughts from a user perspective. 1) Having them be both attached to the user and the node is good. On one forum I'm on, you have a list of your files in your account area and you can delete them there if you want to free up some space in your quota. The downside of this is that posts where the attached files are important (such as screenshots) lose their usefulness. Still, it's important to let people decide what files go when they use up their quota and need room. 2) If the node is deleted, it could show up in the user's file list as unattached and be there ready to attach easily to another node. 3) If the user is deleted, we don't necessarily want to delete his/her files. I have to be honest here and say I don't know what happens to the posts of a deleted user. Are they deleted, too? If so then, yes, the files need to go. But if the posts stay behind, it would be good to keep the files with them. 4) I like the idea of "If you did upload a file/image/whatever, it should be usable in every post/item you write, selectable with some sort of easy-to-use filebrowser..". It would also be nice to retrieve these as a user gallery on their profile. But that's getting off topic, I think. Michelle
On May 8, 2007, at 9:18 AM, Michelle Cox wrote:
2) If the node is deleted, it could show up in the user's file list as unattached and be there ready to attach easily to another node.
3) If the user is deleted, we don't necessarily want to delete his/ her files. I have to be honest here and say I don't know what happens to the posts of a deleted user. Are they deleted, too? If so then, yes, the files need to go. But if the posts stay behind, it would be good to keep the files with them.
When users are deleted, the author is set to anonymous. So putting 2) and 3) together, you could delete the unattached files for the anonymous user and keep the attached files.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Dries Buytaert schrieb:
The current model is that when you delete a node, the files associated with that node are also deleted -- both from the database and from disk. With this proposed patch, the files would remain on disk, and you'd have no way to delete them -- nor from the database, nor from disk. (Unless maybe you use a contrib module).
How do you all feel about this? Does this drawback (if considered a drawback) outweight the advantages of this patch?
I consider it a drawback. If I delete a node, I want all of its info gone including attached files. If I wouldn't want to delete that I would unpublish the node, not delete it. I have not looked enough at the patch to comment on its merits. Cheers, Gerhard -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) iD8DBQFGQEevfg6TFvELooQRAms7AJ4pVuYUHxt71I6ILF1lapZPfBnGpwCgkY2H HaF5FzqXpd7wqdbLrq3tGDw= =VAFi -----END PGP SIGNATURE-----
On 5/8/07, Dries Buytaert <dries.buytaert@gmail.com> wrote:
The current model is that when you delete a node, the files associated with that node are also deleted -- both from the database and from disk. With this proposed patch, the files would remain on disk, and you'd have no way to delete them -- nor from the database, nor from disk. (Unless maybe you use a contrib module).
How do you all feel about this? Does this drawback (if considered a drawback) outweight the advantages of this patch?
-- Dries Buytaert :: http://www.buytaert.net/
I think there is a number of behaviors which could be considered legitimate and more or less common and not be hindered. Of course the question of choosing a default behavior in core remains. - Delete a node - delete attached files, no garbage left behind. - Delete a node - preserve attached files for reuse - supply a role permission and an UI for managing/reassigning/deleting them. - Delete a user - delete owned files (for example, personal galleries). - Delete a user - preserve owned files - supply a role permission and an UI for managing/reassigning/deleting them. These options were the reason that I thought that -- at least at the user level -- a file should feel like an independent entity. In the last option, ownership of orphaned user files by uid #1 should not mean that only user #1 has to manage them. It is also interesting that while we currently have the first option in core, a very popular approach is the one used by the image module where the file is slapped on another node, an "image" node, which you can leave undeleted and possibly accessible through a gallery, and that works somehow like the second option.
Quoting Darrel O'Pry <dopry@thing.net>:
What if I want the files to get deleted when the node is deleted? Would they still be deleted with the proposed patch?
No. The file remains, but is disassociated from the node. There is no way to guarantee the file is not in use elsewhere on your site, or linked from an external site, so the file remains.
It can be modified to do so, but being able to do it sanely will require the addition on drewish's modules column, or a hook delete for reference counting to make sure files are not in use elsewhere.
I haven't looked at the patch so forgive me if this is addressed. Would it be possible to make this a feature within the system settings? Would it be possible to have a count of nodes using the file and if the node being deleted is the only one prompt for the user for file deletion? Earnie
Darrel O'Pry wrote: When working on drupal.org/project/webfm the first thing we decided was that a file has nothing to do with a node until its attached. So the webfm file table associated files with the uploader (user). Associations with nodes are handled by a webfm_attach table that has a fid nid relationship. If core were to move in that direction webfm could be re-jigged to use the core tables and voila - instant file browser for some people (thought not one that could be used in core because it absolutely depends on javascript - no graceful degradation). We've already worked out file renames, file moves, file deletes, dir renames, moves and deletes - (with drag and drop and ajax no less) keeping the db up to date with each user action. I've been waiting to see the direction the wind was blowing for core file handling code - webfm (or some new variant of it - [lots of ideas being developed at the moment - and much needed refactoring]) intends to use as many core functions as possible - so that it is a less radically different file handling tool - and that our efforts may somehow combine to a much improved set of core file handling tools. andre
participants (11)
-
Andre Molnar -
andrew morton -
Cog Rusty -
Darrel O'Pry -
Darren Oh -
Dries Buytaert -
Earnie Boyd -
Gerhard Killesreiter -
Michelle Cox -
Stefan Nagtegaal -
ufuk bayburt