hook_fileapi (file products and download permission)
I'm starting work on an ecommerce module to sell MP3 files. To accomplish this, I'd like to introduce some hooks which the audio.module would invoke, and my ecommerce module would implement. I think the hooks could be applied in other contexts, maybe eventually becoming part of Drupal core. That's why I'm sending this to a large audience. I hope all interested parties will let me know if the direction sounds right. I'd like to start solving my problem immediately, making changes to audio.module and an ecommerce module. But I'm open to other approaches if consensus is that the following ideas need work. SPECIFIC PROBLEM ---------------- I'm creating an ecommerce product to sell access to files, specifically MP3 audio files. While there exists a generic file product in the ecommerce module, the audio module is superior when it comes to MP3s. So, I want to use audio module nodes as my products. The difficulty lies in download permissions. I want only users who have purchased an MP3 to be able to download it. But audio.module controls its own downloads. So how to make audio.module obey my product permissions? GENERAL PROBLEM --------------- In Drupal today there are a handful of modules which deal with the uploading and downloading of files. Examples include upload.module, audio.module and the ecommerce file product. Each of these modules contains it's own functions to handle saving and downloading of files. Each of these modules has overlapping functionality because there is no file management piece of Drupal that meets the needs of all of them. While people are working towards better file management in core, in the 4.7 timeframe we have to accept that many modules will be doing some facet of file management. This is a problem for anyone wishing to develop modules that customize file upload/download behavior. PROPOSAL -------- To solve the problem I propose the following APIs. Any file management module (ie upload.module, audio.module) would have to invoke these hooks to be considered "well behaved". Third-party modules implement the hooks in order to have influence over the file managing modules. The APIs fall into two categories: 1) Notification of file events (uploads, downloads) 2) Grant permission to allow file events. In the APIs, files are identified by a numerical $fid and a string $realm. Together, these combine to form a unique file identifier. In the case of upload.module, each file has an $fid from the files table, and the realm could be 'upload'. In the case of audio.module, each node has one file so the $nid is also the $fid, and the realm would be 'audio'. The file management modules would have to implement logic like this when performing file operations: if (user does not have explicit permission to perform operation) { $result = module_invoke hook_file_access for the operation; if ($result contains TRUE) { perform the operation; module_invoke hook_fileapi; // notify that operation has occurred else do not perform; // (access denied) } else { // user has explicit permission to perform op perform the operation; module_invoke hook_fileapi; // notify that operation has occurred } Here are function signatures for the hooks as I imagine them: /** * hook_fileapi * * This hook is invoked when file events occur. * * @param $fid * A numeric file id * @param $realm * A string, combined with $fid ensures a unique file identification * @param $node * The node, if any, with which the file is associated. * @param $op * The type of event ('upload', 'download', 'update', 'delete') * @param $info * Associative array of file details ('filesize', 'filepath', 'filemime',...) * * @return * Nothing */ function hook_fileapi($fid, $realm, $node, $op, $info) { // third party module code goes here. } /** * hook_file_access * * This hook gives us a chance to allow file operations which would * otherwise not be permitted. * * @param $fid * A numeric file id or NULL if $op is 'upload' * @param $realm * A string, combined with $fid ensures unique file identification * @param $node * The node, if any with which the file is assiciated * @param $op * The type of event ('upload', 'download', 'update', 'delete') * @param $info * Associative array of file details ('filesize', 'filepath', 'filemime',...) * @param $account * The user who may or may not have permission to perform the operation * * @return * TRUE, if and only if the user is allowed to perform the operation */ function hook_file_access($fid, $realm, $node, $op, $info, $account) { // third-party logic here if ($is_allowed) return TRUE; } RELATED READING --------------- http://drupal.org/node/33708 (permissions and audio module) http://groups.drupal.org/file-api (file api discussion on groups.drupal.org) http://dave-cohen.com/node/1045 (my own ideas about Drupal file management) http://dave-cohen.com/node/1050 (creating ecommerce products out of nodes) I realize this is a long post. Thanks for reading.
Dave, So far as I can tell, the problem you're addressing indeed exists as you describe it: there is no opening in the file api to enable nuanced file access control. And it should be addressed--there's no sense in us having to override/replace our file api just to control access (as was done, e.g., in the ecommerce file.module). Your suggested approach for handling access looks basically sound to me--but it would be more useful to hear from more experienced fileapi-ers. Walkah, killes, dopry, Chris Johnson, etc.? Likley we'd want to keep it as close as possible to existing _api and _access hooks. So, e.g., * @param $op * The type of event ('upload', 'download', 'update', 'delete') might be better as * @param $op * The type of event ('insert', 'view', 'update', 'delete') Are you thinking of doing this as a contrib module for 4.7? Part of ecommerce, or a generic file handling module?
Did my original post make it to both developement and ecommerce mailing lists? (It was supposed to but I only received one copy in my inbox.) Nedjo, thanks for the reply. On Monday 08 May 2006 15:45, Nedjo Rogers wrote:
Likley we'd want to keep it as close as possible to existing _api and _access hooks. So, e.g.,
* @param $op * The type of event ('upload', 'download', 'update', 'delete')
might be better as
* @param $op * The type of event ('insert', 'view', 'update', 'delete')
I am fine with that. Not certain the list of ops that apply to files will always be the same as those that apply to nodes, but I can't think of a counterexample now.
Are you thinking of doing this as a contrib module for 4.7? Part of ecommerce, or a generic file handling module?
I'm thinking of an ecommerce module (maybe ecommerce/contrib, maybe modifying ecommerce/file/file.module) which implements the hooks and allows downloads only when a node has been purchased. And modifying audio.module to invoke the hooks as I described in pseudocode. If there is widespread adoption, upload.module (and probably others) should then emulate audio.module. I'm mostly concerned with the MP3 product I am trying to build. But if I build it using these hooks, I'd like to think they could serve others as well. -Dave
On Mon, 2006-05-08 at 17:19 -0700, Dave Cohen wrote:
Did my original post make it to both developement and ecommerce mailing lists? (It was supposed to but I only received one copy in my inbox.)
Nedjo, thanks for the reply.
On Monday 08 May 2006 15:45, Nedjo Rogers wrote:
Likley we'd want to keep it as close as possible to existing _api and _access hooks. So, e.g.,
* @param $op * The type of event ('upload', 'download', 'update', 'delete')
might be better as
* @param $op * The type of event ('insert', 'view', 'update', 'delete')
I am fine with that. Not certain the list of ops that apply to files will always be the same as those that apply to nodes, but I can't think of a counterexample now.
copy, move, derive come to mind quickly. Files do have a unique life cycle compared to database content items. I think my most recent thoughts on access control and the file API was to leave that in the 'files as a node' layer. My current thought is that files can exist in three worlds.. 'files as an os level object' for people who want to do file handling as it is now, and avoid the whole drupal thing, but gets you abstracted storage drivers... 'files as cms objects' drupal provides some hooks for lifecycle management and deals with keeping the db data about files straight. 'files as nodes' The top layer of the files universe, provides the node level interface to files, extensive access control, and all the crazy stuff you can leverage node module for, because you want just an end result from combining modules not to write a module. The interactions/priorities of the three layers are still blurry to me in some areas... access control is one of them.
Are you thinking of doing this as a contrib module for 4.7? Part of ecommerce, or a generic file handling module?
I'm thinking of an ecommerce module (maybe ecommerce/contrib, maybe modifying ecommerce/file/file.module) which implements the hooks and allows downloads only when a node has been purchased.
I'd love to see more contrib file handling / API modules appear to encourage some innovation and get peoples ideas into code. A good pool of disparate ideas to juice up a, how dare I use the buzz phrase, 'best of breed solution'.
And modifying audio.module to invoke the hooks as I described in pseudocode.
If there is widespread adoption, upload.module (and probably others) should then emulate audio.module. I'm mostly concerned with the MP3 product I am trying to build. But if I build it using these hooks, I'd like to think they could serve others as well.
-Dave
If you have immediate needs go for it. If you have additional time on your hands you've already found the file-api group :).
A fileapi hook was proposed nearly 2 years ago, much along the same lines. It was concluded, along the way, that the place for this is in the file.inc file or in some centralized upload manager. I agree. The filemanager[1] module has had the problem of access permissions solved for a very long time. It has a hook that lets modules impose their own access restrictions on downloads, and supports separate public/private download regions (it doesn't force you to choose between one or the other). Furthermore, the filemanager module only provides the API for uploading and downloading, not the user interface. The UI is provided by the attachment module [2]. Other modules (acidfree) have decided to use the filemanager module as their basis for dealing with uploads. I feel that it is close to what we want in core, and that building the proposed fileapi hook would be much easier if this were the case. If you're not familiar with the filemanager module, please review it. I'd be interested in hearing what criticisms exist. I'd be in favor of moving it to core. cheers, Robert [1] http://drupal.org/project/filemanager [2] http://drupal.org/node/10245 Nedjo Rogers wrote:
Dave,
So far as I can tell, the problem you're addressing indeed exists as you describe it: there is no opening in the file api to enable nuanced file access control. And it should be addressed--there's no sense in us having to override/replace our file api just to control access (as was done, e.g., in the ecommerce file.module).
Your suggested approach for handling access looks basically sound to me--but it would be more useful to hear from more experienced fileapi-ers. Walkah, killes, dopry, Chris Johnson, etc.?
Likley we'd want to keep it as close as possible to existing _api and _access hooks. So, e.g.,
* @param $op * The type of event ('upload', 'download', 'update', 'delete')
might be better as
* @param $op * The type of event ('insert', 'view', 'update', 'delete')
Are you thinking of doing this as a contrib module for 4.7? Part of ecommerce, or a generic file handling module?
I'm using the filemanager at the moment and am very happy with it. What I particularly like is the ability to have separate namspaces for files so that renaming doesn't confuse the users. I've not delved into the api but I have overviewed it. It does seem more complete then the drupal api and suits the module I'm developing down to the ground. Thanks Adam On 5/9/06, Robert Douglass <r.douglass@onlinehome.de> wrote:
A fileapi hook was proposed nearly 2 years ago, much along the same lines. It was concluded, along the way, that the place for this is in the file.inc file or in some centralized upload manager. I agree.
The filemanager[1] module has had the problem of access permissions solved for a very long time. It has a hook that lets modules impose their own access restrictions on downloads, and supports separate public/private download regions (it doesn't force you to choose between one or the other). Furthermore, the filemanager module only provides the API for uploading and downloading, not the user interface. The UI is provided by the attachment module [2]. Other modules (acidfree) have decided to use the filemanager module as their basis for dealing with uploads. I feel that it is close to what we want in core, and that building the proposed fileapi hook would be much easier if this were the case.
If you're not familiar with the filemanager module, please review it. I'd be interested in hearing what criticisms exist. I'd be in favor of moving it to core.
cheers,
Robert
[1] http://drupal.org/project/filemanager [2] http://drupal.org/node/10245
Nedjo Rogers wrote:
Dave,
So far as I can tell, the problem you're addressing indeed exists as you describe it: there is no opening in the file api to enable nuanced file access control. And it should be addressed--there's no sense in us having to override/replace our file api just to control access (as was done, e.g., in the ecommerce file.module).
Your suggested approach for handling access looks basically sound to me--but it would be more useful to hear from more experienced fileapi-ers. Walkah, killes, dopry, Chris Johnson, etc.?
Likley we'd want to keep it as close as possible to existing _api and _access hooks. So, e.g.,
* @param $op * The type of event ('upload', 'download', 'update', 'delete')
might be better as
* @param $op * The type of event ('insert', 'view', 'update', 'delete')
Are you thinking of doing this as a contrib module for 4.7? Part of ecommerce, or a generic file handling module?
Robert, Thanks for pointing out those modules. I can see the appeal of filemanager and it seems that a module like audio could have saved some trouble by using it. In my thinking, its a great building block. I'd like to see an even higher level model make its way into core. There should be hooks at both the higher and lower levels so third party modules can control what's going on. To give a concrete example: I want to make an ecommerce product out of an audio node. I want to control download permission based on whether the user has purchased the node. So my logic is a function of the node. Now filemanager has an api to let modules control the download, but its so low level that it is not node aware. So the filemanager_download hook does not include the node, and I could not use it for my logic. So I consider attachment.module to fall into the same category as upload.module and audio.module. Namely, candidates for implementing the APIs I described earlier. (higher level, node aware APIs. Filemanager would invoke it's hook before permitting the download, and attachment would invoke the hook I described.) I have my own wishlist for Drupal file management <http://dave-cohen.com/node/1045>. If I ever get around to implementing it, I'll likely use filemanager.module. -Dave On Tuesday 09 May 2006 03:56, Robert Douglass wrote:
A fileapi hook was proposed nearly 2 years ago, much along the same lines. It was concluded, along the way, that the place for this is in the file.inc file or in some centralized upload manager. I agree.
The filemanager[1] module has had the problem of access permissions solved for a very long time. It has a hook that lets modules impose their own access restrictions on downloads, and supports separate public/private download regions (it doesn't force you to choose between one or the other). Furthermore, the filemanager module only provides the API for uploading and downloading, not the user interface. The UI is provided by the attachment module [2]. Other modules (acidfree) have decided to use the filemanager module as their basis for dealing with uploads. I feel that it is close to what we want in core, and that building the proposed fileapi hook would be much easier if this were the case.
If you're not familiar with the filemanager module, please review it. I'd be interested in hearing what criticisms exist. I'd be in favor of moving it to core.
cheers,
Robert
[1] http://drupal.org/project/filemanager [2] http://drupal.org/node/10245
On 9-May-06, at 6:19 PM, Dave Cohen wrote:
Robert,
Thanks for pointing out those modules. I can see the appeal of filemanager and it seems that a module like audio could have saved some trouble by using it.
Colin did the original, and used core APIs to make it. Drewish/andrew morton has done a ton of updates and actually moved it to filemanager, and we asked/paid him to do the 4.7 version with core File API. Why? Needs to work out of the box with core. If core is difficult/ hard to work with....please, file patches! Big ones, gigantic ones. Otherwise we are left with not good code in core (archive *cough*). We need to work together on these shared pieces.
I have my own wishlist for Drupal file management <http://dave-cohen.com/node/1045>. If I ever get around to implementing it, I'll likely use filemanager.module.
So....try working with filemanager as a replacement for upload/ file.inc (IMHO -- it's either attachment or upload.module and/or re- working upload module to work more like attachment, using new filemanager API calls). Please make patches against core, since this will help everyone long term, rather than just those who build/maintain an alternate file handling structure. Cheers, -- Boris Mann Vancouver 778-896-2747 San Francisco 415-367-3595 SKYPE borismann http://www.bryght.com
On 10 May 2006, at 18:44, Boris Mann wrote:
Please make patches against core, since this will help everyone long term, rather than just those who build/maintain an alternate file handling structure.
I second that. Core's file handling, both the file API and the upload.module need love. -- Dries Buytaert :: http://www.buytaert.net/
On 10-May-06, at 12:50 PM, Dries Buytaert wrote:
On 10 May 2006, at 18:44, Boris Mann wrote:
Please make patches against core, since this will help everyone long term, rather than just those who build/maintain an alternate file handling structure.
I second that. Core's file handling, both the file API and the upload.module need love.
They do, and I have been trying to fight hard for things like audio.module stay the course with the core API and work to improve that rather than everyone doing their own thing (as I have done for image as well). File/Media handling is (as far as I'm concerned) a major hot button to be addressed in the next release, and I intend to start "rallying the troops" in the near future and putting patches where our mouths are. Those who have thoughts on the issue and haven't already (and there's already some great stuff) have a look through http:// groups.drupal.org/file-api. -- James Walker :: http://walkah.net/ :: xmpp:walkah@walkah.net
On Wed, 2006-05-10 at 13:28 -0400, James Walker wrote:
On 10-May-06, at 12:50 PM, Dries Buytaert wrote:
On 10 May 2006, at 18:44, Boris Mann wrote:
Please make patches against core, since this will help everyone long term, rather than just those who build/maintain an alternate file handling structure.
I second that. Core's file handling, both the file API and the upload.module need love.
They do, and I have been trying to fight hard for things like audio.module stay the course with the core API and work to improve that rather than everyone doing their own thing (as I have done for image as well).
File/Media handling is (as far as I'm concerned) a major hot button to be addressed in the next release, and I intend to start "rallying the troops" in the near future and putting patches where our mouths are.
Those who have thoughts on the issue and haven't already (and there's already some great stuff) have a look through http:// groups.drupal.org/file-api.
So do we want to do the iterative cycle on the existing file.inc and upload.module, or throw in something new. ------------------------------------------------------------------------ I have a good chunk of a new file handling api already written. You can see it in cvs. http://cvs.drupal.org/viewcvs/drupal/contributions/modules/filesystem/ some elements are only sketched in at the moment, but they are at least linted, but maybe what is there serves to express my idea. I wanted to write tests for all of the api functions first to exercise them as I develop the lower level abstraction layers. I still need to research how name spaces are handled in filemanager since people seem to have a high opinion of the module in general. -------------------------------------------------------------------------- While I like my ideas, It would probably be better to massage file.inc and upload.module into the shape that is needed so we create the upgrade path as we go. So here is one potential battle plan. 1) The first thing I'd like to do is relate files to uid instead of nid in the {files} table as it simplifies a lot of the preview code which is pretty hairy. I need to write a file browser in the process. So users can delete old files, or unintentional uploads. I can handle the upgrade path and the file.inc and upload.inc changes. I'll struggle with the UI for the file browser though. The initial database migration path is something along the lines of --create table node_files select nid, fid from files'; --alter table files add column uid --update table files join node on files.nid = node.nid set files.uid = node.uid; --alter table files drop column nid Then all of our add file widgets are just a common mini-file browser. plus a single upload field. 2) Then an inclusion of a hook_file($op) in all of the file_* functions that act on files. (this will give us the life cycle hooks to do advanced media handling, create derivatives, modify files, alter output ex) render a flash player or quicktime player, but that can also be done through theme functions currenlty. You can already do some fancy stuff through nodeapi with module weights by making your derivative generating module heavier than upload.module, but it's clunky.) ------------------------------------------------------------------------- Those two account for my biggest desires for 4.8. The rest of it is just pudding. although splitting/improving private files is just off the list but sits in a big pile of file / access control issues I haven't really thought about alot. ------------------------------------------------------------------------- I'd rather not take the time to write and test these patches without some sort of a 'this direction is good' from the rest of the world...
On 10 May 2006, at 6:50 PM, Dries Buytaert wrote:
I second that. Core's file handling, both the file API and the upload.module need love.
Perhaps what we need is proper FAPI'sation of the file api. So that just putting a file field on a form, automatically saves it. Free the developer from every having to call any of the file_ functions. Also, moving the upload ajax stuff out of the upload module, and into the file element, so that all file elements get to do that. -- Adrian Rossouw Drupal developer and Bryght Guy http://drupal.org | http://bryght.com
+ 1 for this idea. Exactly how it should be done. Ditto for images. Image resizing / format changes etc can also be done using values passed to Form API. Modules should be free of this type of code. I've done all this stuff before so am happy to contribute it if the need be there. -- Sammy Spets Synerger Pty Ltd http://www.synerger.com/ On 11-May-06 05:42, Adrian Rossouw wrote:
On 10 May 2006, at 6:50 PM, Dries Buytaert wrote:
I second that. Core's file handling, both the file API and the upload.module need love.
Perhaps what we need is proper FAPI'sation of the file api. So that just putting a file field on a form, automatically saves it. Free the developer from every having to call any of the file_ functions.
Also, moving the upload ajax stuff out of the upload module, and into the file element, so that all file elements get to do that.
-- Adrian Rossouw Drupal developer and Bryght Guy http://drupal.org | http://bryght.com
This is a must. I have something along those lines here: http://cvs.drupal.org/viewcvs/drupal/contributions/sandbox/yogadex/modules/n... On Wednesday 10 May 2006 20:42, Adrian Rossouw wrote:
On 10 May 2006, at 6:50 PM, Dries Buytaert wrote:
I second that. Core's file handling, both the file API and the upload.module need love.
Perhaps what we need is proper FAPI'sation of the file api. So that just putting a file field on a form, automatically saves it. Free the developer from every having to call any of the file_ functions.
Also, moving the upload ajax stuff out of the upload module, and into the file element, so that all file elements get to do that.
-- Adrian Rossouw Drupal developer and Bryght Guy http://drupal.org | http://bryght.com
participants (10)
-
Adam Cooper -
Adrian Rossouw -
Boris Mann -
Darrel O'Pry -
Dave Cohen -
Dries Buytaert -
James Walker -
Nedjo Rogers -
Robert Douglass -
Sammy Spets