Files on previews
Hello all files people. I am getting somewhere with my files stuff, but i'd like some gurus and/or original file.inc developers to comment on an issue: We have a lot of code in upload.module and some in file.inc to cater previews. In case of a preview, we try to serve the files from the temp folder, or we try to generate temporary URLS etc (since we have no nid yet!) This is a bit crufty IMHO. I want to change this, in my code to behave as follows: * A node (or comment or whatever) that has attachements that are valid are inserted into the files system all the way. we only leave the obid (the column in tghe files dir for object ids like nid) empty * invalid files are deleted immediately * Once the object is inserted and saved, we have an obid (nid, mostly) and we update the table. That way, on loads of previews, we should be able to just use the urls, apis and functions we use normally. So: Why did we choose for the current method of writing all the exceptions for previews? Did I miss something important that renders my idea impossible? Should I take care of something else? Ber --
At 1:12 AM +0100 11/11/05, Ber Kessels wrote:
Why did we choose for the current method of writing all the exceptions for previews? Did I miss something important that renders my idea impossible? Should I take care of something else?
I'm not a files guru, but I'll offer a suggestion. What happens if the user: clicks "create new page" attaches a file clicks preview goes and has a cup of coffee and their Windoze box crashes. You're left with a file dangling in the file system which should really be deleted, perhaps by a weekly cron. But because you have now fully inserted this file into the file system there's no easy way of telling if it's really required or not. ...R.
On 11/10/05 7:27 PM, Richard Archer wrote:
At 1:12 AM +0100 11/11/05, Ber Kessels wrote:
Why did we choose for the current method of writing all the exceptions for previews? Did I miss something important that renders my idea impossible? Should I take care of something else?
I'm not a files guru, but I'll offer a suggestion.
What happens if the user:
clicks "create new page" attaches a file clicks preview goes and has a cup of coffee and their Windoze box crashes.
You're left with a file dangling in the file system which should really be deleted, perhaps by a weekly cron.
But because you have now fully inserted this file into the file system there's no easy way of telling if it's really required or not.
Indeed, this is the whole reason that image.module has it's own temp directory - which is really just a scratch area where we can generate thumbnails, etc ... have them in the files directory so that they can be accessed by file_create_url() ... but they're also segmented exactly so that image_cron() can clean them up later in exactly the scenario you bring up (something happens between previews and submit ... leaving dangling files). Image makes no database entries until submit (as per usual). I won't go so far as to say maybe upload.module should do something similar... ;) -- James Walker :: http://walkah.net/ :: xmpp:walkah@walkah.net
On Thu, 10 Nov 2005 22:18:44 -0500 James Walker <walkah@walkah.net> wrote:
I won't go so far as to say maybe upload.module should do something similar... ;)
I like this. It sounds like a much better solution then what upload has now. Now, in my world, I dont have that much trouble with dangling images due to peoples ugly operating systems crashing (or ppl closing the browsers). I /do/ have a lot of trouble with GD and or imagemagic crashing (mostly because morons cannot understand the difference between a 20Meg image and a 500k one!). One of my reasons for starting this general upload stuff, is to be able to handle these kind of problems in a central place. My plan in my new system is to uplaod the image, and even insert it in the table. if its there, if it has a uid/nid/wid (whateverid) it is finished, finito. even if a browser crashes: its there. files without an id are indeed orphans (like walkahs images in /tmp) as are files that were processed, but not yet inserted in that table. A cron can then delete all non-wid files from the table AND the FS. And a cron can delete all files from the FS that are not in the databasetable. Ber
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 12 Nov 2005, at 4:24 PM, Ber Kessels wrote:
My plan in my new system is to uplaod the image, and even insert it in the table. if its there, if it has a uid/nid/wid (whateverid) it is finished, finito. even if a browser crashes: its there.
Create a proper 'file' form element that handles all the background stuff for you. And then a 'files' element for multiple files (ala upload) You could add an extra couple of variable to their defaults .. ie : function system_elements() { $info['file'] = array('#domain' => 'node', '#dest' => variable_get ('files_dir', ''), '#key' => $node->nid) return $info; } This would make it absolutely braindead easy to upload a file. you just add a file element in it, and you'd use either one of the common domains, or create your own. (ie: image, user, node) - -- Adrian Rossouw Drupal developer and Bryght Guy http://drupal.org | http://bryght.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (Darwin) iD8DBQFDdi+bgegMqdGlkasRAvIiAKCVeWkyVw1LppTbsoWSkODM4vkiiwCghmfM EWjGxGiIw0ulJKnOmbJQx/0= =yEQU -----END PGP SIGNATURE-----
Hmm,
Create a proper 'file' form element that handles all the background stuff for you.
And then a 'files' element for multiple files (ala upload)
You could add an extra couple of variable to their defaults .. ie :
function system_elements() { $info['file'] = array('#domain' => 'node', '#dest' => variable_get ('files_dir', ''), '#key' => $node->nid) return $info; }
This would make it absolutely braindead easy to upload a file.
you just add a file element in it, and you'd use either one of the common domains, or create your own. (ie: image, user, node)
I absolutely do not understand what you are talking about. It probably is because i don't understand the formAPI completely. Yet. But I think you misunderstood my code/plans for the file system :) Till now, I did not use the form api niftyness in my file system, so I am very happy if someone could give a hand there, and reduce some of my code/hooks by using that niftyness! Ber
On 11/12/05, Ber Kessels <ber@webschuur.com> wrote:
I absolutely do not understand what you are talking about. It probably is because i don't understand the formAPI completely. Yet. But I think you misunderstood my code/plans for the file system :) Till now, I did not use the form api niftyness in my file system, so I am very happy if someone could give a hand there, and reduce some of my code/hooks by using that niftyness!
Ber
I'm not any kind of Forms ninja, but I think I've got a reasonable understanding of it. Please let me know what I can do to help you with it. andrew
On Sat, 12 Nov 2005 14:36:58 -0800 andrew morton <drewish@katherinehouse.com> wrote:
I'm not any kind of Forms ninja, but I think I've got a reasonable understanding of it. Please let me know what I can do to help you with it.
In a nutshell: I introduced a fileapi system, nearly a 1-to-1 clone of nodapi, only then for any file object. It reacts on "save", "validate", etc. And it also reacts on "form". That hook_fileapi($file, 'form', $form) should return a form array, as well as that it passes along the existing form array. In that way modules can alter the form for a file. Anywhere. For example to turn a certain file form into a list of file forms (to allow uploading more then one file at once). Or to change a title or description. Or to remove something from the form. I am confident that all this no longer needs a hook, since the formapi has its own "override" system. I think that one form of type "upload", will suffice. That type should: * have automatically the javascript/ajax stuff with it. * be called anywhere a module wants to call it. * invoke the correct fileapi actions. Regards, Bèr -- Bèr Kessels Drupal services bler.webschuur.com www.webschuur.com ber@jabber.webschuur.com
Ber Kessels wrote:
Hello all files people.
I am getting somewhere with my files stuff, but i'd like some gurus and/or original file.inc developers to comment on an issue:
We have a lot of code in upload.module and some in file.inc to cater previews. In case of a preview, we try to serve the files from the temp folder, or we try to generate temporary URLS etc (since we have no nid yet!) This is a bit crufty IMHO. I want to change this, in my code to behave as follows:
* A node (or comment or whatever) that has attachements that are valid are inserted into the files system all the way. we only leave the obid (the column in tghe files dir for object ids like nid) empty * invalid files are deleted immediately * Once the object is inserted and saved, we have an obid (nid, mostly) and we update the table.
That way, on loads of previews, we should be able to just use the urls, apis and functions we use normally.
So: Why did we choose for the current method of writing all the exceptions for previews? Did I miss something important that renders my idea impossible? Should I take care of something else?
Ber
Ber you should take a look at the code in my filemanager and attachment modules. They already address this problem and it requires a table to keep track of files separately from their association with nodes. I wrote this code a year ago and it was passed over as to complex specifically because it actually addressed these issues which where ignored by the existing upload module. These modules have not got much love over the last year and may need some feature updates but the filemanager does handle exactly the scenarios you are attempting to address. In addition it's handling of temp/active allows editing and previews of a node to work with out affecting the existing version until the updates are submitted. Good luck, Craig
participants (6)
-
Adrian Rossouw -
andrew morton -
Ber Kessels -
Craig Courtney -
James Walker -
Richard Archer