I've become very interested in participating in the upcoming summer of code for Drupal and I have an idea about what I'd like to do with my proposal. I was just wondering what you all thought.
I was thinking of making a system that automatically mirrored files, scanned folders and otherwise made downloads available through the use of metalinks (http://www.metalinker.org/). So for instance, upload a file with Drupal and then that file is automatically sent to mirrors and a proper metalink file created (or a download page, I suppose there could be a number of preferences here).
So I was just wondering what everyone thought. Any criticisms or expansions to this idea? I'm aware that the isn't a full fledged proposal yet, but that is what I'm trying to get to eventually. Thanks for any input.
Hi Kyle, (Hope I'm not butting in here but) thanks for the interest in metalink. I think your idea would be interesting! We use Drupal on http://metalink.packages.ro/ , but the metalink generation is done by a perl script and something automatic and integrated with Drupal would be great and a lot easier for most people! A few things, I don't know if they're possible or how well they would fit in with Drupal: It would be nice to be able to scan a directory for new files and have .metalinks created, even if they were not uploaded within Drupal. A lot of the projects that use metalinks use them for large files like ISOs, so those are probably uploaded with ftp or synced from mirrors. Also, there are no tools for uploading or repairing integrity with metalinks. For instance, if you want to upload a 4 gig ISO you could create a .metalink before hand listing the full file checksums and chunk checksums. If you upload the file, and there's an error in transfer, the chunk checksums could be used to scan the file and you could just upload the corrected chunk (you'd need to create this, or work with another tool I guess). Likewise, if there's corruption or an error in the file later, it could be automatically repaired from a small chunk from the other mirrors.
I don't know metalinker.org (and I don't have internet access right now) but is metalinker.org's goal to distribute the bandwidth and to help guarantee availability (i.e. distributed file cache). It sounds like a fun project, but I'm not sure it has enough 'meat' to keep you busy for 2 full months. Plus, it might be too vendor-specific.
Hi Dries, yes that's basically the goal! Guaranteeing availability and integrity. It's done with an XML format supported by 11 download programs (mostly open source but some proprietary) and used by projects like OpenOffice.org, cURL, openSUSE, and other distributions. They mainly list mirrors, p2p source, checksums, and other file information. It's not specific to any vendor. I wouldn't know how much meat the proposal has, but I'd be willing to work with anyone interested. more info: http://en.wikipedia.org/wiki/Metalink http://www.geospatialsemanticweb.com/2007/02/25/metalink-unifies-internet-do... http://slashdot.org/article.pl?sid=07/02/25/144209 http://www.freesoftwaremagazine.com/articles/using_metalinks http://www.osresources.com/3_18_en.html -- (( Anthony Bryan )) Metalink [ http://www.metalinker.org ]