[drupal-devel] Core Folksonomy: First Code Available
I've commited into sandbox/morbus/folksonomy_shared (along with a bunch of documentation and design decisions) the first, very rough and not heavily tested, version of folksonomy_shared. There are still a number of things to do (and a few known bugs to fix) but I wanted to show you what I have. Here is the new vocabulary definition screen: http://www.disobey.com/d/2005/drupal_folkdef.jpg My change is in yellow. Now, assume that we've got two vocabularies: a "Folksonomy Tags" (per the screenshot) and a "Controlled Test", which is a normal old Drupal taxonomy. They've both been assigned to node type "page". Here is the screenshot: http://disobey.com/detergent/2005/drupal_folknodenew.jpg Now, let's assume we've entered some data. We've chosen "Cheshire Cat" from the "Controlled Test", and entered the following for tags: "happy, sad, crazy, whoa!" Here is the resulting screenshot: http://disobey.com/detergent/2005/drupal_folknodesubmit.jpg Notice how three new "Folksonomy Tags" (trust me on the proper vid) terms were created (implying that "crazy" already existed in the "Folksonomy Tags" vocabulary). Likewise, note how they are treated equally in the node display ("Cheshire Cat" comes from our "Controlled Test" vocab). Finally, the edit screen, though there's nothing to exciting here: http://disobey.com/detergent/2005/drupal_folknodeedit.jpg More as I have it. -- Morbus Iff ( you are nothing without your robot car, NOTHING! ) Culture: http://www.disobey.com/ and http://www.gamegrene.com/ Spidering Hacks: http://amazon.com/exec/obidos/ASIN/0596005776/disobeycom icq: 2927491 / aim: akaMorbus / yahoo: morbus_iff / jabber.org: morbus
From a UI perspective, this is outstanding work. Love it ... I'll have a look at the code ASAP.
I've commited into sandbox/morbus/folksonomy_shared (along with a bunch of documentation and design decisions) the first, very rough and not heavily tested, version of folksonomy_shared. There are still a number of things to do (and a few known bugs to fix) but I wanted to show you what I have.
On Tuesday 15 March 2005 16:25, Morbus Iff wrote:
I've commited into sandbox/morbus/folksonomy_shared (along with a bunch of documentation and design decisions) the first, very rough and not heavily tested, version of folksonomy_shared. There are still a number of things to do (and a few known bugs to fix) but I wanted to show you what I have.
Really nice work! I can see this being *extremely* useful in allowing users to create new categories that equate to image albums. One suggestion: Should there be a conditional statement to trivially ignore community terms that are empty after the trim() call, in order to reject empty terms created by strings like "cat, dog,, fish, snake"? (I didn't look to see if the core already took care of that; I just didn't see it in your patch.) Overall, it seems to add unique functionality and not to disrupt the behavior that already exists for conventional vocabularies, and the code size is surprisingly small. Administrators who don't like it are not annoyed, because the feature is disabled by default. +1 from me! Scott -- -----------------------+------------------------------------------------------ Scott Courtney | "I don't mind Microsoft making money. I mind them scott@4th.com | having a bad operating system." -- Linus Torvalds http://4th.com/ | ("The Rebel Code," NY Times, 21 February 1999) | PGP Public Key at http://4th.com/keys/scott.pubkey
One suggestion: Should there be a conditional statement to trivially ignore community terms that are empty after the trim() call, in order to reject empty terms created by strings like "cat, dog,, fish, snake"? (I didn't look to see if the core already took care of that; I just didn't see it in your
Excellent. Yes, that's something that'll need to be added. (I suspect that if a blank value was sent to taxonomy_save_term, that module would croak, but I'm not entirely sure how the error would escalate upwards.) Besides further integration, the next thing I'm thinking about is user permissions (and I admit to not giving it alot of thought until this paragraph, so I may "solve" it by the time I'm finished). I've got to see what happens when a non-admin user logs in and creates a node they have permission to. I /think/ the new code will function properly, but I suspect this is exploitable: * To ensure that the right community defined terms are added to the right folksonomy (thus ensuring multiple folksonomies on one site could remain independent, much like normal taxos [normal taxos use the unique tid so knowing the vid isn't important]), the vid is included in the node form: edit[taxonomy][vid][2] (note: "vid" will change to "community") This seemingly allows someone to make their own local form, change the vid to a controlled vid, and add a new term to a vocabulary that has not been designated community created. I'm not sure what the best way to handle this is. The easiest, but still error-prone way is to ensure that the $vid passed has $community enabled. This would require an additional lookup for every vocabulary attached to a particular node on taxonomy_node_save. Then, the user could only "corrupt" community-created folksonomies (where "corrupt" is loosely defined as "maliciously doing what he's already allowed to do"). But, see below. The other alternative would be a brand new taxonomy permission: * add new terms to community-created taxonomies Thus, you'd be able to create tiers of folksy users: all users can create nodes but group A can create new terms and group B can't. The downside of this is that the next "obvious" step would be to have a new permission PER community-created taxonomy: group A can create new terms in 'Folksonomy Tags', group B can create new terms in both 'Folksonomy Tags' and 'Another Example'. I can certainly see that being useful: * "gallery uploaders" can create new tags in 'Image-Only Folksonomy' * "music uploaders" can create new tags in 'MP3-Sharers Anonymous'. but I'm not entirely sure if I'm just being pie-in-the-sky. -- Morbus Iff ( i know a little of everything, a lot of nothing. ) Technical: http://www.oreillynet.com/pub/au/779 Culture: http://www.disobey.com/ and http://www.gamegrene.com/ icq: 2927491 / aim: akaMorbus / yahoo: morbus_iff / jabber.org: morbus
* add new terms to community-created taxonomies
Thus, you'd be able to create tiers of folksy users: all users can create nodes but group A can create new terms and group B can't. The downside of this is that the next "obvious" step would be to have a new permission PER community-created taxonomy: group A can create new terms in 'Folksonomy Tags', group B can create new terms in both 'Folksonomy Tags' and 'Another Example'. I can certainly see that being useful:
* "gallery uploaders" can create new tags in 'Image-Only Folksonomy' * "music uploaders" can create new tags in 'MP3-Sharers Anonymous'.
i suggest avoiding this for phase 1. drupal is rightfully suspicious of massive simultaneous change.
On 16 Mar 2005, at 02:23, Moshe Weitzman wrote:
* add new terms to community-created taxonomies
Thus, you'd be able to create tiers of folksy users: all users can create nodes but group A can create new terms and group B can't. The downside of this is that the next "obvious" step would be to have a new permission PER community-created taxonomy: group A can create new terms in 'Folksonomy Tags', group B can create new terms in both 'Folksonomy Tags' and 'Another Example'. I can certainly see that being useful:
* "gallery uploaders" can create new tags in 'Image-Only Folksonomy' * "music uploaders" can create new tags in 'MP3-Sharers Anonymous'.
i suggest avoiding this for phase 1. drupal is rightfully suspicious of massive simultaneous change.
I agree with Moshe. Let's keep step #1 very simple. -- Dries Buytaert :: http://www.buytaert.net/
I've updated the patch, but this is mostly bugfixes: - the form element name changed from 'vid' to 'community'. - corrected some array warnings in taxonomy_node_form. - corrected the loss of form data on a node "preview". - no longer attempt to add an empty tag on a comma-splice. - vids are checked to ensure they're community-enabled on save. - removed the "preview form" for vocabularies. silly. The only two things remaining on my todo are: * Still haven't touched any of the UID saving stuff. Forthcoming. * How to handle the taxonomy GUI for 1000+ terms. Pagers? The UID stuff should be relatively easy (two or three revisions to INSERT/DELETE statements should do it), and I'll make some screenshots once I have an example of the pager-enabled code ready. -- Morbus Iff ( you are nothing without your robot car, NOTHING! ) Culture: http://www.disobey.com/ and http://www.gamegrene.com/ Spidering Hacks: http://amazon.com/exec/obidos/ASIN/0596005776/disobeycom icq: 2927491 / aim: akaMorbus / yahoo: morbus_iff / jabber.org: morbus
Morbus Iff <morbus@disobey.com> Tue, 15 Mar 2005 18:35:03
* To ensure that the right community defined terms are added to the right folksonomy (thus ensuring multiple folksonomies on one site could remain independent, much like normal taxos [normal taxos use the unique tid so knowing the vid isn't important]), the vid is included in the node form:
edit[taxonomy][vid][2] (note: "vid" will change to "community")
I solved this with a field name of edit[folksonomy][2 where 2 is the vocab for this edit field and then converting the comma delimited string to an array of tids during nodeapi('update') creating new tids as necessary. I had a corresponding folksonomy_form_all() which paralleled taxonomy_form_all and created as many fields as there were folksonomy vocabs defined for this node type.
Thus, you'd be able to create tiers of folksy users: all users can create nodes but group A can create new terms and group B can't. The downside of this is that the next "obvious" step would be to have a new permission PER community-created taxonomy: group A can create new terms in 'Folksonomy Tags', group B can create new terms in both 'Folksonomy Tags' and 'Another Example'. I can certainly see that being useful:
I definitely thought that anyone with edit rights for the node would have add term rights to the vocabs. Maybe it does need the additional control you're talking about. So given that we're developing very similar code in parallel, how do we work closer together? This isn't a problem while we're all experimenting and still trying to clarify the approach. There's another recent post in this forum where I've pointed to game plans. -- Julian Bond Email&MSM: julian.bond at voidstar.com Webmaster: http://www.ecademy.com/ Personal WebLog: http://www.voidstar.com/ M: +44 (0)77 5907 2173 T: +44 (0)192 0412 433 S: callto://julian.bond/
So given that we're developing very similar code in parallel, how do we work closer together? This isn't a problem while we're all experimenting and still trying to clarify the approach.
The biggest difference between our efforts is that I'm now angling toward a) all core, b) smallest code possible. As such, the core patch I'll be contributing will: * NOT have any sort of "My Tags" functionality (it'll log the uids as I've previously planned, but it won't actually do anything with them), * NOT have any sort of "Similar Keywords" functionality http://www.disobey.com/detergent/2005/similar_keywords.jpg * NOT have any new block to show Neighboring Keywords. The goal here is just to get SOMETHING functional into core.
So given that we're developing very similar code in parallel, how do we work closer together? This isn't a problem while
With that said, after the core patch, I'll begin working on a new contrib module called "taxonomy_similar" (or something) to display the "Similar Keywords" I need. Perhaps you could do a "taxonomy_mytags", module based around the term_node.uid addition (which, in my current core patch, doesn't work, but should tonight or tomorrow)? -- Morbus Iff ( i've eaten fruity pebbles with p-diddy ) Technical: http://www.oreillynet.com/pub/au/779 Culture: http://www.disobey.com/ and http://www.gamegrene.com/ icq: 2927491 / aim: akaMorbus / yahoo: morbus_iff / jabber.org: morbus
Perhaps you could do a "taxonomy_mytags", module based around the term_node.uid addition (which, in my current core patch, doesn't work, but should tonight or tomorrow)? A related question. Is it feasible to implement folksonomies not shy of hierarchies - since your code uses the core taxonomy, at least in theory it should be possible to do that in a module, but I'm not sure of the folksonomy implementation. What I'm after is to be able to improve the 'similarity' measure based on partial taxonomy paths. But for folksonomies, or soicial classification, it could be a nice future improvement.
Cheers, Vlado
A related question. Is it feasible to implement folksonomies not shy of hierarchies - since your code uses the core taxonomy, at least in theory it should be possible to do that in a module, but I'm not sure of the folksonomy implementation. What I'm after is to be able to improve the 'similarity' measure based on partial taxonomy paths. But for folksonomies, or soicial classification, it could be a nice future
Well, the current patch "technically" supports your request, but not from a community-created viewpoint. If you've got 1000 terms that people have submitted, you'd be able to go in (with "administer taxonomy" permissions) and manually assign the terms hierarchy and relations and synonyms. All that stuff would work as a normal taxonomy would. The big problem is the whole "community defined" part. Say one person added "Cat, Animal, Dog, Video". Your taxonomy would have four flat tags. For you, as the administrator, it'd be easy as pie to make "Cat" and "Dog" a child of "Animal". And the next time someone types in "Cat" or "Dog", the code will properly assign the node to "Cat" or "Dog", now children of the "Animal" term. But, what if you also have a "Cartoon Characters" parent? There's a cartoon called "Cat and Dog" (seriously), where the characters are called, well, "Cat" and "Dog". So, if someone types in "Cat" and "Dog", there is now no indication of WHICH parent ("Animal" or "Cartoon Characters") the node should belong to. The current code will randomly pick one. I guess the only solution would be to support an ASCII-ish hierarchy, such that "Animal > Cat, Cartoon Characters > Dog, Video" could be entered and understood, but the database lookups and code to do so would be (at first glance) prohibitive, so it's something I'd throw on the faaar back burner. -- Morbus Iff ( you are nothing without your robot car, NOTHING! ) Culture: http://www.disobey.com/ and http://www.gamegrene.com/ Spidering Hacks: http://amazon.com/exec/obidos/ASIN/0596005776/disobeycom icq: 2927491 / aim: akaMorbus / yahoo: morbus_iff / jabber.org: morbus
Well, the current patch "technically" supports your request, but not from a community-created viewpoint. That's what I wasn't really sure about, since I'm not that versed in the taxonomy code.
The big problem is the whole "community defined" part. ... If the rest, including the ascii-ish hierarchy editing form, can be overloaded in a module via a hook, it would probably be best. This is what I had in mind anyway.
Cheers, Vlado
Morbus Iff <morbus@disobey.com> Wed, 16 Mar 2005 09:45:42
I guess the only solution would be to support an ASCII-ish hierarchy, such that "Animal > Cat, Cartoon Characters > Dog, Video" could be entered and understood, but the database lookups and code to do so would be (at first glance) prohibitive, so it's something I'd throw on the faaar back burner.
Beware of creating more hierarchies when the whole point of folksonomy is to derive them if they exist at all. -- Julian Bond Email&MSM: julian.bond at voidstar.com Webmaster: http://www.ecademy.com/ Personal WebLog: http://www.voidstar.com/ M: +44 (0)77 5907 2173 T: +44 (0)192 0412 433 S: callto://julian.bond/
Morbus Iff <morbus@disobey.com> Wed, 16 Mar 2005 07:36:17
With that said, after the core patch, I'll begin working on a new contrib module called "taxonomy_similar" (or something) to display the "Similar Keywords" I need. Perhaps you could do a "taxonomy_mytags", module based around the term_node.uid addition (which, in my current core patch, doesn't work, but should tonight or tomorrow)?
Let's try and pull this together. - Use taxonomy tables. With two additions: 1) Add a UID field to term_node. 2) Add a flag on vocabulary. "Folksonomy" - Extend taxonomy.module. My reading is that the following issues will arise:- Required 1) Taxonomy_form needs to provide a choice of input field depending on the folksonomy flag. If it's turned on then it uses a form_textfield rather a form_select for each folksonomy vocabulary that is valid for this node type. 2) _save() needs to iterate through the returning comma delimited strings from the new text fields. It splits them, cleans them up and then creates new entries in term_data if needed. This will need some helper functions to convert from comma string to clean array of tids/names and back again. The list of tids is then passed to the existing _save() code to create entries in term_node. This should also set the uid value on term_node. Optional 3) Term display and edit is turned off in admin_overview for folksonomy vocabs. This is optional and is waiting for a better way of displaying and editing the expected very large numbers of terms in a folksonomy vocabulary. 4) The term display on nodes is improved so that terms from vocabs are grouped by vocab and have an optional vocab name before them. Hidden text links are added to provide technorati with something to scrape and find the tags for this node. 5) A hook in the save() code populates term_relation to provide "tags like this one" cluster analysis and navigation. 6) A hook in the _form_all() code makes it easy to extend the $help description with suggested terms from Most used terms, My terms, Related terms, Synonyms and anything else we think of later. 7) Contrib modules get built that add things like a Most popular terms page, All terms page, a block for navigating related and my terms for the current term page, links to technorati and so on. 8) Work is started on "nice" URLs for taxonomy category pages. eg /taxonomy/fterm/horse+riding //"horse riding" from any vocabulary /taxonomy/vterm/keywords/horse+riding //"horse riding" from the "keywords" vocabulary /taxonomy/vterm/keywords //All nodes with a term from the "keywords" vocabulary -- Julian Bond Email&MSM: julian.bond at voidstar.com Webmaster: http://www.ecademy.com/ Personal WebLog: http://www.voidstar.com/ M: +44 (0)77 5907 2173 T: +44 (0)192 0412 433 S: callto://julian.bond/
These look like reasonable changes to me. [snip]
Optional 3) Term display and edit is turned off in admin_overview for folksonomy vocabs. This is optional and is waiting for a better way of displaying and editing the expected very large numbers of terms in a folksonomy vocabulary.
see the theme configuration where we have toggle switches for this sort of thing.
5) A hook in the save() code populates term_relation to provide "tags like this one" cluster analysis and navigation.
not sure whats being accomplished here. in any case, we already have hook_taxonomy() which should do what you need.
6) A hook in the _form_all() code makes it easy to extend the $help description with suggested terms from Most used terms, My terms, Related terms, Synonyms and anything else we think of later.
again, not sure whats meant here. but its Optional so lets omit it for the moment.
8) Work is started on "nice" URLs for taxonomy category pages. eg /taxonomy/fterm/horse+riding //"horse riding" from any vocabulary /taxonomy/vterm/keywords/horse+riding //"horse riding" from the "keywords" vocabulary /taxonomy/vterm/keywords //All nodes with a term from the "keywords" vocabulary
this is much needed. perhaps the naming scheme from commentrss.module will be sueful. see its readme file.
Moshe Weitzman <weitzman@tejasa.com> Wed, 16 Mar 2005 09:21:19
5) A hook in the save() code populates term_relation to provide "tags like this one" cluster analysis and navigation.
not sure whats being accomplished here. in any case, we already have hook_taxonomy() which should do what you need.
del.icio.us and flickr make a big thing of showing "related tags" when viewing results for a tag. The underlying data to present this should presumably be in term_relation. You can calculate it on the fly but it's DBMS intensive. So it needs calculating in background via cron or at save time. And yes, it could be done via hook_taxonomy. -- Julian Bond Email&MSM: julian.bond at voidstar.com Webmaster: http://www.ecademy.com/ Personal WebLog: http://www.voidstar.com/ M: +44 (0)77 5907 2173 T: +44 (0)192 0412 433 S: callto://julian.bond/
Julian Bond <julian_bond@voidstar.com> Wed, 16 Mar 2005 15:26:02
del.icio.us and flickr make a big thing of showing "related tags" when viewing results for a tag. The underlying data to present this should presumably be in term_relation. You can calculate it on the fly but it's DBMS intensive. So it needs calculating in background via cron or at save time. And yes, it could be done via hook_taxonomy.
Actually there is no hook in _node_save(). It would need an array of tids and the hook would need to be after all the term_node entries had been created right at the end of the function. -- Julian Bond Email&MSM: julian.bond at voidstar.com Webmaster: http://www.ecademy.com/ Personal WebLog: http://www.voidstar.com/ M: +44 (0)77 5907 2173 T: +44 (0)192 0412 433 S: callto://julian.bond/
Let's try and pull this together.
- Use taxonomy tables. With two additions: 1) Add a UID field to term_node. 2) Add a flag on vocabulary. "Folksonomy"
Right. These changes have already been patched out and are in my sandbox. Per the install instructions in the README.txt, you should be able to: * Run updatedb.$DATABASE against your current HEAD installation. * Patch your modules/taxonomy.module per the provided patch. * Create a taxonomy as normal; notice the new "community" option. * Test out a few nodes and created-terms from this taxonomy. I chose "community" INSTEAD OF "folksonomy" per the following reasons: Next up, we modify the vocabulary table with a new "community" column. This column determines if we should display a taxonomy selectbox or input box on the associated forms, as well as how we should expect our user-chosen terms to be sent to us upon submission. The name "community" is intent-focused, as opposed to "is_folksonomy" or "default_ui", arguably more descriptive, but also more breakable.
1) Taxonomy_form needs to provide a choice of input field depending on the folksonomy flag. If it's turned on then it uses a form_textfield rather a form_select for each folksonomy vocabulary that is valid for
This is already done. Have you seen my patch and UI?
2) _save() needs to iterate through the returning comma delimited strings from the new text fields. It splits them, cleans them up and then creates new entries in term_data if needed. This will need some helper functions to convert from comma string to clean array of tids/names and back again.
This is already done. Have you seen my patch and UI?
3) Term display and edit is turned off in admin_overview for folksonomy vocabs. This is optional and is waiting for a better way of displaying and editing the expected very large numbers of terms in a folksonomy
Per my TODO, I'm planning on experimenting with turning the current vocabulary display into a pager display, similar to all the other "lots of data" menus within Drupal. To ensure that the hierarchy is still visibly noticable across multiple pagers (where the parent could be on one page, and the child on another), the full hierarchy will always be spit: Instead of: Parent Name (edit term) Child 1 (edit term) Child 2 (edit term) The planned pager display will become: Taxonomy/Term | Operations ----------------------------------------- Parent Name/ | edit Parent Name/Child 1 | edit Parent Name/Child 2 | edit Folksonomy | edit Folksonomy/Cat | edit and so on. Regarding the rest of your stuff, all good, and all stuff I haven't yet thought about. Minor comments below.
4) The term display on nodes is improved so that terms from vocabs are grouped by vocab and have an optional vocab name before them. Hidden
Less a folksonomy issue and more a taxonomy bug. I'd consider making a new Issue on this and patching it independently of our folksonomy work.
text links are added to provide technorati with something to scrape and
This should be entirely optional. I'd never want this on my stuff.
5) A hook in the save() code populates term_relation to provide "tags like this one" cluster analysis and navigation.
I won't be working on this.
6) A hook in the _form_all() code makes it easy to extend the $help description with suggested terms from Most used terms, My terms, Related terms, Synonyms and anything else we think of later.
Hmm. I'm not sure about this. I see the $help stuff as "help about the Drupal feature" instead of "help about choosing the right tags for this site which just happens to be using Drupal".
7) Contrib modules get built that add things like a Most popular terms page, All terms page, a block for navigating related and my terms for the current term page, links to technorati and so on.
Right - this is where my (planned) taxonomy_similar module comes in - it'll allow the user to visually choose similar keywords "better" than the ones he may have previously chosen. Or, he can just ignore it and close the browser immediately. Your "links to technorati" in this "Contrib module" item is the solution to my concern above ("make technorati links entirely optional").
8) Work is started on "nice" URLs for taxonomy category pages. eg
I've still not figured out the perfect way of implementing this. -- Morbus Iff ( you are nothing without your robot car, NOTHING! ) Culture: http://www.disobey.com/ and http://www.gamegrene.com/ Spidering Hacks: http://amazon.com/exec/obidos/ASIN/0596005776/disobeycom icq: 2927491 / aim: akaMorbus / yahoo: morbus_iff / jabber.org: morbus
Morbus Iff <morbus@disobey.com> Wed, 16 Mar 2005 09:35:58
text links are added to provide technorati with something to scrape and This should be entirely optional. I'd never want this on my stuff.
Fair enough. It really needs nice URLs as Technorati uses the last directory in a URL to extract the tag.
6) A hook in the _form_all() code makes it easy to extend the $help description with suggested terms from Most used terms, My terms, Related terms, Synonyms and anything else we think of later.
Hmm. I'm not sure about this. I see the $help stuff as "help about the Drupal feature" instead of "help about choosing the right tags for this site which just happens to be using Drupal".
I think it's important that there are feedback mechanisms that encourage convergence on tags. And the nutricious interface to del.icio.us (now available as a beta bookmarklet on del.icio.us) makes del.icio.us way more usable. I really want a mechanism to allow this to be built in although I'll concede that it probably shouldn't be in core because it's a prime area for experiments. -- Julian Bond Email&MSM: julian.bond at voidstar.com Webmaster: http://www.ecademy.com/ Personal WebLog: http://www.voidstar.com/ M: +44 (0)77 5907 2173 T: +44 (0)192 0412 433 S: callto://julian.bond/
On 15 Mar 2005, at 22:25, Morbus Iff wrote:
I've commited into sandbox/morbus/folksonomy_shared (along with a bunch of documentation and design decisions) the first, very rough and not heavily tested, version of folksonomy_shared. There are still a number of things to do (and a few known bugs to fix) but I wanted to show you what I have.
Here is the new vocabulary definition screen:
Looks good. Not sure about the first screenshot ('community' checkbox). I'm sure there is room for improvement there. *looks at the usability guys* -- Dries Buytaert :: http://www.buytaert.net/
participants (6)
-
Dries Buytaert -
Julian Bond -
Morbus Iff -
Moshe Weitzman -
Scott Courtney -
Vladimir Zlatanov