CCK field generation confusion (and how to solve)
Hi, I have been fighting CCK fields for the last two days again. My general conclusion already was that it is a little too hard to grok, so let us get that solved. Why? Because it is too hard to grok. Looking closer at the field that are there, you will notice that the developers were: a) inconsistent. Very inconsistent. Every module with fields is completely different. b) using fields where they should have used widgets. And using widgets where they should have used fields. c) seem to be overly confused about the 'multiple' thing. How? Documentation. This is a catch22, because how much I would love to make docs, I am too unfamiliar with CCK to write docs. Even some sketchy notes by/from the original designers (why multiple, what was the idea behind fields vs widgets) can help. Are they anywhere? And more technical: Documentation can help us solve some of the issues. But even with docs and examples, this material is very hard to grok. The workflow is like spaghetti, the hooks, using call-by-refs trace back up-to 5(!) hook calls deep! (meaning: hook_foo calls hook_bar, which calls hook_baz etc, 5 of them is the most I counted). Call by reference: Either all of the stuff should be call by ref in all hooks at all points. Or none. Not have $node as called by ref in some $ops (load) but not in others (insert). Hooks vs FAPI. Because the general Drupal-way seems to be towards _altering of FAPI forms, instead of the 'old' hook mechanism, CCK uses many hooks. To alter forms. But not everywhere. These CCK hooks are well documented already, but when it mixes with form alters, you get a messy spaghetti-bowl again. I'd say: either _alters all over the place, or hooks, but never both. Hence I suggest to drop the hook_*() $op == 'form' in favour of form_alters all trough CCK. Fields widgets and the rest. This is confusing. When I read peoples fields-modules my general idea is that no-one really groks this. Eventhough I understand the database architecture behind it , I still think that a) the names are confusing. b) the associated hook ops are badly named. Hence I suggest to take the widgets out of CCK entirely, and stick them in a widget_library.module. CCK-field modules can pick widgets from that library. If they need additional widgets, then they can add them to the library. This way we can do away with the current 'widgets and fields' confusion. And we can enforce people to think slightly better about their fields vs their widgets. Right now, nearly all of the modules with CCK fields have widgets-who-validate-field-data, or fields-whom-dictate-widgets. An additional benefit is that any module can easily re-use fancy widgets without needing the whole of CCK. Another benefit, is that the amount of code that needs to be changed can be minimised. A big downside is the dependency of CCK on the widget_library.module. Who? There are a lot of users, but very few developers for CCK. The main reason, IMO is the very steep learning curve of CCK development. (see above) But I think that enough people are willing to contribute if CCK were a little more accessible. If we can agree on the abovementioned ideas, or provide better ones, then I hope we can get this started with some patches and issues right away. Bèr -- [ Bèr Kessels | Drupal services www.webschuur.com ]
On 10 Oct 2006, at 16:52, Bèr Kessels wrote:
There are a lot of users, but very few developers for CCK. The main reason, IMO is the very steep learning curve of CCK development. (see above) But I think that enough people are willing to contribute if CCK were a little more accessible. If we can agree on the abovementioned ideas, or provide better ones, then I hope we can get this started with some patches and issues right away.
I agree. I hope we can simplify the CCK -- or at least make it more accessible -- when we move the remaining bits to core. It took many iterations to get the current CCK stuff in core as simple as possible but I'm determined to do the same with what is left. I said it before and I'll say it again: hackability matters a lot. Complexity is a disease. -- Dries Buytaert :: http://www.buytaert.net/
On 10-Oct-06, at 10:52 AM, Bèr Kessels wrote:
How? Documentation. This is a catch22, because how much I would love to make docs, I am too unfamiliar with CCK to write docs.
I'm the same. However, I went ahead and documented what I did know and could figure out here: http://drupal.org/node/82661 Then, other people started chiming in with what they knew, and between the original docs and the comments, there's a lot of great information there now! Never let the fact that you don't understand something stop you from documenting it. :) Documentation is and should be a community process. -Angie
Op dinsdag 10 oktober 2006 18:49, schreef Angela Byron:
Never let the fact that you don't understand something stop you from documenting it. :) Documentation is and should be a community process.
I wholeheartedly agree. Documentation is something that must be 'Just Done' like so many other things: However, Documenting stuff that is fundamentally flawed (I am not saying CCK is), or documenting stuff that is simply broken doen't really help. It is easy to say 'this is not a bug, because we documented it'. Its better to just fix the bug. Hence I want to start off with making some innards of CCK simpler and more logical from developers POV. And in that same process document these innards. Right now one can spend ages on explaining what the difference is between widgets and fields. But without the code supporting that difference, without examples that show these differences, it is an uphill battle. Bèr -- [ Bèr Kessels | Drupal services www.webschuur.com ]
On Tue, 2006-10-10 at 16:52 +0200, Bèr Kessels wrote:
Hi,
I have been fighting CCK fields for the last two days again. My general conclusion already was that it is a little too hard to grok, so let us get that solved.
Agreed.
Why? Because it is too hard to grok. Looking closer at the field that are there, you will notice that the developers were: a) inconsistent. Very inconsistent. Every module with fields is completely different.
Can you point out some specific examples. hook_field provides the data definitions, validation, and crud necessary stuff. hook_widget is for building user interfaces. ie) select lists, ajaxy things, flash movies and preparing the results for validation.
b) using fields where they should have used widgets. And using widgets where they should have used fields.
Do you know the distinction? You've already expressed that you don't understand it, so maybe you should take the time to grok what is there now before making judgments on what is right or wrong.
c) seem to be overly confused about the 'multiple' thing.
Multiple... a field can be a single value or multiple value field. What's confusing about it? I admit its complex and often difficult to deal with. I
How? Documentation. This is a catch22, because how much I would love to make docs, I am too unfamiliar with CCK to write docs. Even some sketchy notes by/from the original designers (why multiple, what was the idea behind fields vs widgets) can help. Are they anywhere?
Why multiple because some fields will want to allow for more than a single value, aka lists. The multiple flag has a couple implications... 1) any field instance that has multiple set as true will get its own field. 2) there is an assumption that any single value field can become a multiple value field so to make everything more consistent all field structures are built as arrays, even those returning a single value. hence all the $field_foo[0]['view'] and while($field = each($field_foo)) etc. fields vs widgets is separating ui and data model. (I agree these should probably be re-factored to be easier to grok. maybe we'll see some good ideas as it heads into core.)
And more technical: Documentation can help us solve some of the issues. But even with docs and examples, this material is very hard to grok. The workflow is like spaghetti, the hooks, using call-by-refs trace back up-to 5(!) hook calls deep! (meaning: hook_foo calls hook_bar, which calls hook_baz etc, 5 of them is the most I counted).
Yes the hook system is deep, but you can't really treat the field modules as full drupal modules. They should only implement module hooks necessary. Since content.module (soon to be field.module??) Handles delegating tasks via hook content so it can aggressively cache the field modules output so these deep calls should not occur too often. As we move closer to FAPI3/APIAPI the field modules should simpler. They can begin becoming UI's to create field_instances and help build the data model instead of doing the heavy lifting.
Call by reference: Either all of the stuff should be call by ref in all hooks at all points. Or none. Not have $node as called by ref in some $ops (load) but not in others (insert).
Depends on the expected return of the hook, but yeah I don't like teh api hooks that much... I like simple direct hook_field_insert instead of hook_field(op == insert). I also think code is more understandable when broken into the little chunks like that. (FAPI is taking us this direction I believe).
Hooks vs FAPI. Because the general Drupal-way seems to be towards _altering of FAPI forms, instead of the 'old' hook mechanism, CCK uses many hooks. To alter forms. But not everywhere. These CCK hooks are well documented already, but when it mixes with form alters, you get a messy spaghetti-bowl again. I'd say: either _alters all over the place, or hooks, but never both. Hence I suggest to drop the hook_*() $op == 'form' in favour of form_alters all trough CCK.
Then CCK can't cache aggregate output. Which would mean going through those deep layers of callbacks to all the field modules for each field_instance associated to the content type...
Fields widgets and the rest. This is confusing. When I read peoples fields-modules my general idea is that no-one really groks this. Eventhough I understand the database architecture behind it , I still think that a) the names are confusing. b) the associated hook ops are badly named.
maybe true... I think a lot of what is going on is hybrid drupal/cck and tradition node module development. I'm guilty of this myself and have been seriously rethinking how my field modules are implemented. JonBob asked me a while back why I didn't implement all the file data storage through CCK... I thought at the time sharing drupal's files tables would be a great idea, especially since it would make it easier to interact with non-cck modules ideally... and because I want a centralized file api that give me access to the total pool of files and does the heavy lifting for me... I am now realizing that shared field_instances allow me to create ad hoc pools of files and give me instant views integration. This may be the way to go and I think was in the original vision.
Hence I suggest to take the widgets out of CCK entirely, and stick them in a widget_library.module. CCK-field modules can pick widgets from that library. If they need additional widgets, then they can add them to the library. This way we can do away with the current 'widgets and fields' confusion. And we can enforce people to think slightly better about their fields vs their widgets. Right now, nearly all of the modules with CCK fields have widgets-who-validate-field-data, or fields-whom-dictate-widgets. An additional benefit is that any module can easily re-use fancy widgets without needing the whole of CCK. Another benefit, is that the amount of code that needs to be changed can be minimised. A big downside is the dependency of CCK on the widget_library.module.
I don't think widgets should be separate from fields. In a Utopian world, User Interface would not be bound at all by the data structures they interface with... This isn't so Utopian. Part of the widget's job is formatting data for the underlying data model. I'm not sure that the two can be cleanly separated. If you got a good idea, write the patch I'll be happy to review it. Validation and input preparation are kind of grey area to some people and they have differing opinions on where it goes... I personally believe the widget should handle generating the UI and preparing the input for the data model. The data model should handle its validation and crud work.
Who? There are a lot of users, but very few developers for CCK. The main reason, IMO is the very steep learning curve of CCK development. (see above) But I think that enough people are willing to contribute if CCK were a little more accessible.
I don't think the learning curve for cck is steep. I figured most of it out in about 2 days of poking around and actually writing code for it. A little trial and error and reading the code in content.module goes a long way. Yes, there is are additional abstraction layers for fields and widgets, and you've got to figure out where it interfaces with the normal node building and life-cycle code. Angie has began an effort to document CCK. Which I'll agree isn't an easy task. It also changes underneath you on occasion without any sort of notice, which can be challenging.
If we can agree on the abovementioned ideas, or provide better ones, then I hope we can get this started with some patches and issues right away.
It would be nice if you can file bugs for where you see these inconsistencies in the field modules. Maybe there is a reason for them and if we can suss out the reasons for those work arounds we will probably find something in cck that can be improved upon.
Bèr
Thanks for bringing this stuff up Ber. JonBob has been in a communications Bermuda triangle for a while. I'm not sure that he has the bandwidth to document a lot of things currently.
On Tue, 2006-10-10 at 17:31 -0400, Darrel O'Pry wrote:
1) any field instance that has multiple set as true will get its own field.
correction 1) any field instance that has multiple set as true will get its own table.
I don't think the learning curve for cck is steep. I figured most of it out in about 2 days of poking around and actually writing code for it.
I'd think that '2 days' is pretty steep. -- Dries Buytaert :: http://www.buytaert.net/
On Wed, 2006-10-11 at 13:53 +0200, Dries Buytaert wrote:
I don't think the learning curve for cck is steep. I figured most of it out in about 2 days of poking around and actually writing code for it.
I'd think that '2 days' is pretty steep.
Not so bad with no docs and only example code to work with. Basic Docs exist now. I think proper documentation will go a long ways towards making it easier and simpler. It takes some people longer to figure out the hooks system and FormAPI and they're sorta well documented (I think we miss out on a lot of 'theory of operation' in our docs... and focus on the syntax and behavior.) Plus it can still take better advantage of FormAPI than it does currently. It was originally designed pre-FormAPI if I remember correctly. Then there were only api hooks to work with. It still has that mindset in many parts of its core. .darrel.
Op dinsdag 10 oktober 2006 23:31, schreef Darrel O'Pry:
On Tue, 2006-10-10 at 16:52 +0200, Bèr Kessels wrote:
Hi,
I have been fighting CCK fields for the last two days again. My general conclusion already was that it is a little too hard to grok, so let us get that solved.
Agreed.
Why? Because it is too hard to grok. Looking closer at the field that are there, you will notice that the developers were: a) inconsistent. Very inconsistent. Every module with fields is completely different.
Can you point out some specific examples. hook_field provides the data definitions, validation, and crud necessary stuff.
Yup. datefields do a lot of data-refactoring inside the widget parts. Your imagefields do most of the validation inside the 'prepare' op of the widget (prepar, is mean to, yup, prepare the data from the DB for the display in a form', never for validation of files, IMO). My point was, that I see nearly all the modules with fields and widgets hae completely different code to: * handle multiple fields * manage / share data beween widgets and fields And on top of that, they are all different when it comes to calling functions etc. One module has many lines of code inside the hook_field, another one calls all kinds of private functions in that hook_field. All these flavours of coding add to the confusion.
hook_widget is for building user interfaces. ie) select lists, ajaxy things, flash movies and preparing the results for validation.
b) using fields where they should have used widgets. And using widgets where they should have used fields.
Do you know the distinction? You've already expressed that you don't understand it, so maybe you should take the time to grok what is there now before making judgments on what is right or wrong.
Yes, I know what a) The distinction should be (you call it Utopian) and b) it is now. Both are different and leave a large grey ara. That area is so large, and so grey, that right now, not one CCK module actually needs both to be different. Every CCK module defines both widgets and fields and nearly all of them are of form one-field-needs-one-widget. As it stands now, as modules use it, that distinction would not be needed.
c) seem to be overly confused about the 'multiple' thing. Multiple... a field can be a single value or multiple value field. What's confusing about it? I admit its complex and often difficult to deal with. I
Sure. But selectlist module abuses multiple to turn radios into checkboxes, for example. Other modules simply ignore the multiple values, and the 'better' modules need one if and one foreach on *each* op in fields. That is a LOT of complexity for little benefit. In the database architecture this is used, which is cool, but the useage (for developers) of the multiple thing is still rather hard to grok.
Yes the hook system is deep, but you can't really treat the field modules as full drupal modules. They should only implement module hooks necessary.
I don't really care what we treat what module as. :) flexinode has its private .inc system, and it takes 20 minutes of copy-pasting to make new fields. The architecture might suck, the performance may be horific (I am curious about benchmarks CCK vs Flexinode, I have a gut feeling it wont differ that much), but making a field is a breeze!
Since content.module (soon to be field.module??) Handles delegating tasks via hook content so it can aggressively cache the field modules output so these deep calls should not occur too often. As we move closer to FAPI3/APIAPI the field modules should simpler. They can begin becoming UI's to create field_instances and help build the data model instead of doing the heavy lifting.
I think we should not lean too hard on FAPIAPAPAPIAPI, but rather make CCK simpler by itself :). Off course this will mean using more existing Drupalisms, but the main task is to focus on simplicity IMO, which I doubt that APIAPI will bring. (yes, I am rather sceptical about APIAPI, since I am not overly enthusiastic about all the 'good' FAPI brought us. but thats a different issue, just to put the statement above in perspective).
Call by reference: Either all of the stuff should be call by ref in all hooks at all points. Or none. Not have $node as called by ref in some $ops (load) but not in others (insert).
Depends on the expected return of the hook, but yeah I don't like teh api hooks that much... I like simple direct hook_field_insert instead of hook_field(op == insert). I also think code is more understandable when broken into the little chunks like that. (FAPI is taking us this direction I believe).
Well, imagefield has bugs because it assumes stuff is called by ref, which its not. We already discussed this, and indeed CCK HEAD has this better sorted then 4.7, still its overly confusing. You need to trace back stuff with dprints sometimes five hooks deep, to find whether something should have been returned, or called-by-ref. This is unneeded complexity. I think we can start from here, with simplification. I will add some issues in the next days, but I ned to give it some more thought. Plan is to split out hook_field $op into hook_field_op (e.g. hook_field_insert). That way we have less functions, doing more dedicated jobs and erceiving clearer arguments.
Hooks vs FAPI. Because the general Drupal-way seems to be towards _altering of FAPI forms, instead of the 'old' hook mechanism, CCK uses many hooks. To alter forms. But not everywhere. These CCK hooks are well documented already, but when it mixes with form alters, you get a messy spaghetti-bowl again. I'd say: either _alters all over the place, or hooks, but never both. Hence I suggest to drop the hook_*() $op == 'form' in favour of form_alters all trough CCK.
Then CCK can't cache aggregate output. Which would mean going through those deep layers of callbacks to all the field modules for each field_instance associated to the content type...
Not in its current form, but I am sure with some creativity we can still cache, even if we use form_alters.
I don't think widgets should be separate from fields. In a Utopian world, User Interface would not be bound at all by the data structures they interface with... This isn't so Utopian. Part of the widget's job is formatting data for the underlying data model. I'm not sure that the two can be cleanly separated. If you got a good idea, write the patch I'll be happy to review it.
Just because we cannot reach a utopia does not mean we cannot try to get closer. "part of the widgets job" is to prepare stuff for FAPI. But certainly not to set/unset file SESSION data, just to name something :) If we make fields a tad more powerfull, then they can eat anythign that FAPI feeds it. FRom that point onwards, widgets are no longer something from CCK, then widgets are a FAPI thing. And then widgets can be re-used all trough drupal. See e.g. helpers module where I made a country_select() widget to render a selectfield to select a country. Everyone can use that "widget". A CCK field would need to be able to eat what country_select(), trough FAPI, feeds it. It will strip at least 60% of the complexity out of CCK.
Validation and input preparation are kind of grey area to some people and they have differing opinions on where it goes... I personally believe the widget should handle generating the UI and preparing the input for the data model. The data model should handle its validation and crud work.
Well, right now, as mentioned above, the widget provides the interface, part of the data-manipulation, then FAPI handles the data and passes it on to the field. I think we can safely cut the "part of the data-manipulation" out of that, and leave that to FAPI and fields.
Who? There are a lot of users, but very few developers for CCK. The main reason, IMO is the very steep learning curve of CCK development. (see above) But I think that enough people are willing to contribute if CCK were a little more accessible.
I don't think the learning curve for cck is steep. I figured most of it out in about 2 days of poking around and actually writing code for it. A little trial and error and reading the code in content.module goes a long way. Yes, there is are additional abstraction layers for fields and widgets, and you've got to figure out where it interfaces with the normal node building and life-cycle code.
I spent two weeks developing taxonomy-fields. And only when I drew out a complete ERD on a whiteboard, did I start to grok the concepts of CCK. But even then, no-one can be certain that he/she really grokked it. Even when a module appears to work from the outside, that is not a proof that the author really grokked CCK :), there are many roads to Rome, and apparently for CCK no-one found the Right road, yet. I am not exactly a good developer, but it took me far less time to grok flexinode, for example. I think 2 days is only realistic for the good developers amoungst us. Joe Developer will have his time measured in months to develop a field, I fear.
If we can agree on the abovementioned ideas, or provide better ones, then I hope we can get this started with some patches and issues right away.
It would be nice if you can file bugs for where you see these inconsistencies in the field modules.
I've started this. But its really a huge challenge to make patches for a module that is complex, runs on an overly complex foundation, and wich lacks any documentation. Look, I consider the imagefields session manipulation in the widgets $ op == prepare a bug. You mention it above as though you did that on purpose. As long as such complex things aren't agreed upon, then nothing is a really a bug :).
Maybe there is a reason for them and if we can suss out the reasons for those work arounds we will probably find something in cck that can be improved upon.
Please note, that I have nothing against any of the mentioned modules. I only mention them, to illustrate the result of the complexity of CCK. ot because i dislike them or anything. Bèr -- [ End-user Drupal services and hosting | Sympal.nl ]
On Wed, 2006-10-11 at 16:17 +0200, Bèr Kessels wrote:
Op dinsdag 10 oktober 2006 23:31, schreef Darrel O'Pry:
On Tue, 2006-10-10 at 16:52 +0200, Bèr Kessels wrote:
Hi,
I have been fighting CCK fields for the last two days again. My general conclusion already was that it is a little too hard to grok, so let us get that solved.
Agreed.
Why? Because it is too hard to grok. Looking closer at the field that are there, you will notice that the developers were: a) inconsistent. Very inconsistent. Every module with fields is completely different.
Can you point out some specific examples. hook_field provides the data definitions, validation, and crud necessary stuff.
Yup. datefields do a lot of data-refactoring inside the widget parts. Your imagefields do most of the validation inside the 'prepare' op of the widget (prepar, is mean to, yup, prepare the data from the DB for the display in a form', never for validation of files, IMO).
I think you are confusing what is going on in the imagefield modules. There is very little validation going on in the widget. On the required check, and widget specific validation... was a file attached. The prepare op is not doing any validation of the db input in image field. It is preparing the data provided through the widget... aka dealing with the file upload, and getting an fid to be passed to the data model. It is doing exactly what the prepare op should be doing. Prepping complex data from the widget and converting it to an fid for the data model. I could theoretically add a file_get widget which sucked in URLs or streams instead of file attachments save them to the fs and passed an fid down to the data layer.
My point was, that I see nearly all the modules with fields and widgets hae completely different code to: * handle multiple fields * manage / share data beween widgets and fields And on top of that, they are all different when it comes to calling functions etc. One module has many lines of code inside the hook_field, another one calls all kinds of private functions in that hook_field. All these flavours of coding add to the confusion.
This kind of inconsistent code style, private function called from 'api' callbacks or monolithic switch statements, are common throughout drupal. As a programmer it should be easy to follow down to the private function. I prefer using private functions so other modules can access that functionality directly without needing the indirection of hook_whatever. If its not in the style guide and we don't enforce one way or the other, you're just going to have to deal with that. personally I don't think it should be enforced. It should be up to whoever is maintaining that chunk of code.
hook_widget is for building user interfaces. ie) select lists, ajaxy things, flash movies and preparing the results for validation.
b) using fields where they should have used widgets. And using widgets where they should have used fields.
Do you know the distinction? You've already expressed that you don't understand it, so maybe you should take the time to grok what is there now before making judgments on what is right or wrong.
Yes, I know what a) The distinction should be (you call it Utopian) and b) it is now. Both are different and leave a large grey ara. That area is so large, and so grey, that right now, not one CCK module actually needs both to be different. Every CCK module defines both widgets and fields and nearly all of them are of form one-field-needs-one-widget. As it stands now, as modules use it, that distinction would not be needed.
I haven't had time to create alternate widgets for image field, yet. I plan to use the feature I just haven't had the opportunity. I'd rather not see it go yet.
c) seem to be overly confused about the 'multiple' thing. Multiple... a field can be a single value or multiple value field. What's confusing about it? I admit its complex and often difficult to deal with. I
Sure. But selectlist module abuses multiple to turn radios into checkboxes, for example. Other modules simply ignore the multiple values, and the 'better' modules need one if and one foreach on *each* op in fields. That is a LOT of complexity for little benefit.
If they ignore the multiple value they are broken. I suggest using a while instead, then you don't need the if. Most of the node data after load and content type definition is cached, so it will not be called a lot in most cases.
In the database architecture this is used, which is cool, but the useage (for developers) of the multiple thing is still rather hard to grok.
I still don't think its hard to grok, you just have to work under the assumption, everything has potential multiple values and single values are the unique case. You can eliminate a lot of the uncertainty using better loop structures than foreach. The if($multiple)/foreach is left over from a time when single value fields weren't stored in $field_foo[0] but were $field_foo itself. Its use should probably be deprecated in exchange for while ($field = each()) loops. since they will be more resilient to how single and multiple value fields are currently handled.
Yes the hook system is deep, but you can't really treat the field modules as full drupal modules. They should only implement module hooks necessary.
I don't really care what we treat what module as. :) flexinode has its private .inc system, and it takes 20 minutes of copy-pasting to make new fields. The architecture might suck, the performance may be horific (I am curious about benchmarks CCK vs Flexinode, I have a gut feeling it wont differ that much), but making a field is a breeze!
The caching may make quite a difference. I'm not as clear on the internals of flexinode any more, but a lot of things from cck are cached. A whole lot. Personally I also like the .inc system, but its not as efficient if you have a compile cache like zend or apc. But I do like selective loading of needed code myself as well. OT: My personal curiosity is how much impact the cache overhead has. Its not free and comes at the beginning of page load I think. I'm not an expert on drupal's caching by any means... Does it load the entire cache from the db each load, or just selectively loads cache 'domains' as needed? It would be nice if it just loaded selective cache namespaces as requested. I think the lowered memory overhead and bootstrap time would be good for drupal. if it's an issue.
Since content.module (soon to be field.module??) Handles delegating tasks via hook content so it can aggressively cache the field modules output so these deep calls should not occur too often. As we move closer to FAPI3/APIAPI the field modules should simpler. They can begin becoming UI's to create field_instances and help build the data model instead of doing the heavy lifting.
I think we should not lean too hard on FAPIAPAPAPIAPI, but rather make CCK simpler by itself :). Off course this will mean using more existing Drupalisms, but the main task is to focus on simplicity IMO, which I doubt that APIAPI will bring. (yes, I am rather sceptical about APIAPI, since I am not overly enthusiastic about all the 'good' FAPI brought us. but thats a different issue, just to put the statement above in perspective).
Yeah it touches into the territory of overly complex. I can follow, I think, because I've been observing and engaged in its development. I think it has a much higher learning curve than the hooks system or cck. However the concept of dynamically constructed content that manages its own db schema is not something prone to simplification, and in many ways will be difficult to simplify. It also probably might not work when over simplified. I think better documentation will be good for it as well.
Call by reference: Either all of the stuff should be call by ref in all hooks at all points. Or none. Not have $node as called by ref in some $ops (load) but not in others (insert).
Depends on the expected return of the hook, but yeah I don't like teh api hooks that much... I like simple direct hook_field_insert instead of hook_field(op == insert). I also think code is more understandable when broken into the little chunks like that. (FAPI is taking us this direction I believe).
Well, imagefield has bugs because it assumes stuff is called by ref, which its not. We already discussed this, and indeed CCK HEAD has this better sorted then 4.7, still its overly confusing. You need to trace back stuff with dprints sometimes five hooks deep, to find whether something should have been returned, or called-by-ref. This is unneeded complexity.
Yeah its a real pain. If you can document those hooks I'd love it. I've been working blind with CCK from day one. There have been some little changes here and there. The call by ref is one I think I missed happening. Same with the formatters I missed.
I think we can start from here, with simplification. I will add some issues in the next days, but I ned to give it some more thought. Plan is to split out hook_field $op into hook_field_op (e.g. hook_field_insert). That way we have less functions, doing more dedicated jobs and erceiving clearer arguments.
Ber, you da man. I also really appreciate you bringing this up, as well as Angie's attempts to document CCK. I know KarenS and I have pretty much been working from example code, watching JonBob's commits, reading throught cck, trial and error, and debugging. There is little or no documentation and changes often occur without documentation... Its hard for me to keep up with it. Especially, when I have other pet projects I'm working on.
Hooks vs FAPI. Because the general Drupal-way seems to be towards _altering of FAPI forms, instead of the 'old' hook mechanism, CCK uses many hooks. To alter forms. But not everywhere. These CCK hooks are well documented already, but when it mixes with form alters, you get a messy spaghetti-bowl again. I'd say: either _alters all over the place, or hooks, but never both. Hence I suggest to drop the hook_*() $op == 'form' in favour of form_alters all trough CCK.
Then CCK can't cache aggregate output. Which would mean going through those deep layers of callbacks to all the field modules for each field_instance associated to the content type...
Not in its current form, but I am sure with some creativity we can still cache, even if we use form_alters.
I don't think widgets should be separate from fields. In a Utopian world, User Interface would not be bound at all by the data structures they interface with... This isn't so Utopian. Part of the widget's job is formatting data for the underlying data model. I'm not sure that the two can be cleanly separated. If you got a good idea, write the patch I'll be happy to review it.
Just because we cannot reach a utopia does not mean we cannot try to get closer. "part of the widgets job" is to prepare stuff for FAPI. But certainly not to set/unset file SESSION data, just to name something :) If we make fields a tad more powerfull, then they can eat anythign that FAPI feeds it. FRom that point onwards, widgets are no longer something from CCK, then widgets are a FAPI thing. And then widgets can be re-used all trough drupal. See e.g. helpers module where I made a country_select() widget to render a selectfield to select a country. Everyone can use that "widget". A CCK field would need to be able to eat what country_select(), trough FAPI, feeds it. It will strip at least 60% of the complexity out of CCK.
Actually it does have to handle the SESSION data, since I'm using that to store the file meta data until the node is submitted and I can actually add the file to the files table. It is a necessary part of preparing and tracking the file that needs to be saved with the vid/nid of the node/revision we are about to create. That prepare event is responsible for getting a file save to a temp location, so its meta data can be passed on the data layer. That preparation includes processing the _FILES array via file_check_upload and storing the file data in the session until the data layer needs it. If a file is removed before the data layer is called it is removed from the session. It's batching what should be asynchronous actions behavior with the node creation/update cycle, because of data requirements for the crud layer which isn't call until submit. This is one of the more complex prepare use cases currently. It is the same as upload's file handling. I don't think you have widgets quite right yet. They're for things more complex than simple input fields. Think of something like a geo location field. You may way a google map widget to input the data, or just and address field, or maybe even a simple zip code input. All three of these widgets need a certain amount of 'preparation' to get the long/lat for the data model. You should be using a checkbox widget on your 'multivalue select' field instead of a module that addes on more formAPI and logic overhead. While I really like the alter stuff as there are some cool tricks you can do with it, its real power is for customization in my opinion, not as a tool to be used for core modules. Especially now that we have the form pull methodology to work with. We should build the right forms the first time. Although a #prepare, #validate, and #submit maybe all that is needed to be added by the field modules to a subtree of the processing model and much of cck's indirection for handling this stuff through 'api hooks' can disappear. It is also much easier for cck centric people to select the proper widget for a field instance than install a generic help module that alters all form elements of type... Which kind of feels like using a hammer to fix a window.
Validation and input preparation are kind of grey area to some people and they have differing opinions on where it goes... I personally believe the widget should handle generating the UI and preparing the input for the data model. The data model should handle its validation and crud work.
Well, right now, as mentioned above, the widget provides the interface, part of the data-manipulation, then FAPI handles the data and passes it on to the field. I think we can safely cut the "part of the data-manipulation" out of that, and leave that to FAPI and fields.
yeah but the widget still needs to define the data manipulation and I like the path of eliminating the 'prepare' hook and just adding something to #prepare when we create the form. I'm not sure if this exists with the current FAPI.. chx, adrian??
Who? There are a lot of users, but very few developers for CCK. The main reason, IMO is the very steep learning curve of CCK development. (see above) But I think that enough people are willing to contribute if CCK were a little more accessible.
I don't think the learning curve for cck is steep. I figured most of it out in about 2 days of poking around and actually writing code for it. A little trial and error and reading the code in content.module goes a long way. Yes, there is are additional abstraction layers for fields and widgets, and you've got to figure out where it interfaces with the normal node building and life-cycle code.
I spent two weeks developing taxonomy-fields. And only when I drew out a complete ERD on a whiteboard, did I start to grok the concepts of CCK. But even then, no-one can be certain that he/she really grokked it. Even when a module appears to work from the outside, that is not a proof that the author really grokked CCK :), there are many roads to Rome, and apparently for CCK no-one found the Right road, yet. I am not exactly a good developer, but it took me far less time to grok flexinode, for example. I think 2 days is only realistic for the good developers amoungst us. Joe Developer will have his time measured in months to develop a field, I fear.
I mean I still haven't gotten it right. Its part of developing on sand as well. I have grokked parts of cck, then they have changed on me, and I have to go back and re-grok. Which is a pita for me, I'm currenlty stretched thin on time and haven't really had a chance to properly take care of my issue queue for the last couple weeks.
If we can agree on the abovementioned ideas, or provide better ones, then I hope we can get this started with some patches and issues right away.
It would be nice if you can file bugs for where you see these inconsistencies in the field modules.
I've started this. But its really a huge challenge to make patches for a module that is complex, runs on an overly complex foundation, and wich lacks any documentation. Look, I consider the imagefields session manipulation in the widgets $ op == prepare a bug. You mention it above as though you did that on purpose. As long as such complex things aren't agreed upon, then nothing is a really a bug :).
Maybe there is a reason for them and if we can suss out the reasons for those work arounds we will probably find something in cck that can be improved upon.
Please note, that I have nothing against any of the mentioned modules. I only mention them, to illustrate the result of the complexity of CCK. ot because i dislike them or anything.
We know you hate them you Curmudgeon :)... just kidding. But it is nice to kind of share and refactor this information. I shared a good chunk of my reasoning for the session manipulation in the prepare hook of imagefield. Please retort as I can only confirm or correct my understanding, as is the way of Wittgenstein's ladder. I know it is a lot of work. I'm here to assist where I can. I'll put some heavy thought into this next time I run my imagefield queue and have time for more development on filefield...
participants (4)
-
Angela Byron -
Bèr Kessels -
Darrel O'Pry -
Dries Buytaert