[development] CCK field generation confusion (and how to solve)

[Darrel O'Pry]

On Tue, 2006-10-10 at 16:52 +0200, Bèr Kessels wrote:
> Hi,
> 
> I have been fighting CCK fields for the last two days again. My general 
> conclusion already was that it is a little too hard to grok, so let us get 
> that solved.

Agreed.

> Why? 
> Because it is too hard to grok. Looking closer at the field that are there, 
> you will notice that the developers were:
>  a) inconsistent. Very inconsistent. Every module with fields is completely 
> different.  

 Can you point out some specific examples. hook_field provides the data
definitions, validation, and crud necessary stuff. 

  hook_widget is for building user interfaces. ie) select lists, ajaxy
things, flash movies and preparing the results for validation.

>  b) using fields where they should have used widgets. And using widgets where 
> they should have used fields. 

  Do you know the distinction? You've already expressed that you don't
understand it, so maybe you should take the time to grok what is there
now before making judgments on what is right or wrong.

>  c) seem to be overly confused about the 'multiple' thing.

  Multiple... a field can be a single value or multiple value field.
What's confusing about it? I admit its complex and often difficult to
deal with. I

> How?
> Documentation. This is a catch22, because how much I would love to make docs, 
> I am too unfamiliar with CCK to write docs. 
> Even some sketchy notes by/from the original designers (why multiple, what was 
> the idea behind fields vs widgets) can help. Are they anywhere? 

Why multiple because some fields will want to allow for more than a
single value, aka lists. The multiple flag has a couple implications...

1) any field instance that has multiple set as true will get its own
field.

2) there is an assumption that any single value field can become a
multiple value field so to make everything more consistent all field
structures are built as arrays, even those returning a single value. 
hence all the $field_foo[0]['view'] and while($field = each($field_foo))
etc.

fields vs widgets is separating ui and data model. (I agree these should
probably be re-factored to be easier to grok. maybe we'll see some good
ideas as it heads into core.)

> And more technical:
> Documentation can help us solve some of the issues. But even with docs and 
> examples, this material is very hard to grok. The workflow is like spaghetti, 
> the hooks, using call-by-refs trace back up-to 5(!) hook calls deep! 
> (meaning: hook_foo calls hook_bar, which calls hook_baz etc, 5 of them is the 
> most I counted).

  Yes the hook system is deep, but you can't really treat the field
modules as full drupal modules. They should only implement module hooks
necessary. 

  Since content.module (soon to be field.module??) Handles delegating
tasks via hook content so it can aggressively cache the field modules
output so these deep calls should not occur too often. As we move closer
to FAPI3/APIAPI the field modules should simpler. They can begin
becoming UI's to create field_instances and help build the data model
instead of doing the heavy lifting.
 
> Call by reference: 
> Either all of the stuff should be call by ref in all hooks at all points. Or 
> none. Not have $node as called by ref in some $ops (load) but not in others 
> (insert). 

Depends on the expected return of the hook, but yeah I don't like teh
api hooks that much... I like simple direct hook_field_insert instead of
hook_field(op == insert). I also think code is more understandable when
broken into the little chunks like that. (FAPI is taking us this
direction I believe).

> Hooks vs FAPI.
> Because the general Drupal-way seems to be towards _altering of FAPI forms, 
> instead of the 'old' hook mechanism, CCK uses many hooks. To alter forms. But 
> not everywhere. These CCK hooks are well documented already, but when it 
> mixes with form alters, you get a messy spaghetti-bowl again. I'd say: either 
> _alters all over the place, or hooks, but never both. Hence I suggest to drop 
> the hook_*() $op == 'form' in favour of form_alters all trough CCK.

Then CCK can't cache aggregate output. Which would mean going through
those deep layers of callbacks to all the field modules for each
field_instance associated to the content type...

> Fields widgets and the rest. 
>  This is confusing. When I read peoples fields-modules my general idea is that 
> no-one really groks this. Eventhough I understand the database architecture 
> behind it , I still think that
>  a) the names are confusing.
>  b) the associated hook ops are badly named.

maybe true...  I think a lot of what is going on is hybrid drupal/cck
and tradition node module development. I'm guilty of this myself and
have been seriously rethinking how my field modules are implemented.
JonBob asked me a while back why I didn't implement all the file data
storage through CCK... I thought at the time sharing drupal's files
tables would be a great idea, especially since it would make it easier
to interact with non-cck modules ideally... and because I want a
centralized file api that give me access to the total pool of files and
does the heavy lifting for me... 

I am now realizing that shared field_instances allow me to create ad hoc
pools of files and give me instant views integration. This may be the
way to go and I think was in the original vision.

> Hence I suggest to take the widgets out of CCK entirely, and stick them in a 
> widget_library.module. CCK-field modules can pick widgets from that library. 
> If they need additional widgets, then they can add them to the library. This 
> way we can do away with the current 'widgets and fields' confusion. And we 
> can enforce people to think slightly better about their fields vs their 
> widgets. Right now, nearly all of the modules with CCK fields have 
> widgets-who-validate-field-data, or fields-whom-dictate-widgets. 
> An additional benefit is that any module can easily re-use fancy widgets 
> without needing the whole of CCK.
> Another benefit, is that the amount of code that needs to be changed can be 
> minimised. A big downside is the dependency of CCK on the 
> widget_library.module.

I don't think widgets should be separate from fields. In a Utopian
world, User Interface would not be bound at all by the data structures
they interface with...  This isn't so Utopian. Part of the widget's job
is formatting data for the underlying data model. I'm not sure that the
two can be cleanly separated. If you got a good idea, write the patch
I'll be happy to review it.

Validation and input preparation are kind of grey area to some people
and they have differing opinions on where it goes... I personally
believe the widget should handle generating the UI and preparing the
input for the data model. The data model should handle its validation
and crud work.

> Who?
> There are a lot of users, but very few developers for CCK. The main reason, 
> IMO is the very steep learning curve of CCK development. (see above) 
> But I think that enough people are willing to contribute if CCK were a little 
> more accessible.

I don't think the learning curve for cck is steep. I figured most of it
out in about 2 days of poking around and actually writing code for it.
A little trial and error and reading the code in content.module goes a
long way. Yes, there is are additional abstraction layers for fields and
widgets, and you've got to figure out where it interfaces with the
normal node building and life-cycle code.

Angie has began an effort to document CCK.  Which I'll agree isn't an
easy task. It also changes underneath you on occasion without any sort
of notice, which can be challenging.

>  
> If we can agree on the abovementioned ideas, or provide better ones, then I 
> hope we can get this started with some patches and issues right away.

It would be nice if you can file bugs for where you see these
inconsistencies in the field modules. Maybe there is a reason for them
and if we can suss out the reasons for those work arounds we will
probably find something in cck that can be improved upon.

> Bèr

Thanks for bringing this stuff up Ber. JonBob has been in a
communications Bermuda triangle for a while. I'm not sure that he has
the bandwidth to document a lot of things currently.