[development] CCK field generation confusion (and how to solve)

[Bèr Kessels]

Op dinsdag 10 oktober 2006 23:31, schreef Darrel O'Pry:
> On Tue, 2006-10-10 at 16:52 +0200, Bèr Kessels wrote:
> > Hi,
> >
> > I have been fighting CCK fields for the last two days again. My general
> > conclusion already was that it is a little too hard to grok, so let us
> > get that solved.
>
> Agreed.
>
> > Why?
> > Because it is too hard to grok. Looking closer at the field that are
> > there, you will notice that the developers were:
> >  a) inconsistent. Very inconsistent. Every module with fields is
> > completely different.
>
>  Can you point out some specific examples. hook_field provides the data
> definitions, validation, and crud necessary stuff.

Yup. datefields do a lot of data-refactoring inside the widget parts. Your 
imagefields do most of the validation inside the 'prepare' op of the widget 
(prepar, is mean to, yup, prepare the data from the DB for the display in a 
form', never for validation of files, IMO). 
My point was, that I see nearly all the modules with fields and widgets hae 
completely different code to:
 * handle multiple fields
 * manage / share data beween widgets and fields
And on top of that, they are all different when it comes to calling functions 
etc. One module has many lines of code inside the hook_field, another one 
calls all kinds of private functions in that hook_field. All these flavours 
of coding add to the confusion.

>   hook_widget is for building user interfaces. ie) select lists, ajaxy
> things, flash movies and preparing the results for validation.
>
> >  b) using fields where they should have used widgets. And using widgets
> > where they should have used fields.
>
>   Do you know the distinction? You've already expressed that you don't
> understand it, so maybe you should take the time to grok what is there
> now before making judgments on what is right or wrong.

Yes, I know what
a) The distinction should be (you call it Utopian) and b) it is now. Both are 
different and leave a large grey ara. That area is so large, and so grey, 
that right now, not one CCK module actually needs both to be different. Every 
CCK module defines both widgets and fields and nearly all of them are of form 
one-field-needs-one-widget. 
As it stands now, as modules use it, that distinction would not be needed. 

> >  c) seem to be overly confused about the 'multiple' thing.
>   Multiple... a field can be a single value or multiple value field.
> What's confusing about it? I admit its complex and often difficult to
> deal with. I

Sure. But selectlist module abuses multiple to turn radios into checkboxes, 
for example. 
Other modules simply ignore the multiple values, and the 'better' modules need 
one if and one foreach on *each* op in fields. That is a LOT of complexity 
for little benefit. 
In the database architecture this is used, which is cool, but the useage (for 
developers) of the multiple thing is still rather hard to grok. 

>   Yes the hook system is deep, but you can't really treat the field
> modules as full drupal modules. They should only implement module hooks
> necessary.

I don't really care what we treat what module as. :) flexinode has its 
private .inc system, and it takes 20 minutes of copy-pasting to make new 
fields. The architecture might suck, the performance may be horific (I am 
curious about benchmarks CCK vs Flexinode, I have a gut feeling it wont 
differ that much), but making a field is a breeze!

>   Since content.module (soon to be field.module??) Handles delegating
> tasks via hook content so it can aggressively cache the field modules
> output so these deep calls should not occur too often. As we move closer
> to FAPI3/APIAPI the field modules should simpler. They can begin
> becoming UI's to create field_instances and help build the data model
> instead of doing the heavy lifting.

I think we should not lean too hard on FAPIAPAPAPIAPI, but rather make CCK 
simpler by itself :). Off course this will mean using more existing 
Drupalisms, but the main task is to focus on simplicity IMO, which I doubt 
that APIAPI will bring. 
(yes, I am rather sceptical about APIAPI, since I am not overly enthusiastic 
about all the 'good' FAPI brought us. but thats a different issue, just to 
put the statement above in perspective).

> > Call by reference:
> > Either all of the stuff should be call by ref in all hooks at all points.
> > Or none. Not have $node as called by ref in some $ops (load) but not in
> > others (insert).
>
> Depends on the expected return of the hook, but yeah I don't like teh
> api hooks that much... I like simple direct hook_field_insert instead of
> hook_field(op == insert). I also think code is more understandable when
> broken into the little chunks like that. (FAPI is taking us this
> direction I believe).

Well, imagefield has bugs because it assumes stuff is called by ref, which its 
not. We already discussed this, and indeed CCK HEAD has this better sorted 
then 4.7, still its overly confusing. You need to trace back stuff with 
dprints sometimes five hooks deep, to find whether something should have been 
returned, or called-by-ref. This is unneeded complexity. 

I think we can start from here, with simplification. I will add some issues in 
the next days, but I ned to give it some more thought. Plan is to split out 
hook_field $op into hook_field_op (e.g. hook_field_insert). 
That way we have less functions, doing more dedicated jobs and erceiving 
clearer arguments.

> > Hooks vs FAPI.
> > Because the general Drupal-way seems to be towards _altering of FAPI
> > forms, instead of the 'old' hook mechanism, CCK uses many hooks. To alter
> > forms. But not everywhere. These CCK hooks are well documented already,
> > but when it mixes with form alters, you get a messy spaghetti-bowl again.
> > I'd say: either _alters all over the place, or hooks, but never both.
> > Hence I suggest to drop the hook_*() $op == 'form' in favour of
> > form_alters all trough CCK.
>
> Then CCK can't cache aggregate output. Which would mean going through
> those deep layers of callbacks to all the field modules for each
> field_instance associated to the content type...

Not in its current form, but I am sure with some creativity we can still 
cache, even if we use form_alters. 

> I don't think widgets should be separate from fields. In a Utopian
> world, User Interface would not be bound at all by the data structures
> they interface with...  This isn't so Utopian. Part of the widget's job
> is formatting data for the underlying data model. I'm not sure that the
> two can be cleanly separated. If you got a good idea, write the patch
> I'll be happy to review it.

Just because we cannot reach a utopia does not mean we cannot try to get 
closer. 
"part of the widgets job" is to prepare stuff for FAPI. But certainly not to 
set/unset file SESSION data, just to name something :)
If we make fields a tad more powerfull, then they can eat anythign that FAPI 
feeds it. FRom that point onwards, widgets are no longer something from CCK, 
then widgets are a FAPI thing. And then widgets can be re-used all trough 
drupal. See e.g. helpers module where I made a country_select() widget to 
render a selectfield to select a country. Everyone can use that "widget". 
A CCK field would need to be able to eat what country_select(), trough FAPI, 
feeds it. It will strip at least 60% of the complexity out of CCK.

> Validation and input preparation are kind of grey area to some people
> and they have differing opinions on where it goes... I personally
> believe the widget should handle generating the UI and preparing the
> input for the data model. The data model should handle its validation
> and crud work.

Well, right now, as mentioned above, the widget provides the interface, part 
of the data-manipulation, then FAPI handles the data and passes it on to the 
field. 
I think we can safely cut the "part of the data-manipulation" out of that, and 
leave that to FAPI and fields. 

> > Who?
> > There are a lot of users, but very few developers for CCK. The main
> > reason, IMO is the very steep learning curve of CCK development. (see
> > above) But I think that enough people are willing to contribute if CCK
> > were a little more accessible.
>
> I don't think the learning curve for cck is steep. I figured most of it
> out in about 2 days of poking around and actually writing code for it.
> A little trial and error and reading the code in content.module goes a
> long way. Yes, there is are additional abstraction layers for fields and
> widgets, and you've got to figure out where it interfaces with the
> normal node building and life-cycle code.

I spent two weeks developing taxonomy-fields. And only when I drew out a 
complete ERD on a whiteboard, did I start to grok the concepts of CCK. But 
even then, no-one can be certain that he/she really grokked it. Even when a 
module appears to work from the outside, that is not a proof that the author 
really grokked CCK :), there are many roads to Rome, and apparently for CCK 
no-one found the Right road, yet.
I am not exactly a good developer, but it took me far less time to grok 
flexinode, for example. I think 2 days is only realistic for the good 
developers amoungst us. Joe Developer will have his time measured in months 
to develop a field, I fear.

> > If we can agree on the abovementioned ideas, or provide better ones, then
> > I hope we can get this started with some patches and issues right away.
>
> It would be nice if you can file bugs for where you see these
> inconsistencies in the field modules. 

I've started this. But its really a huge challenge to make patches for a 
module that is complex, runs on an overly complex foundation, and wich lacks 
any documentation. Look, I consider the imagefields session manipulation in 
the widgets $ op == prepare a bug. You mention it above as though you did 
that on purpose. As long as such complex things aren't agreed upon, then 
nothing is a really a bug :).

> Maybe there is a reason for them
> and if we can suss out the reasons for those work arounds we will
> probably find something in cck that can be improved upon.

Please note, that I have nothing against any of the mentioned modules. I only 
mention them, to illustrate the result of the complexity of CCK. ot because i 
dislike them or anything. 

Bèr
-- 
[ End-user Drupal services and hosting | Sympal.nl ]
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 191 bytes
Desc: not available
Url : http://lists.drupal.org/pipermail/development/attachments/20061011/c2e579f2/attachment-0001.pgp