Re: [development] Freezing the Drupal API
On 5/15/06, Richard Archer <drupal.org@juggernaut.com.au> wrote:
Recent discussions on the Consultants list has raised the issue of the cost of doing business using Drupal, notably the high cost of upgrading existing installations due to the ever-changing nature of Drupal's API.
I wonder if there would be any interest in forming a group to tackle this by identifying where the current API has potential for improvement and perhaps even writing some code!
My aim would be that as of Drupal 5.0 with CCK, the published API would be treated as "stable" and changes to it would be resisted strongly.
IMHO, if this could be achieved it would give Drupal a much more mature look and make it more attractive to large users.
It would also increase the manpower available to work on future Drupal development because right now extra resources are being wasted maintaining the 4.5 and 4.6 trees, simply because the API changes between releases makes it impractical for many users to upgrade to 4.7.
The shifting API from release to release is a pain, yes, but the benefits of that outweigh the drawbacks. The core developers innovate as they see fit, and do not let the API freeze limit their creativity, the features to be implemented, or the overall flexibility, power and compactness of Drupal's core. Perhaps it is best to spend energy on migration paths (e.g. Flexinode to CCK, forms updater, ...etc.) rather than freeze the API early. Let the creative juices flow ...
At 5:31 PM -0400 15/5/06, Khalid B wrote:
The core developers innovate as they see fit, and do not let the API freeze limit their creativity, the features to be implemented, or the overall flexibility, power and compactness of Drupal's core.
I know. This, along with "code is gold" is the Drupal mantra. And I would argue that this attitude has to change. It is a sign that Drupal is immature, without focus and not much more than a plaything for the core developers. This "moving target" API model makes Drupal frustrating for someone building a site which they want to run in the long term. And as for trying to build a business around Drupal... well, there's surely a lot of money in helping people upgrade their installations. But personally I want to be able to offer a reliable, up-to-date, feature-rich hosting environment, and all Drupal promises me is a world of hurt. I would also argue that the lack of a robust API has a detrimental effect on the quality of the code in Drupal. For a simple example, trace through the code that displays a menu item and see how much code is executed to perform this task! I think this is a good example of where "code is gold" and not having a well-defined API to work within has led code which is complex, convoluted and inefficient.
Perhaps it is best to spend energy on migration paths (e.g. Flexinode to CCK, forms updater, ...etc.) rather than freeze the API early.
I'm not talking about freezing the API early. I mentioned that my goal would be to have a stable API for Drupal 5.0 with CCK. And upgrade paths don't help people upgrade their custom modules. I think this is where the most pain is felt with upgrades. I realise there will always be major new features which require changes to the API. But what I want to do is work to create a robust API that will allow incremental changes to be made behind the scenes without impacting so heavily on both contrib and custom modules. ...R.
Richard Archer wrote:
At 5:31 PM -0400 15/5/06, Khalid B wrote:
The core developers innovate as they see fit, and do not let the API freeze limit their creativity, the features to be implemented, or the overall flexibility, power and compactness of Drupal's core.
I know. This, along with "code is gold" is the Drupal mantra. And I would argue that this attitude has to change. It is a sign that Drupal is immature, without focus and not much more than a plaything for the core developers.
This "moving target" API model makes Drupal frustrating for someone building a site which they want to run in the long term.
And as for trying to build a business around Drupal... well, there's surely a lot of money in helping people upgrade their installations.
If you want to imply that I am pro "breaking APIs where neccessary" because I want to make money from updating people's sites you are quite mistaken. I only upgrade sites for clients that already were my clients before. I'd turn down requests for "please update my site to 4.7" right away, because there isn't that much money to be made and the money/paperwork as well as fun/work ratio isn't high enough. And curiously I don't get any such requests, so it can't be that good a business.
But personally I want to be able to offer a reliable, up-to-date, feature-rich hosting environment, and all Drupal promises me is a world of hurt.
I haven't heard Boris, James, or Adrian cry very loudly recently. Maybe it is more a quiet sobbing. Cheers, Gerhard
On 5/15/06, Gerhard Killesreiter <gerhard@killesreiter.de> wrote:
And as for trying to build a business around Drupal... well,
there's surely a lot of money in helping people upgrade their installations.
Pando.com just got funded for $7M...parts of the website run on Drupal. NowPublic.com, which has gone through some pains over the past 2 years, just got a round of funding. There are others out there.
But personally I want to be able to offer a
reliable, up-to-date, feature-rich hosting environment, and all Drupal promises me is a world of hurt.
I haven't heard Boris, James, or Adrian cry very loudly recently. Maybe it is more a quiet sobbing.
*sobs* ...but it has nothing to do with Drupal. Drupal is the *easy* part of hosting. Well, maybe not the easiest. As James stated elsewhere, we run backports with matching patches in the forward version only -- no forking, but continual improvement. 4.5 was painful (Bryght helped finalize multisite and ran multisite-compatable modules). 4.6 was/is a bit less painful (multisite standard, maintain .install files ourselves). 4.7 is going to be great (install profiles in beta plus dependencies, form API for fantastic theme/module level modifications). -- Boris Mann Vancouver 778-896-2747 San Francisco 415-367-3595 Skype borismann http://www.bryght.com
Op dinsdag 16 mei 2006 03:24, schreef Richard Archer:
And as for trying to build a business around Drupal... well, there's surely a lot of money in helping people upgrade their installations. But personally I want to be able to offer a reliable, up-to-date, feature-rich hosting environment, and all Drupal promises me is a world of hurt.
My idea is that this is a very true fact, indeed. And because it is all OSS, it acts like water: it will find its way trough. If you, just like me, need 4.6 (and for me there are a large range of reasons not upgrading yet, if ever) there are surely more people. And if there are more, with the same needs and wishes, then a group will form around 4.6 (or .7) Maybe to maintain some backported features and/or bugfixes. Maybe (but preferably not) to maintain a future fork with these backported features and/or bugfixes. If there are enough people at hand, I am sure Dries et al will not be unwilling to continue minmal development (bug/sec. fixes) on the 4.6 branch. If there is enough need, maybe even for years, who knows. I am sure that drupal 4.6 will be used for years. One thing I have been playing with, is a module that acts like wrappers around the form api. So that we can (back again) call form_select(). For those who feel unconfy with the Fapi (like me). Or those not willing/able to invest a lot of time to learn Fapi (like me). The actual plan/implemenation set aside: if there is a need for some "old fashioned bakwards, and in the eyes of a lot of developers plain stupid" concepts. It will make it. My point therefore is: Lets please leave the current development as it goes. Some people need this progress. And let us also look at some other needs besides getting more and newer APIs. Because some people do not need the progress, but need stability. But let us also speak up (as Richard did very well) about your own needs. Because if you do so, others with the same needs might think "hey I am not alone". And a team is formed ;). If you do not speak up, you will not be heard, its as easy as that. Bèr
One thing I have been playing with, is a module that acts like wrappers around the form api. So that we can (back again) call form_select(). For those who feel unconfy with the Fapi (like me). Or those not willing/able to invest a lot of time to learn Fapi (like me). The actual plan/implemenation set aside: if there is a need for some "old fashioned bakwards, and in the eyes of a lot of developers plain stupid" concepts. It will make it.
The initial plans was to keep the old form_*() functions around for one release to help in transition. However, this was scrapped. I don't remember the details. Also not that there is a backport of FAPI for 4.6 that nedjo et. al. did. This would help the other way (4.7 modules can easily work on 4.6 with minimal changes). Having said all that, the basics of FAPI (form handling) are not hard at all to learn, specially for someone like you Ber. It is a mental barrier more than anything else. Use formupdater and it will do a lot of the grudge work for you. Most of my modules are converted now to 4.7, and much of that was done by a relative newcomer to Drupal (my partner). The advanced uses of FAPI are another matter though ...
Khalid B wrote:
One thing I have been playing with, is a module that acts like wrappers around the form api. So that we can (back again) call form_select(). For those who feel unconfy with the Fapi (like me). Or those not willing/able to invest a lot of time to learn Fapi (like me). The actual plan/implemenation set aside: if there is a need for some "old fashioned bakwards, and in the eyes of a lot of developers plain stupid" concepts. It will make it.
....
Use formupdater and it will do a lot of the grudge work for you. Most of my modules are converted now to 4.7, and much of that was done by a relative newcomer to Drupal (my partner).
The advanced uses of FAPI are another matter though ...
As the resident annoying newbie. I echo that. My impression of both systems, before I'd used either, was they both solved my problem - my hatred of coding forms in HTML. If anything, FAPI *looked* easier for a newcomer. My first Drupal patch was a FAPI upgrade with the help of formupdater. That was with less than a year programming web.
Op dinsdag 16 mei 2006 04:47, schreef Khalid B:
Having said all that, the basics of FAPI (form handling) are not hard at all to learn, specially for someone like you Ber. It is a mental barrier more than anything else.
Oh I know! I was playing devils advocate more then anything else :), Ive had my share of forms api, developed with it, was one of the first to find where its limits lie (node forms w taxonomy cannot be altered properly) and I even build a Fapi builder for KDE. Having said that, stuff like form alter. Theming of forms and file forms is not what I call fun to do. Not saying that it used to be better or simpler. Just saying that this stuff has hardly improved. But lets leave this issue in rest. Too much has been said about Fapi being good or not. My point of view is clear. As is most of other peoples. I am not a fan of working with it. Bèr
Bèr Kessels wrote:
Op dinsdag 16 mei 2006 04:47, schreef Khalid B:
Having said all that, the basics of FAPI (form handling) are not hard at all to learn, specially for someone like you Ber. It is a mental barrier more than anything else.
Oh I know! I was playing devils advocate more then anything else :), Ive had my share of forms api, developed with it, was one of the first to find where its limits lie (node forms w taxonomy cannot be altered properly) and I even build a Fapi builder for KDE.
Having said that, stuff like form alter. Theming of forms and file forms is not what I call fun to do. Not saying that it used to be better or simpler. Just saying that this stuff has hardly improved.
But lets leave this issue in rest. Too much has been said about Fapi being good or not. My point of view is clear. As is most of other peoples. I am not a fan of working with it.
The important point about fapi is the following[1]: It lets you modify any form including validation and submit function. This is a great step forward for Drupal, because it is now much more flexible and extensible. Which - btw - is an important point for me as a consultant in order to be able to "sell" Drupal. Just yesterday I had to make a modification to a module that I have developed for a client where he required me to check values on the profile field form for consistency. I was able to do that for the 4.7 version, but not for the 4.6 version. Sure, the code to do that doesn't look _really_ pretty and is a bit hard to grok, but at least it works and I don't have to patch a core file. Cheers, Gerhard [1] Of course the real reason for committing it that late was security.
Sure, the code to do that doesn't look _really_ pretty and is a bit hard to grok, but at least it works and I don't have to patch a core file.
This is important. Earlier in the thread, Dries asked if we came to Drupal because it offered complete API support for every possible action, or because it offered clean, easy-to-read code. I jumped on board the Drupal train for neither of those reasons. :-) I'm here because Drupal allows me to accomplish more WITHOUT MODIFYING THE CORE FILES than any other OSS web-app framework I've worked with. It's been noted that upgrading from one version of Drupal to another can be tough. Having lived through epic migrations of PHPBB, PostNuke, and Xoops, I can say that the BEST defense against painful upgrades is isolating 'hacks' into modules. If those modules have to be reworked when new versions come out, and the APIs they use shift, it's a pain. But it's nowhere NEAR as much a pain as walking your way through a gigantic .diff, trying to figure out what hacked portions of node.module you need to migrate over when 4.7 goes to 4.7.1. --Jeff
On 16 May 2006, at 04:16, Bèr Kessels wrote:
If there are enough people at hand, I am sure Dries et al will not be unwilling to continue minmal development (bug/sec. fixes) on the 4.6 branch. If there is enough need, maybe even for years, who knows. I am sure that drupal 4.6 will be used for years.
Yes, and I stated this before. If there is enough interest in any of the older branches, it will be maintained. I'm still looking for a dedicated Drupal 4.6 maintainer. Send me a mail in _private_ if you think you are up for the job. -- Dries Buytaert :: http://www.buytaert.net/
I agree with what Richard is saying in a way. Using Drupal for a commercial client doesn't provide that client with a guaranteed hassle-free future of upgrades. The two reasons I see for this are: 1. Custom modules require the creator to provide the upgrade path 2. Theme system changes and UI changes may require site changes I don't see how API changes affect the client directly. I only see those changes affecting the client via modules. If a client uses modules (unmodified) provided as part of core only they are covered for upgrades. Add to this modules maintained by commercially supported developers there is little for them to worry about. If they use custom modules or those maintained by part-time/transient developers, they are in trouble. This is exactly what CivicSpace have addressed. They are responsible for the upgrade path. It is true that a client will always require some amount of custom modules. In that respect, you can provide them with the source for those modules and they can pay anyone to perform the upgrade for them. If you choose not to provide source, then it's time to negotiate a maintenance contract. What do I think about freezing the API? I believe it's a bad idea. As others have said, the freezing will prevent innovation. Innovation is key to providing a timely, secure and valuable product. You'd want your clients to have that type of product wouldn't you? Imagine where we would be now (for Winblowz users) if Microsoft hadn't developed Windows 95. We'd still be using a crappy OS. The transition from 3.11 to 95 was a very scary one for businesses, but it happened and the world has never looked back. How do I believe we produce a commercially viable distro of Drupal? Write modules properly. Many modules are written to address a small subset of the functionality required globally. Improving modules to address 80%+ of the functionality required, and maintaining them properly, will provide the upgrade paths for commercial clients to depend on. I'm currently working to do just this. I'd like more people working towards that same goal. Drupal being a open source (read as volunteer) project makes it impossible to expect this to happen. As others have already said, find likeminded people and form a team. Here is my shoutout! :) With my methodology above the core developers can go on their merry way to keep their goals met. Their goals differ from mine. Provided I respect their goals and speak to them nicely (and perhaps submit a few patches here and there to either fix bugs and/or make my goals possible) everyone wins. -- Sammy Spets Synerger Pty Ltd http://www.synerger.com/ On 16-May-06 11:24, Richard Archer wrote:
At 5:31 PM -0400 15/5/06, Khalid B wrote:
The core developers innovate as they see fit, and do not let the API freeze limit their creativity, the features to be implemented, or the overall flexibility, power and compactness of Drupal's core.
I know. This, along with "code is gold" is the Drupal mantra. And I would argue that this attitude has to change. It is a sign that Drupal is immature, without focus and not much more than a plaything for the core developers.
This "moving target" API model makes Drupal frustrating for someone building a site which they want to run in the long term.
And as for trying to build a business around Drupal... well, there's surely a lot of money in helping people upgrade their installations. But personally I want to be able to offer a reliable, up-to-date, feature-rich hosting environment, and all Drupal promises me is a world of hurt.
I would also argue that the lack of a robust API has a detrimental effect on the quality of the code in Drupal. For a simple example, trace through the code that displays a menu item and see how much code is executed to perform this task! I think this is a good example of where "code is gold" and not having a well-defined API to work within has led code which is complex, convoluted and inefficient.
Perhaps it is best to spend energy on migration paths (e.g. Flexinode to CCK, forms updater, ...etc.) rather than freeze the API early.
I'm not talking about freezing the API early. I mentioned that my goal would be to have a stable API for Drupal 5.0 with CCK.
And upgrade paths don't help people upgrade their custom modules. I think this is where the most pain is felt with upgrades.
I realise there will always be major new features which require changes to the API. But what I want to do is work to create a robust API that will allow incremental changes to be made behind the scenes without impacting so heavily on both contrib and custom modules.
...R.
On 16 May 2006, at 3:24 AM, Richard Archer wrote:
I would also argue that the lack of a robust API has a detrimental effect on the quality of the code in Drupal. For a simple example, trace through the code that displays a menu item and see how much code is executed to perform this task! I think this is a good example of where "code is gold" and not having a well-defined API to work within has led code which is complex, convoluted and inefficient.
I'm going to step in here and post what I hope to be possible with FAPI 2.0 Fapi is still remarkably badly named, because it's not really what it's about. In FAPI 2.0, you define the form fields, separately from the form elements (ie: widgets). You can have multiple forms, or output view, based on the same set of fields (ie: different views of the same model). Additionally, you get multiple actions that can be executed on that same model (ie: the controller, or the _submit functionality). Due to the separation of the forms / actions from the page model, we will no longer have to generate the entire page, and then do the form_builder process to extract the field definitions. All models, views and controllers will have be accessible through a path (ie: the user login form can be retrieved by doing drupal_get_form('user/login'); , without having to create an array and push it through the entire process. It will just pull what it needs, when it needs it. It's also an implmentation of the Command Pattern (http:// en.wikipedia.org/wiki/Command_pattern), but where it gets really interesting is the following. Every thing that drupal can do via a form, drupal can do through a standard programmable interface. Take the following for example : $request = drupal_get_request('node/add/event'); $request['title'] => 'something'; $request['body'] => 'something'; $result = drupal_send_request('node/add/event', $request); What this does, is it gets the correct model(s) for that specific action, then it allows you to populate the variables, however you want. And then when you send the request, it goes back through the original form _validate, and finally hits the original _submit. This can be done for any form, that you have sufficient access to, and each of these actions can then be publicly exported via ReST, xmlrpc, soap, you name it. What it essentially does is create a standard API for everything to use, and for that reason I've been joking with calling it the API API or API ^ 2.0 -- Adrian Rossouw Drupal developer and Bryght Guy http://drupal.org | http://bryght.com
On Tue, 2006-05-16 at 11:24 +1000, Richard Archer wrote:
At 5:31 PM -0400 15/5/06, Khalid B wrote:
The core developers innovate as they see fit, and do not let the API freeze limit their creativity, the features to be implemented, or the overall flexibility, power and compactness of Drupal's core.
I know. This, along with "code is gold" is the Drupal mantra. And I would argue that this attitude has to change. It is a sign that Drupal is immature, without focus and not much more than a plaything for the core developers.
This "moving target" API model makes Drupal frustrating for someone building a site which they want to run in the long term.
The API is stable for a released version. Build on your version. By the time the next version is released the upgrade tools will be even better than 4.6 -> 4.7. Keep your modules up to date. Those of us who like to work on core will make sure you data makes it through the upgrade. Some people do a lot of work to make the moving API possible. It is stated drupal don't freeze. And some of us do plan our Proof of Concepts. I will say all of this is moot and useless. Anyone who has explored thoroughly CCK/Views/Workflows/Actions knows there is absolutely no competition in the market. I think this kind of innovation comes from the 'do as thou wilt, shall be the whole of the law' plan to drupal's development. If you get what you did past the gate keepers... one of which is that angsty curmudgeon killes which can be as difficult as the passage between charbydis and scylla.
And as for trying to build a business around Drupal... well, there's surely a lot of money in helping people upgrade their installations. But personally I want to be able to offer a reliable, up-to-date, feature-rich hosting environment, and all Drupal promises me is a world of hurt.
There are also a lot of people who will deal with the upgrade issues for you. If you're having problems with personal hacks and modifications I suggest you explore doing your hacks in you theme's or in some sort of site specific module, at the very least maintain a patchset, or go to full on revision control and keep your core drupal and contrib modules as vendor branches and merge where necessary. Merges are not that painful as long as you know drupal.
I would also argue that the lack of a robust API has a detrimental effect on the quality of the code in Drupal. For a simple example, trace through the code that displays a menu item and see how much code is executed to perform this task! I think this is a good example of where "code is gold" and not having a well-defined API to work within has led code which is complex, convoluted and inefficient.
Well no, drupal is a growing project and is experiencing iterative development... wittgenstein's ladder comes to mind. Someone came up with a good functional idea... the menu system. It got improved a while back, then you came along and made it better. It is much easier to improve upon existing things, than create new ones... So yeah code is gold, and improvements are better.
Perhaps it is best to spend energy on migration paths (e.g. Flexinode to CCK, forms updater, ...etc.) rather than freeze the API early.
I'm not talking about freezing the API early. I mentioned that my goal would be to have a stable API for Drupal 5.0 with CCK.
And upgrade paths don't help people upgrade their custom modules. I think this is where the most pain is felt with upgrades.
I realise there will always be major new features which require changes to the API. But what I want to do is work to create a robust API that will allow incremental changes to be made behind the scenes without impacting so heavily on both contrib and custom modules.
...R.
Richard... I don't know that many core function calls have changed with the exception of the formAPI... I'm sure diffing 4.6 and 4.7 | grep ^function will reveal just how little of the actual API changed. At that there isn't one 'DrupalAPI'.. There are like 7 or 8 apis... and they are all being improved independently. ok thats my morning rant...
participants (11)
-
Adrian Rossouw -
Boris Mann -
Bèr Kessels -
Darrel O'Pry -
Dries Buytaert -
Gerhard Killesreiter -
Jeff Eaton -
Khalid B -
Richard Archer -
Sammy Spets -
sime