On Tue, 23 Jan 2007, Earl Miles wrote:

The $file would be a file that should live in the same directory as the .module file and tells the system which file to conditionally load when that callback is invoked. This is one of the things that creates a big advantage in this system: We could move callbacks into conditionally loaded files, allowing Drupal's minimum per-page codesize to be smaller. The 'nodeapi' hook is not run for every page load, but because hooks are anonymous right now, that code *must* be present.

Well, I am increasingly adopting a similar coding practice. A few days ago, I also split up archive.module into a small stub which registers the menu item and has a page callback, and then that page callback includes archive.inc which contains all the magic to display an archive page. This does eliminate a great deal of parsing for PHP. Although we use an opcode cache here, we observed greatly reduced memory usage if we take care of doing our own modules this way. (I don't have hard numbers unfortunately). Gabor

+1 from me provided the performance gain is real. -- Sammy Spets Synerger Pty Ltd http://synerger.com On 23-Jan-07 14:21, Earl Miles wrote:

After some discussion this morning, I am putting up a proposal to eliminate hooks entirely (though not the concept of hooks) and replace them with officially registered callbacks. I have identified several areas where this will improve Drupal (primarily in terms of probable performance gains). There are 2 parts to this.

Part 1: drupal_register_callback

function drupal_register_callback($callback, $function = NULL, $file = NULL)

This function is only called from a particular location; ideally in its own separate .register file that only needs to be loaded when the callback registry is being rebuilt. However, it could also be a function within the .module file. There are some questions here, primarily balancing ease of use for the developer versus performance for the system. When it is called, the system should already know which module context is being used.

This function would register a given callback (examples: nodeapi, form_alter, etc) with the Drupal system. The registry would replace the logic module_implements uses to determine which module implements which hook.

The $function would be what's called. In today's world the function would be called mymodule_nodeapi; and it would likely not change, but we should allow registration to specify the function used. If left blank, the default function name would be constructed. (i.e, modulename_hookname).

The $file would be a file that should live in the same directory as the .module file and tells the system which file to conditionally load when that callback is invoked. This is one of the things that creates a big advantage in this system: We could move callbacks into conditionally loaded files, allowing Drupal's minimum per-page codesize to be smaller. The 'nodeapi' hook is not run for every page load, but because hooks are anonymous right now, that code *must* be present. When you can conditionally load, however, the required size of the .module becomes very small. (in fact you would only keep items in that file that are called on every or nearly every page load. Currently that's hook_init and hook_menu but with changes to hook_menu that one may well be eliminated from every-page-load calls, and the .module file itself may not even need to be loaded for every page).

Invoking a callback would still be the same: module_invoke('callback', arguments...);

Part 2: function drupal_register_theme($theme, $args, $function = NULL, $file = NULL)

This works very similarly to callbacks, but it registers a theming function that can be overridden by the theme. The only difference in is the $args argument, which is an array that tells the function how to translate function arguments into the $vars array, for use in .tpl.php files.

For example, let's say I have the following theme function:

function theme_foo($node, $bar, $baz);

The $args array would be: array('node', 'bar', 'baz').

Themes would also have a similar registry; we may need to make it slightly more complex so that it could work with the theme engine to automatically register a template. Which means we might want to find a way to make it simpler for themers to register the templates they're using (for example, a directory scan could find well-named template files and automatically register them based upon already known registered info).

This entire system (callbacks and functions) would be cached, so information would only need to be re-read when modules and theme settings are updated. A 'development' mode might be necessary that re-reads this information every page load. There would probably be some way for devel.module to expose this setting and automatically disable development mode when devel.module is enabled.

Overall benefits to this system:

1) Some execution time is saved looking up module hooks. I don't believe this is a great deal of execution time, but it is definitely some.

2) A great deal of code can be loaded conditionally, reducing Drupal's footprint. I believe this could be a *very* significant boost to Drupal and could vastly reduce the amount of code that is loaded, making Drupal far, far friendlier to hosts that cannot use an opcode cache.

3) Certain namespace collisions will be avoided. Developers will no longer be penalized for forgetting that mymodule_user() is the user hook.

4) Theme functions will no longer have to hunt around for their function; they will not have to test the file-system to see if templates exist. They will know exactly where to go when the time is right.

5) Theme template files will be easier to include in modules. In fact, if implemented properly, modules could actually include .tpl.php files, make the proper registration, and no theme code will be harmed by being in the .module file *at all*.

6) phptemplate.engine can have much of its code removed to core, making it easier to implement alternative template engines.

7) As an interesting side effect, themes could be made to implement hook_form_alter, if we coded it that way. There is a value to this, and also an argument why this might not be a good thing.

Overall cons:

1) Having to register a callback is less convenient than simply naming a function. It's even more inconvenient to have a .register file. However, using the existing naming convention, the module_builder.module *could* be made to scan a module and produce a .register file or a register function.

2) Having to register theme functions is similarly inconvenient but I think it's no worse than we have now.

3) Having to reregister upon making changes is very inconvenient; a development mode would be a must.

Thoughts?

A big -1 on eliminating the hook concept. A big +1 on registering and adding file concept. The way I see the future of hooks: a) hook_define_hooks, herein you can define hooks. b) on module enable time, Drupal scans your module for hooks and registers them for you -- this would merely be a cached version of module_implements. But, if you want, you can register for yourself. You could define whether you want to come before or after other module's implementation (see weights.php in my sandbox for an implementation of this -- THAT is prior art :) ) and now with Merlin's idea you could define files as well. Yes, files, it needs to be an array for what I believe, obvious reasons. This way if you do not want to bother with registering then you do not need to. If you want to then it benefits you. Split mode is woe because of references and because everything becomes call_user_func_array called -- cufa is not fast enough. Also, too many files need to be included. Let's task the humans with grouping their functions together into chunks. Regards, NK

Assorted thoughts in no special order: The biggest benefit, IMO, is being able to conditionally load code that on a production site is needed only rarely. Eg, hook_menu after chx's mega-patch goes in, most of the views definition hooks, etc. Those are only needed on cache rebuilding. Parsing them on every page load is a waste. At the same time, however, parsing and compiling PHP is not the slowest step in a page hit. The disk IO is. Every single PHP-Internals person I've talked to online or off has been very adamant that disk IO is the #1 killer of PHP applications, and simply reading the source file off disk is slower than compiling it. I vaguely recall an issue queue that was working on splitting out all theme functions to .tpl.php files many months ago that saw a noticeable performance hit when theme.inc was dissected into a bunch of files. How much compiling we'd have to remove to make up for the increased disk IO on those page loads that would require it is something only benchmarking will tell us. I'm not sure why we would need arbitrary files in which to put conditional functions, though. I'd think a logical grouping done by the developer would be better, as it forces us to consider when things will be used as well as saves some processing logic. For instance, .module for always-load, .cache for things that only need to be loaded to rebuild a cache (hook_menu2, hook_views_tables(), etc.), I'd even go as far as a .node for the node method/callback/pseudo-hook things. From an organizational standpoint I'd prefer having many files to one huge file, if it weren't for the performance hit. I also agree with chx that most hooks/callbacks/whatevers can and should be auto-derived. That reduces the amount of code needed in .module to define what's not in .module. :-) Theme functions couldn't be auto-detected because of the parameters to them being so variable, but I'm perfectly fine with that. In fact, hook_theme() in which one just calls drupal_register_theme() a bunch of times would be another perfect example to go in the "not always" file. I don't think themes themselves need to register what they override. That could be scanned and cached, too, on the theme admin page. A theme() call then just becomes a lookup to see which of the three locations, in order, a theme file exists. Just borrow some of chx's menu magic SQL to find the "right" template. As for development, a flag in devel module to clear the cache table on every page load, a more draconian form of cacheexclude[1], shouldn't be too difficult to add. All kinds spiffiness. :-) [1] http://drupal.org/project/cacheexclude On Tuesday 23 January 2007 4:21 pm, Earl Miles wrote:

After some discussion this morning, I am putting up a proposal to eliminate hooks entirely (though not the concept of hooks) and replace them with officially registered callbacks. I have identified several areas where this will improve Drupal (primarily in terms of probable performance gains). There are 2 parts to this.

Part 1: drupal_register_callback

function drupal_register_callback($callback, $function = NULL, $file = NULL)

This function is only called from a particular location; ideally in its own separate .register file that only needs to be loaded when the callback registry is being rebuilt. However, it could also be a function within the .module file. There are some questions here, primarily balancing ease of use for the developer versus performance for the system. When it is called, the system should already know which module context is being used.

This function would register a given callback (examples: nodeapi, form_alter, etc) with the Drupal system. The registry would replace the logic module_implements uses to determine which module implements which hook.

The $function would be what's called. In today's world the function would be called mymodule_nodeapi; and it would likely not change, but we should allow registration to specify the function used. If left blank, the default function name would be constructed. (i.e, modulename_hookname).

The $file would be a file that should live in the same directory as the .module file and tells the system which file to conditionally load when that callback is invoked. This is one of the things that creates a big advantage in this system: We could move callbacks into conditionally loaded files, allowing Drupal's minimum per-page codesize to be smaller. The 'nodeapi' hook is not run for every page load, but because hooks are anonymous right now, that code *must* be present. When you can conditionally load, however, the required size of the .module becomes very small. (in fact you would only keep items in that file that are called on every or nearly every page load. Currently that's hook_init and hook_menu but with changes to hook_menu that one may well be eliminated from every-page-load calls, and the .module file itself may not even need to be loaded for every page).

Invoking a callback would still be the same: module_invoke('callback', arguments...);

Part 2: function drupal_register_theme($theme, $args, $function = NULL, $file = NULL)

This works very similarly to callbacks, but it registers a theming function that can be overridden by the theme. The only difference in is the $args argument, which is an array that tells the function how to translate function arguments into the $vars array, for use in .tpl.php files.

For example, let's say I have the following theme function:

function theme_foo($node, $bar, $baz);

The $args array would be: array('node', 'bar', 'baz').

Themes would also have a similar registry; we may need to make it slightly more complex so that it could work with the theme engine to automatically register a template. Which means we might want to find a way to make it simpler for themers to register the templates they're using (for example, a directory scan could find well-named template files and automatically register them based upon already known registered info).

This entire system (callbacks and functions) would be cached, so information would only need to be re-read when modules and theme settings are updated. A 'development' mode might be necessary that re-reads this information every page load. There would probably be some way for devel.module to expose this setting and automatically disable development mode when devel.module is enabled.

Overall benefits to this system:

1) Some execution time is saved looking up module hooks. I don't believe this is a great deal of execution time, but it is definitely some.

2) A great deal of code can be loaded conditionally, reducing Drupal's footprint. I believe this could be a *very* significant boost to Drupal and could vastly reduce the amount of code that is loaded, making Drupal far, far friendlier to hosts that cannot use an opcode cache.

3) Certain namespace collisions will be avoided. Developers will no longer be penalized for forgetting that mymodule_user() is the user hook.

4) Theme functions will no longer have to hunt around for their function; they will not have to test the file-system to see if templates exist. They will know exactly where to go when the time is right.

5) Theme template files will be easier to include in modules. In fact, if implemented properly, modules could actually include .tpl.php files, make the proper registration, and no theme code will be harmed by being in the .module file *at all*.

6) phptemplate.engine can have much of its code removed to core, making it easier to implement alternative template engines.

7) As an interesting side effect, themes could be made to implement hook_form_alter, if we coded it that way. There is a value to this, and also an argument why this might not be a good thing.

Overall cons:

1) Having to register a callback is less convenient than simply naming a function. It's even more inconvenient to have a .register file. However, using the existing naming convention, the module_builder.module *could* be made to scan a module and produce a .register file or a register function.

2) Having to register theme functions is similarly inconvenient but I think it's no worse than we have now.

3) Having to reregister upon making changes is very inconvenient; a development mode would be a must.

Thoughts?

-- Larry Garfield AIM: LOLG42 larry@garfieldtech.com ICQ: 6817012 "If nature has made any one thing less susceptible than all others of exclusive property, it is the action of the thinking power called an idea, which an individual may exclusively possess as long as he keeps it to himself; but the moment it is divulged, it forces itself into the possession of every one, and the receiver cannot dispossess himself of it." -- Thomas Jefferson

On 23-Jan-07, at 5:21 PM, Earl Miles wrote:

Part 1: drupal_register_callback

My favourite part of this technique is that it should make Drupal much more "deterministic" (?)... You can check your registered callbacks and have a much clearer idea of what will happen on any given function call instead of having to search through every single module and inc file to find something that looks similar to an invoke_all hook call. And this maps well to the "event listener" type thing used on the client site. Careful with all the .something files or you'll turn Drupal into Helma ;) While Adrian works harder at turning it into Lisp or Smalltak . hehe. -Rowan

Hi everyone, I'm all for the use of some hook caching. I totally agree with chx's proposal that all Drupal hooks get automatically scanned for when modules are installed. Similar to the menu system. It would be handy if a page was presented allowing an admin to fiddle with the ordering (i'm sure chx is working on this). In addition, it would be fantastic to allow modules to perform automatic hook registration on its sub-modules. For example, E-commerce would need this for its hooks. +1 for registration of hooks with an automatic registration for known hooks. -1 for getting rid of the hook concept (it's the same thing as callbacks!) -- Sammy Spets Synerger Pty Ltd http://synerger.com On 23-Jan-07 14:21, Earl Miles wrote:

After some discussion this morning, I am putting up a proposal to eliminate hooks entirely (though not the concept of hooks) and replace them with officially registered callbacks. I have identified several areas where this will improve Drupal (primarily in terms of probable performance gains). There are 2 parts to this.

Part 1: drupal_register_callback

function drupal_register_callback($callback, $function = NULL, $file = NULL)

This function is only called from a particular location; ideally in its own separate .register file that only needs to be loaded when the callback registry is being rebuilt. However, it could also be a function within the .module file. There are some questions here, primarily balancing ease of use for the developer versus performance for the system. When it is called, the system should already know which module context is being used.

This function would register a given callback (examples: nodeapi, form_alter, etc) with the Drupal system. The registry would replace the logic module_implements uses to determine which module implements which hook.

The $function would be what's called. In today's world the function would be called mymodule_nodeapi; and it would likely not change, but we should allow registration to specify the function used. If left blank, the default function name would be constructed. (i.e, modulename_hookname).

The $file would be a file that should live in the same directory as the .module file and tells the system which file to conditionally load when that callback is invoked. This is one of the things that creates a big advantage in this system: We could move callbacks into conditionally loaded files, allowing Drupal's minimum per-page codesize to be smaller. The 'nodeapi' hook is not run for every page load, but because hooks are anonymous right now, that code *must* be present. When you can conditionally load, however, the required size of the .module becomes very small. (in fact you would only keep items in that file that are called on every or nearly every page load. Currently that's hook_init and hook_menu but with changes to hook_menu that one may well be eliminated from every-page-load calls, and the .module file itself may not even need to be loaded for every page).

Invoking a callback would still be the same: module_invoke('callback', arguments...);

Part 2: function drupal_register_theme($theme, $args, $function = NULL, $file = NULL)

This works very similarly to callbacks, but it registers a theming function that can be overridden by the theme. The only difference in is the $args argument, which is an array that tells the function how to translate function arguments into the $vars array, for use in .tpl.php files.

For example, let's say I have the following theme function:

function theme_foo($node, $bar, $baz);

The $args array would be: array('node', 'bar', 'baz').

Themes would also have a similar registry; we may need to make it slightly more complex so that it could work with the theme engine to automatically register a template. Which means we might want to find a way to make it simpler for themers to register the templates they're using (for example, a directory scan could find well-named template files and automatically register them based upon already known registered info).

This entire system (callbacks and functions) would be cached, so information would only need to be re-read when modules and theme settings are updated. A 'development' mode might be necessary that re-reads this information every page load. There would probably be some way for devel.module to expose this setting and automatically disable development mode when devel.module is enabled.

Overall benefits to this system:

1) Some execution time is saved looking up module hooks. I don't believe this is a great deal of execution time, but it is definitely some.

2) A great deal of code can be loaded conditionally, reducing Drupal's footprint. I believe this could be a *very* significant boost to Drupal and could vastly reduce the amount of code that is loaded, making Drupal far, far friendlier to hosts that cannot use an opcode cache.

3) Certain namespace collisions will be avoided. Developers will no longer be penalized for forgetting that mymodule_user() is the user hook.

4) Theme functions will no longer have to hunt around for their function; they will not have to test the file-system to see if templates exist. They will know exactly where to go when the time is right.

5) Theme template files will be easier to include in modules. In fact, if implemented properly, modules could actually include .tpl.php files, make the proper registration, and no theme code will be harmed by being in the .module file *at all*.

6) phptemplate.engine can have much of its code removed to core, making it easier to implement alternative template engines.

7) As an interesting side effect, themes could be made to implement hook_form_alter, if we coded it that way. There is a value to this, and also an argument why this might not be a good thing.

Overall cons:

1) Having to register a callback is less convenient than simply naming a function. It's even more inconvenient to have a .register file. However, using the existing naming convention, the module_builder.module *could* be made to scan a module and produce a .register file or a register function.

2) Having to register theme functions is similarly inconvenient but I think it's no worse than we have now.

3) Having to reregister upon making changes is very inconvenient; a development mode would be a must.

Thoughts?