This is my first attempt at writing something like this, to me it's not as much about creating a useful DEP out of this, as it is getting a working format going. I apologise for having left this so long, it's been in my drafts for week now. I am just sending the incomplete document to see where we can go from. If you have any other suggestions, please make them. I'd like to keep the format as simple as possible. Title : Cascading variable system. Abstract: This DEP defines a method to allow for more flexibility in the way we handle variables. Author: Adrian Rossouw Dependencies: None Repository: Core Status: Suggestion? 1. Introduction : This DEP specifies a way to clear up and centralise the variable system in Drupal, and hopefully abstracts some code that we are often using for specific cases. 2. Motivation : Variables in Drupal ( records in the variable table ), are primarily used to store settings for the site. Frequently however, we have the need to override the specified variable. Two such cases are a. variables that can be overridden for each user (the theme setting, for instance) b. variables that can be overridden for each theme (such as whether or not to display the site title). This is done to allow us to specify defaults for the site, and then allow these defaults to be altered. In the case of (a.), we manually test for the overridden value with each occurence of the variable, and in the case of (b.) we have centralised all the theme variable changes by introducing the theme_get_variable() function. This has the benefit of allowing us to define even more layers of variables, such as .. blog specific settings, node specific settings, locale specific settings (allowing us to localise the variables like site_name etc.) and even install specific settings (so install profiles can provide default values for the system). Additionally, it allows host administrators to specify system wide overrides. variables that an administrator can define as non-changeable (the files path, for instance) 3. Approach : This DEP proposes that we add 2 columns to the variable table, namely 'domain' and 'key'. This extra information would be used to store data against a certain object. In the case of the user system saving variables, it would save the variable with 'user' as the domain and $user->uid as the key. In the case of themes, it would save the variable with 'theme' as the domain and the theme name as the key. By default, only the system wide settings would be loaded, and then each module that wants to add a new layer to the variable stack, can add it's own layer of settings. (ie: user module will load all the settings related to the user). The only variables that will actually be overridden are variables that an explicit interface element is defined for, and the only variables that will be saved are the ones that differ from the original variables. -- Adrian Rossouw Drupal developer and Bryght Guy http://drupal.org | http://bryght.com
This is a very cool idea Adrian. I have a few questions... * Other than the theme_get_variable function is there other code this helps you abstract? * Do you got good ideas of use cases where different users want different settings variables? I definitely see the coolness of letting people have different configs for their blog / organic groups, but not quite getting the advantage of letting users configure the site to their own preference. * What in the heck is the user interface going to look like for managing different sets of settings? -Zack On Nov 29, 2005, at 1:59 PM, Adrian Rossouw wrote:
This is my first attempt at writing something like this, to me it's not as much about creating a useful DEP out of this, as it is getting a working format going. I apologise for having left this so long, it's been in my drafts for week now. I am just sending the incomplete document to see where we can go from.
If you have any other suggestions, please make them. I'd like to keep the format as simple as possible.
Title : Cascading variable system. Abstract: This DEP defines a method to allow for more flexibility in the way we handle variables. Author: Adrian Rossouw Dependencies: None Repository: Core Status: Suggestion?
1. Introduction : This DEP specifies a way to clear up and centralise the variable system in Drupal, and hopefully abstracts some code that we are often using for specific cases.
2. Motivation : Variables in Drupal ( records in the variable table ), are primarily used to store settings for the site. Frequently however, we have the need to override the specified variable.
Two such cases are a. variables that can be overridden for each user (the theme setting, for instance) b. variables that can be overridden for each theme (such as whether or not to display the site title).
This is done to allow us to specify defaults for the site, and then allow these defaults to be altered. In the case of (a.), we manually test for the overridden value with each occurence of the variable, and in the case of (b.) we have centralised all the theme variable changes by introducing the theme_get_variable() function.
This has the benefit of allowing us to define even more layers of variables, such as .. blog specific settings, node specific settings, locale specific settings (allowing us to localise the variables like site_name etc.) and even install specific settings (so install profiles can provide default values for the system). Additionally, it allows host administrators to specify system wide overrides. variables that an administrator can define as non-changeable (the files path, for instance)
3. Approach : This DEP proposes that we add 2 columns to the variable table, namely 'domain' and 'key'.
This extra information would be used to store data against a certain object. In the case of the user system saving variables, it would save the variable with 'user' as the domain and $user->uid as the key. In the case of themes, it would save the variable with 'theme' as the domain and the theme name as the key.
By default, only the system wide settings would be loaded, and then each module that wants to add a new layer to the variable stack, can add it's own layer of settings. (ie: user module will load all the settings related to the user).
The only variables that will actually be overridden are variables that an explicit interface element is defined for, and the only variables that will be saved are the ones that differ from the original variables.
-- Adrian Rossouw Drupal developer and Bryght Guy http://drupal.org | http://bryght.com
On 30 Nov 2005, at 2:00 AM, Zack Rosen wrote:
This is a very cool idea Adrian. I have a few questions...
* Other than the theme_get_variable function is there other code this helps you abstract? * Do you got good ideas of use cases where different users want different settings variables? I definitely see the coolness of letting people have different configs for their blog / organic groups, but not quite getting the advantage of letting users configure the site to their own preference. We already do this for all user specific settings (like whether or not tinymce is enabled.)
* What in the heck is the user interface going to look like for managing different sets of settings?
(i'll add this to the bottom) User interface doesn't change, only the flexibility of where saved settings take effect is added. So instead of saving settings to the data column in the user table, or stuffing all of the theme specific settings in a single variable, we store them in a nice normalised table. You would set variables using function variable_set($variable, $value, $domain = null, $id = null) { and you would get variable using function variable_set($variable, $default, $domain = null, $id = null) { The code specified default would go to the bottom of the queue, followed by the defaults the install profile specified (install profiles need this. otherwise any default setting change involves changing code wherever it happens) followed by the system wide settings, followed by the layout/section/ theme specific settings (this is the next DEP I want to write) and then at the very top of the stacks, the system overrides (like when a hosting provider doesn't want people to touch the files dir) [5] $GLOBALS['override'] [4] user settings. system table, user domain, userid key. [3] theme settings. system table, theme domain, theme name key. [2] system settings. system table, system domain. [1] $GLOBALS['defaults'] [0] $default -- Adrian Rossouw Drupal developer and Bryght Guy http://drupal.org | http://bryght.com
* What in the heck is the user interface going to look like for managing different sets of settings?
(i'll add this to the bottom)
I think this can be doe a lot easier. Just leave it all as it its now. Forms API (fapi) will detect where it gets teh setting from, and will grey out the form field if a variable cannot be changed by a user. No idea how to solve this technically though. Bèr -- [ Bèr Kessels | Drupal services www.webschuur.com ]
On 30 Nov 2005, at 12:14 PM, Bèr Kessels wrote:
* What in the heck is the user interface going to look like for managing different sets of settings?
(i'll add this to the bottom)
I think this can be doe a lot easier. Just leave it all as it its now.
Forms API (fapi) will detect where it gets teh setting from, and will grey out the form field if a variable cannot be changed by a user.
I'm explicitly not getting involved in forms api with this DEP. forms api can be extended to handle different levels of setting variables, but i don't believe it's the right place. -- Adrian Rossouw Drupal developer and Bryght Guy http://drupal.org | http://bryght.com
Op woensdag 30 november 2005 11:21, schreef Adrian Rossouw:
forms api can be extended to handle different levels of setting variables, but i don't believe it's the right place.
I think it is important to have a place in the DEP to say what it does /not/ involve too. Bèr -- | Bèr Kessels | webschuur.com | website development | | Jabber & Google Talk: ber@jabber.webschuur.com | http://bler.webschuur.com | http://www.webschuur.com |
On 30 Nov 2005, at 1:39 PM, Bèr Kessels wrote:
Op woensdag 30 november 2005 11:21, schreef Adrian Rossouw:
forms api can be extended to handle different levels of setting variables, but i don't believe it's the right place.
I think it is important to have a place in the DEP to say what it does /not/ involve too.
and perhaps a 'security considerations' section. -- Adrian Rossouw Drupal developer and Bryght Guy http://drupal.org | http://bryght.com
Op woensdag 30 november 2005 11:21, schreef Adrian Rossouw:
forms api can be extended to handle different levels of setting variables, but i don't believe it's the right place.
I think it is important to have a place in the DEP to say what it does /not/ involve too.
and perhaps a 'security considerations' section.
and a status - draft, ..., core, contrib
Op woensdag 30 november 2005 13:43, schreef vlado:
Op woensdag 30 november 2005 11:21, schreef Adrian Rossouw:
forms api can be extended to handle different levels of setting variables, but i don't believe it's the right place.
I think it is important to have a place in the DEP to say what it does /not/ involve too.
and perhaps a 'security considerations' section.
and a status - draft, ..., core, contrib
Added those: http://drupal.org/node/39407 Bèr -- | Bèr Kessels | webschuur.com | website development | | Jabber & Google Talk: ber@jabber.webschuur.com | http://bler.webschuur.com | http://www.webschuur.com |
[5] $GLOBALS['override']
[4] user settings. system table, user domain, userid key. [3] theme settings. system table, theme domain, theme name key. [2] system settings. system table, system domain. [1] $GLOBALS['defaults'] [0] $default
so variable_get() will return the right value based on the priority above? please add that to the DEP if it isn't there already. about #4. I'd like for user variables to be loaded into the $user object so that we don't go to the DB for them multiple times on a page view. For example, lets say we invent a user pref called 'show signatures' and I set that to FALSE because I am on a low bandwidth connection. I would not want a variable_get('show signatures') to hit the DB every time we show a node or comment. maybe a static cache would solve this. in general, i'm a bit hesitant about storing user variables in the variables table. thats what $user object and users.data column and $_SESSION are for. Would you get rid of users.data? Maybe get rid of that users.data and change users.module to write to variables table instead. while you are there, move users.timezone and other non critical fields from columns in the users table to records in the variables table. one other nit - in drupal, higher numbers have lower priority so the table above could be numbered in reverse. FEATURE REQUESTS for a rainy day ... - have some way to cleanup the variables table of cruft from disabled modules - in variable_get(), return a hint about what layer provided the value, so the front end can grey out the form field if necessary.
Moshe Weitzman wrote:
about #4. I'd like for user variables to be loaded into the $user object so that we don't go to the DB for them multiple times on a page view. For example, lets say we invent a user pref called 'show signatures' and I set that to FALSE because I am on a low bandwidth connection. I would not want a variable_get('show signatures') to hit the DB every time we show a node or comment. maybe a static cache would solve this.
Umm, doesn't variable_get() only hit the database the _first_ time the variable is requested on a page? After that, it's in the global $conf, isn't it? In general, I think it makes sense to keep user variables in user-related tables, *not* in the {variables} table. ..chrisxj
On 30 Nov 2005, at 7:36 PM, Chris Johnson wrote:
Moshe Weitzman wrote:
about #4. I'd like for user variables to be loaded into the $user object so that we don't go to the DB for them multiple times on a page view. For example, lets say we invent a user pref called 'show signatures' and I set that to FALSE because I am on a low bandwidth connection. I would not want a variable_get('show signatures') to hit the DB every time we show a node or comment. maybe a static cache would solve this.
Umm, doesn't variable_get() only hit the database the _first_ time the variable is requested on a page? After that, it's in the global $conf, isn't it?
In general, I think it makes sense to keep user variables in user- related tables, *not* in the {variables} table.
This is only for places where we want to override the defaults specified by the system. Like for instance whether or not to display tinymce, the user's frontpage (if that's customisable), the user's theme (if that's customisable). So that every time we want to use (say) the site_frontpage variable, we don't have to explicitly code and check for the different kinds of overrides. We could put the user's variables in the user table, and variable_get needs to know how to check variables for different domains. I don't like the 'data' field. It's great if you are doing fast development, but what happens when (in recent memory), we disable comment paging on drupal.org, and the interface element to configure that has been removed. You'd need to open up every user object and unset that, and then save it back to the database. With that in the variables table, it would be a single line of sql. SO what happens when you stop a theme that some people had as a default.? With these variables (not properties) in the variables table, we can control stuff like this. -- Adrian Rossouw Drupal developer and Bryght Guy http://drupal.org | http://bryght.com
Chris Johnson wrote:
Moshe Weitzman wrote:
about #4. I'd like for user variables to be loaded into the $user object so that we don't go to the DB for them multiple times on a page view. For example, lets say we invent a user pref called 'show signatures' and I set that to FALSE because I am on a low bandwidth connection. I would not want a variable_get('show signatures') to hit the DB every time we show a node or comment. maybe a static cache would solve this.
Umm, doesn't variable_get() only hit the database the _first_ time the variable is requested on a page? After that, it's in the global $conf, isn't it?
Right. More specifically: The cached serialized array of variables is retrieved from the cache table and the vars are assigned to the $conf array. You can override those with vars from settings.php
In general, I think it makes sense to keep user variables in user-related tables, *not* in the {variables} table.
Anybody pursuing the user-variables-in-variable-table idea in earnest has just had a bad idea. (no, that is not waht I intended to write originally) Cheers, Gerhard
Anybody pursuing the user-variables-in-variable-table idea in earnest has just had a bad idea. (no, that is not waht I intended to write originally)
Noone suggested this. Adrian proposed automatically loading *system* variables into memory like our current $conf. I think Chris missed this distinction and then asked for assurance about how our current system works.
What about module settings per user? Let's say I write an Album Module and I want each user to be able to select the way his gallery should be presented, or the theme that should be used for it. Of course I could put these on a DB table, but that wouldn't that just add more tables and module logic? These wouldn't be needed if this is implemented. Gerhard Killesreiter wrote:
Chris Johnson wrote:
Moshe Weitzman wrote:
about #4. I'd like for user variables to be loaded into the $user object so that we don't go to the DB for them multiple times on a page view. For example, lets say we invent a user pref called 'show signatures' and I set that to FALSE because I am on a low bandwidth connection. I would not want a variable_get('show signatures') to hit the DB every time we show a node or comment. maybe a static cache would solve this.
Umm, doesn't variable_get() only hit the database the _first_ time the variable is requested on a page? After that, it's in the global $conf, isn't it?
Right. More specifically: The cached serialized array of variables is retrieved from the cache table and the vars are assigned to the $conf array. You can override those with vars from settings.php
In general, I think it makes sense to keep user variables in user-related tables, *not* in the {variables} table.
Anybody pursuing the user-variables-in-variable-table idea in earnest has just had a bad idea. (no, that is not waht I intended to write originally)
Cheers, Gerhard
On 30 Nov 2005, at 8:55 PM, Michelangelo Partipilo wrote:
What about module settings per user? Let's say I write an Album Module and I want each user to be able to select the way his gallery should be presented, or the theme that should be used for it. Of course I could put these on a DB table, but that wouldn't that just add more tables and module logic? These wouldn't be needed if this is implemented. Yeah.
It would also allow each blog to have it's own settings. Or each section of the site. -- Adrian Rossouw Drupal developer and Bryght Guy http://drupal.org | http://bryght.com
Michelangelo Partipilo wrote:
What about module settings per user? Let's say I write an Album Module and I want each user to be able to select the way his gallery should be presented, or the theme that should be used for it. Of course I could put these on a DB table, but that wouldn't that just add more tables and module logic? These wouldn't be needed if this is implemented.
Hi, glad you are back. it is a bad idea because you would have a variable for each user. On Drupal.org it would mean 40000 variables. Those would be (with the current design) loaded per page view and maybe break you php memory limit. The current recommended way of adding user variables is to a) store them in the user object. This way they will end up in the user.data column. Example: block.module pro: easy to do contra: hard to get the setting for all users as it is in a serialized array inside user.data or b) make a db table for this and store it yourself pro: easy to get all users with setting "foo" contra: needs a little bit more work There is also option c): add to the $_SESSION variable pro: easy to do contra: only lasts for the current session Cheers, Gerhard
On 30 Nov 2005, at 10:12 PM, Gerhard Killesreiter wrote:
Michelangelo Partipilo wrote:
What about module settings per user? Let's say I write an Album Module and I want each user to be able to select the way his gallery should be presented, or the theme that should be used for it. Of course I could put these on a DB table, but that wouldn't that just add more tables and module logic? These wouldn't be needed if this is implemented.
Hi, glad you are back.
it is a bad idea because you would have a variable for each user. On Drupal.org it would mean 40000 variables. Those would be (with the current design) loaded per page view and maybe break you php memory limit.
They would not all be loaded into memory at the same time.. each layer gets loaded individually. So the only ones loaded by default are the ones specified in code (default or override), and then each other layer would get initialised seperately. ie: variable_load('user', $user->uid); blog module might have variable_load('blog', $user->uid); themes might have variable_load($themes, $GLOBALS['theme']); This would only load something IF the user has overridden any specific settings, and then only specifically those variables that have been changed. -- Adrian Rossouw Drupal developer and Bryght Guy http://drupal.org | http://bryght.com
I have a feature request: get_or_set_variable() This will look to see if a variable is in the system, and if not, write the default value to the database. This stops the situation where some of the state is in the database and some is in the code (in the form of defaults). I find it better to have all the state in one place. -Robert
Op woensdag 30 november 2005 01:16, schreef Adrian Rossouw:
[5] $GLOBALS['override'] [4] user settings. system table, user domain, userid key. [3] theme settings. system table, theme domain, theme name key. [2] system settings. system table, system domain. [1] $GLOBALS['defaults'] [0] $default
What about: [6] $GLOBALS['override'] [5] $SESSION [4] user settings. system table, user domain, userid key. [3] theme settings. system table, theme domain, theme name key. [2] system settings. system table, system domain. [1] $GLOBALS['defaults'] [0] $default Taht would allow us to do fancy stuff, like themeswitching in a very easey way. Bèr
Another use case : http://drupal.org/node/39567 On 30 Nov 2005, at 10:47 PM, Ber Kessels wrote:
Op woensdag 30 november 2005 01:16, schreef Adrian Rossouw:
[5] $GLOBALS['override'] [4] user settings. system table, user domain, userid key. [3] theme settings. system table, theme domain, theme name key. [2] system settings. system table, system domain. [1] $GLOBALS['defaults'] [0] $default
What about: [6] $GLOBALS['override'] [5] $SESSION [4] user settings. system table, user domain, userid key. [3] theme settings. system table, theme domain, theme name key. [2] system settings. system table, system domain. [1] $GLOBALS['defaults'] [0] $default
Taht would allow us to do fancy stuff, like themeswitching in a very easey way.
Bèr
-- Adrian Rossouw Drupal developer and Bryght Guy http://drupal.org | http://bryght.com
My number one feature request for variable handling in Drupal is this: 1. Every variable has its default value defined in ONE place (e.g. perhaps inside hook_init() for each module, have function calls such as "variable_define('some_value', 42);", where 42 is the default value). 2. Remove that evil second argument from variable_get(). Default values should not be repeated every time a variable is used. Regarding the actual proposal of a cascading variable system, sounds good to me! If variables can be overridden in multiple layers, in such a way that module authors don't get really confused, that will certainly improve the system. Jaza.
On Sat, 2005-12-03 at 03:29 +1100, Jeremy Epstein wrote:
My number one feature request for variable handling in Drupal is this:
1. Every variable has its default value defined in ONE place (e.g. perhaps inside hook_init() for each module, have function calls such as "variable_define('some_value', 42);", where 42 is the default value). 2. Remove that evil second argument from variable_get(). Default values should not be repeated every time a variable is used.
Regarding the actual proposal of a cascading variable system, sounds good to me! If variables can be overridden in multiple layers, in such a way that module authors don't get really confused, that will certainly improve the system.
Jaza.
I think defining system variables every page load would be a bad idea performance wise... plus checking each one to make sure you aren't overwriting data pulled from the database... However adding a hook_activate, hook_deactivate that is only called when a module is enabled or disabled respectively, would create a great place for variable_define... .darrel.
On 02 Dec 2005, at 6:29 PM, Jeremy Epstein wrote:
1. Every variable has its default value defined in ONE place (e.g. perhaps inside hook_init() for each module, have function calls such as "variable_define('some_value', 42);", where 42 is the default value). i'm not opposed to the idea, but what I have in mind is that each module during install can set up defaults into the bottom layer.
Each install profile can then set up defaults which are in a layer above the module specified defaults. I also believe we need a code based set of defaults. kind of like we use $conf now. Not sure where that would fit in.
2. Remove that evil second argument from variable_get(). Default values should not be repeated every time a variable is used. I kind of like it. =)
Regarding the actual proposal of a cascading variable system, sounds good to me! If variables can be overridden in multiple layers, in such a way that module authors don't get really confused, that will certainly improve the system. Well, i'm thinking we could pull it off in a few simple api functions. It would make a lot of modules simpler. take the content settings for instance.
-- Adrian Rossouw Drupal developer and Bryght Guy http://drupal.org | http://bryght.com
participants (12)
-
Adrian Rossouw -
Ber Kessels -
Bèr Kessels
-
Chris Johnson -
Darrel O'Pry -
Gerhard Killesreiter -
Jeremy Epstein -
Michelangelo Partipilo -
Moshe Weitzman -
Robert Douglass -
vlado -
Zack Rosen