- Development - lists.drupal.org

A gotcha with auto-increment and MySQL
by Barry Jaspan 04 May '07

04 May '07

The most recent thread on using auto-incrementing columns in Drupal seems to have led to the decision that it is okay to use them (acknowledging that both our mysql and pgsql tables already do use them), and the upcoming database schema data structure supports them. However, while in my waking-up haze this morning, out of nowhere I realized one of the reasons they were probably previously avoided. On an auto-incrementing column, a race condition exists if db_next_id() is sometimes used and sometimes not used. The problem is that while db_next_id() is atomic it does not alter MySQL's idea of the next id to assign. Suppose at T0 the maximum nid in the node table is 10 and the sequences table reflects that. At T1, one Drupal process calls db_next_id('{node}_nid'). The sequences table is atomically updated to 11 and the caller gets 11. At T2, a second Drupal process inserts a new row into the node table without providing a nid; since the maximum nid is 10, the new row is assigned nid 11. At T3, the first process tries to insert a row using the nid 11 it got from db_next_id() and fails. This problem does not occur with pgsql because the db_next_id() and the auto-incrementing column both use the same underlying 'sequence' object. I see two ways to solve this problem: 1. Disallow auto-incrementing columns everywhere in Drupal. Note that they are used on numerous core tables both with mysql and pgsql. To remove them, we would have to verify that every insert into a table uses db_next_id() to generate the new id. 2. Just don't use db_next_id() and auto-increment on the same columns, ever. We could just declare this a bug and fix it when discovered. However, now that we (will soon) have a database schema, we could also make db_next_id() actively warn about it: function db_next_id($name) { list($table, $col) = preg_split_table_and_column_name($name); $schema = drupal_get_schema(); if ($schema[$table]['cols'][$col]['type'] == 'serial') { // log an error here } // rest of db_next_id() here } Thoughts? Barry

3 2

Re: [development] low hanging fruit for Drupal 6: variable defaults
by Peter Wolanin 04 May '07

04 May '07

I don't think the suggestion below makes sense (setting variable defaults via hook_install), at least if we assume that the way to reset a module setting is to delete the corresponding variable. -Peter --- From: "Greg Knaddison - GVS" <Greg(a)GrowingVentureSolutions.com> To: development(a)drupal.org Date: Thu, 3 May 2007 09:03:37 -0600 Subject: Re: [development] low hanging fruit for Drupal 6: variable defaults On 5/3/07, Gabor Hojtsy <gabor(a)hojtsy.hu> wrote: > **We need a central place to define variable defaults** Why can't that central place be the variable table? We could remove the second argument to variable_get and refactor some code to do the variable_set at install time which eliminates the need for the defaults in the variable_get() calls themselves.

4 4

Re: [development] low hanging fruit for Drupal 6: variable defaults
by Peter Wolanin 03 May '07

03 May '07

If you are concerned about cleaning up unused variables, this simple patch for cleaning up unused node-type variables should also be added to the mix: http://drupal.org/node/104572 -Peter ---------- Forwarded message ---------- From: "Greg Knaddison - GVS" <Greg(a)GrowingVentureSolutions.com> To: development(a)drupal.org, "Mohammed Sameer" <msameer(a)foolab.org> Date: Thu, 3 May 2007 12:55:58 -0600 Subject: Re: [development] low hanging fruit for Drupal 6: variable defaults On 5/3/07, Mohammed Sameer <msameer(a)foolab.org> wrote: > > If we have it as a hook then we don't really need to clean them up in hook_uninstall() because > drupal can invoke the variables defaults hook, get the list of variables and simply delete > them from the "registry". We now only have one problem. hook uninstall runs when the module > is unloaded but then, we can implement the extra hook in the .install file ? > > Hope I'm not talking nonsense ;-) > I think it makes at least some sense. My experience from looking at hook_uninstall solutions is that lots of variables get left around. The proposed hook_settings() would create some basic variables. It would probably not know about _all_ of the variables that the module may create. Many variable names are based on variables: e.g. modulename_contenttype_something. My solution for that has been to delete all rows from variable table that start with modulename_ I think if you try to delete specific variables I think you'll end up leaving lots of unused keys around. The realm idea would help a lot with the cleanup, but is not "low hanging fruit" IMO. Greg

1 0

Why do we avoid auto-incrementing db columns?
by Barry Jaspan 03 May '07

03 May '07

Could someone either clue me in or point to a prior discussion about why Drupal avoids using auto-incrementing database columns? If it is database compatibility, which versions of which databases that we support do not have it? Thanks, Barry

21 37

Re: [development] low hanging fruit for Drupal 6: variable defaults
by Larry Garfield 03 May '07

03 May '07

+1 to centralized defaults. I don't know from localization, so I'll leave it to Gabor to decide what's best for that. The "variable registry" hook seems consistent with what Drupal 6 is doing. We have a menu callback registry (hook_menu), a forms registry (hook_forms), we now have a theme registry (hook_theme), why not a variable registry (hook_variables)? :-) I'm open to either that or mandated variable_set() in .install. I think the registry would be a bit easier for development (easier to add/change quickly), while mandated variable_set() would mean we don't need to add another hook to each module (less code++). I'm not wild about the namespace concept, until I see a code sample of how I'd use it. Would that replace the second parameter? variable_get('system', 'site_name')? Remember that modules call variables across the module-line all the time, so that needs to be kept as simple as possible. As long as we're mucking about in the variable system, the cache system recently introduced a "serialized" flag to specify if a given value needs to be serialized or not. (I believe the default was object/array = serialize else don't bother.) That saves the unserialization time on a string or int. Given that the variable table is loaded completely on a page load, I think the same thing could work here to reduce the amount of deserialization required. I dare say half of the stuff stored there, probably more, is just ints and strings that don't need to be serialized. (And it would make manually setting clean URLs off easier, a common support request. <g>) I'm not sure how that impacts the registry/install question, but consider that a feature request while you're at it. Hm, here's another use case I just thought of... Do we have cases where the default from variable_get() is itself dynamic, such as the return value from another function? If so, is that rare enough that we can say "just use NULL there and throw in your own if statement"? --Larry Garfield On Thu, 3 May 2007 08:15:40 -0700, "John Vandervort" <jvandervort(a)gmail.com> wrote: > Nice. I hadn't read adrian's thread before. If only I'd searched on > 'relm' > :) > > So if we change the variable table to (system.install): > > db_query("CREATE TABLE {variable} ( > namespace varchar(128) NOT NULL default '' > name varchar(128) NOT NULL default '', > value longtext NOT NULL, > language varchar(12) NOT NULL default '', > PRIMARY KEY (namespace, name, language) > ) /*!40100 DEFAULT CHARACTER SET UTF8 */ "); > > We'd be halfway there. > > a patch to variable_get, set, del, and init... in bootstrap.inc and we'd > maintain full backward compatibility > with the current system, but allow namespace uniqueness and a helper > function for mass loading... > > > > On 5/3/07, Moshe Weitzman <weitzman(a)tejasa.com> wrote: >> >> > - we can save extra typing and value passing on variable_get calls >> > - we have a hook to reuse when we need a list of variables >> > to present for localization >> > >> > adds pluses over just using constants. >> > >> > Sure, only using constants for default variable values could be make >> > variable usage better, but I think a hook do more just by being a >> > central place and not a set of unrelated constants. >> >> i'd like to see this centralized in a hook too. we'll be one step closer >> to >> the hierarchy that adrian proposed long ago: >> http://lists.drupal.org/archives/development/2006-06/msg00309.html >> > > > > -- > -> JV > >

5 5

Oracle patches for drupal 5.1, first alpha
by Gabriele Turchi 03 May '07

03 May '07

Hi all. these are my first attempt to provide Oracle support for drupal 5.1 (core - CCK and views will follow shortly). Php 5.2.1 with oci8 module is needed (there are many oci bugs in the previous versions). The database provider is named "oci8", as in php. My source was the pgsql version. The porting for "blog" and "forum" modules is incomplete. http://www.l39a.com/drupal/20070503-drupal-5.11-oracle.patch.gz This is a (linux) patch file, change directory to a drupal 5.1 root and run "zcat 20070503-drupal-5.11-oracle.patch.gz | patch -p0" http://www.l39a.com/drupal/database.oci8.inc.gz http://www.l39a.com/drupal/install.oci8.inc.gz These files must be un-zipped and installed in <drupal-5.1>/includes. Best regards, Gabriele P.S.: I'm sorry, my english is alpha version.

1 0

Actions in core
by John VanDyk 02 May '07

02 May '07

What are actions? ----------------- Warning: please disassociate any thoughts you have about workflow and actions. They are two separate modules. Workflow implements a state machine to move a node through various states. That is its main purpose. Actions only come into play because the workflow module knows how to assign and fire actions. So, putting any thoughts of workflow aside, let's proceed. Actions are like stored procedures that can be loosely coupled with events in Drupal. Let's examine a ridiculously simple example. Suppose you want to be draconian with your Drupal site's comment policy. Any user who leaves a comment gets blocked immediately. You can implement this quite easily with the comment hook: foo_comment($a1, $op) { if ($op == 'insert') { global $user; // block user here... } } What an action does is wrap up this code into a nice package. That package can then be associated with the nodeapi hook. Or some other hook. Actions takes this function, which we'll call "Block current user", and allows you to assign it to any hook-op combination. We do that by several coding conventions. Here is the skeleton of a simple action: /** * Implementation of a Drupal action. */ function action_block_user($op, $context = array(), &$object) { switch ($op) { case 'do': global $user; // block user here... break; case 'metadata': return array( 'description' => t('Block current user'), 'type' => 'User', 'batchable' => TRUE, 'configurable' => FALSE, 'supported hooks' => array( 'nodeapi' => array( 'delete' => 1, 'insert' => 1, 'update' => 1, 'view' => 1 ), 'comment' => array( 'insert' => 1, 'update' => 1, 'delete' => 1 ) ) ); } } We've implemented two ops. The 'metadata' op describes the action by declaring some things, including what hook/op combinations it supports. The 'do' op is actually where the code runs. We list the 'do' op first in the switch statement for performance, since Drupal will then encounter it first during execution. The above is a nonconfigurable action. There's nothing about it you can change. It will block the current user and that's that. It's basically a singleton action. On the other hand, an action like "Send email" needs to know some things in order to run. It needs a subject, and body, a recipient, and such. Configurable actions are not available for use until they have been configured (obviously). They can be configured from the main actions interface at admin/build/actions. Check out the HEAD version of actions and install it to play along. Note that I'm currently developing against Drupal 5 until things are a bit more stable. What's new? ---------- All of this is not new. I've been saying the above for years now. So here's what's new. 1. We now have a central interface to assign actions to hooks through the browser. There's a new "assign actions" section at admin/build/actions/assign. I demonstrated that at OSCMS for those of you who were there. So you can now assign the "Block current user" action to the nodeapi view op, and so on. You can't assign it to the cron hook, because that would be stupid. 2. The actions engine has been pulled out into a separate .inc file which will live at includes/actions.inc. This is so that, like menu.module, actions.module can be disabled if you're not using actions, or are using actions only within your own modules. What are the current problems? ------------------------------ We are still feeling around for the best way to unify information fed into actions. The current approach is to pass an action-specific $op (like 'do' or 'metadata', see above), a context array, and an object. So when an action gets an object, it can look at the context and know whether it's receiving a node object or a comment object or whatever. See the code in default_actions.inc for examples of actions figuring out what to do based on the context. I think this is a simple approach that will work. Another issue is, uh, factoring of Drupal code. Core has a lot of things that have never been factored out. In our example above, you'd expect to be able to call a function like user_block_user($uid) so that the action's 'do' op just becomes a wrapper around existing functionality. Instead, currently user blocking is done inside user_save(). So you could say "well, just do user_load(), change the status, and do user_save() and stop mumbling about factoring!" OK, that leads me to the next issue. Recursion. If you take that approach, and you do a user_load() and a user_save(), lots of hooks are going to fire. If you're not careful, you might have assigned actions to those hooks which call functions which call other hooks. The complexity of the execution path can become an issue. Currently the actions engine keeps a stack and if more than 25 calls are made to actions_do() it bails out. For more about recursion, see the beginning of this paragraph. What is the future look like for actions? ---------------------------------------- I'd like to see actions for each module in core. If we refactor some code in a smart way, adding actions will just be adding wrappers around existing functionality. So statistics.module can have a "Clear access log" action, taxonomy.module can have a "Add title to vocabulary" action, etc. etc. The current default_actions.inc contains actions that will be distributed to the various core modules. The next major hurdle for actions consists of action sets and conditionals. Action sets are groups of actions that go together. Conditionals evaluate and determine whether an action set actually executes. eaton has paved the way for both of these in his voting actions module. Unless we're very busy bees, that will have to wait for Drupal 7. What can you do to help? ------------------------ Install Drupal 5. Download the HEAD version of actions module. Throw the actions.inc file into your includes directory. Poke around. Write some actions. Give some feedback. Cheers, John

3 5

Re: [development] Actions in core
by Boris Mann 02 May '07

02 May '07

On 5/1/07, John VanDyk <jvandyk(a)iastate.edu> wrote: > What are actions? <snip> +1, awesome example, someone please copy / paste into the first of the "Actions" handbook pages :P > Another issue is, uh, factoring of Drupal code. Core has a lot of > things that have never been factored out. In our example above, you'd > expect to be able to call a function like user_block_user($uid) so > that the action's 'do' op just becomes a wrapper around existing > functionality. Instead, currently user blocking is done inside > user_save(). So you could say "well, just do user_load(), change the > status, and do user_save() and stop mumbling about factoring!" OK, > that leads me to the next issue. And this "factoring" should lead to CRUD-type functions that can be used easily in install profiles as well.... -- Boris Mann Office 604-682-2889 Skype borismann http://www.bryght.com

2 1

Drupal 6 status update
by Dries Buytaert 02 May '07

02 May '07

Hello, I started writing a Drupal 6 status update for publication on the drupal.org main page. The post is currently unpublished (!) but lives at http://drupal.org/ drupal-6-status-update. I'm inviting all site maintainers to have a look and to extend the post a bit. Many of you can edit the post and might have something to add (if only a link). So please, go ahead and make some improvements and additions. When you do, keep in mind that you're writing for a broad audience and try to avoid developer speak. If you're not sure if something is worth adding, add a note at the bottom of the post. Also make sure to keep the post unpublished. I plan to go over the latest version of the post, wrap it up, and publish it tomorrow afternoon (24 hours from now). Thanks for the help! :) -- Dries Buytaert :: http://www.buytaert.net/

1 1

Sign-up procedure
by Karthik 02 May '07

02 May '07

In a site with ~33000 users, I found that about ~1900 sign-ups were never completed and essentially clutter up the user table. I am looking for opinions on the pros and cons, if any, of streamlining the default sign-up procedure to: a) Sign-up form only contains an e-mail field that the user fills in to confirm authenticity. b) The e-mail field is stored in a separate table along with the sign-up URL token. c) The sign-up e-mail contains nothing besides a sign-up URL which the user clicks as opposed to the current e-mail which contains a number of choices. d) The user is presented with a web page where he selects his username, password and assorted details (other than his e-mail for the first time) and is logged in immediately. e) Only after successfully completing step d) is the user account added into the user table. f) Perhaps an e-mail is sent upon successful completion of the procedure for future reference. Thanks, -K

8 11