November 2005 - Development - lists.drupal.org

Administration Survey: Theme improvements, theme help system, theme mailing list
by Kieran Lal 13 Dec '05

13 Dec '05

As part of the Drupal administration user experience survey we identified Drupal theme issues as the most difficult administration issues( http://www.surveymonkey.com/DisplaySummary.asp? SID=1425065&U=142506581557). There was some confusion about the results so Trae McCombs, from CivicSpace Labs, conducted six additional theme development interviews. Trae identified some common goals of theme developers, basic theme tasks, and difficult theme tasks. I have created a documentation page, http://drupal.org/node/39451, which outlines all this. I also spoke with Dries yesterday and requested that a theme developers mailing list be started to focus on helping with Drupal's most difficult administration task. In summary here are the actions we are taking to improve administration specifically focused on Drupale theme development. 1) Identified Drupal theming as the most difficult Drupal administration task. 2) Improved the project module to allow for categorization so that themes can be categorized to help themers pick a solid base theme. Designed by Dries and implemented by Nedjo Rogers for CivicSpace Labs. This is coming in the next few weeks on Drupal.org. 3) Provided a "Manage inconsistency in themes" documentation page with best practices. http://drupal.org/node/37156 4) Created a support channel #cstheme to support development of the CivicSpace theme. This may lead to a dedicated Drupal theme channel. 5) Added a link to theme help from the theme admin interface using the CivicSpace theme. Added several base layouts to the CS theme for theme developers to customize with styles. 6) Created a theme help documentation page to focus on basic and difficult theme tasks. http://drupal.org/node/39451 7) Requested a themers mailing list to focus specifically on the tasks of Drupal theme development Please help us to develop theme help documentation(http://drupal.org/ node/39451). Understanding basic and difficult theme tasks, and how to solve them, will allow us to attract more themers to the platform and make it easier for all of us to theme our sites. I'll leave you with a quote: Dries_:it's the reason I didn't convert buytaert.net yet -- it takes too f*****g long to make a theme Cheers, Kieran Development Manager CivicSpace Labs

18 33

Drupal 4.6.4 / 4.5.6 released
by Dries Buytaert 12 Dec '05

12 Dec '05

Hello world, Drupal 4.6.4 is available for download. Drupal 4.6.4 is a maintenance release that fixes problems reported using the bug tracking system, as well as 3 security vulnerabilities (two "less critical", one "not critical") that affect all previous versions of Drupal. Since the vulnerabilities are also present in the Drupal 4.5 series, Drupal 4.5.6 is released as well. Upgrading your existing Drupal sites is strongly recommended. More information about the Drupal 4.6.4 and 4.5.6 release is available at http://drupal.org/drupal-4.6.4. Security advisories are available at http://drupal.org/security. Kudos to the security team for their work on getting these releases out, -- Dries Buytaert :: http://www.buytaert.net/

6 8

node access issue
by Gordon Heydon 04 Dec '05

04 Dec '05

Hi, I have just been given a hairy request from a client and need to guidance to what would be the best method of doing this. The site is a heavily customised drupal 4.6 which is running the organic groups. Organic groups is using the node access table already, which causes a conflict with the node_priv_by_role module. But what they want I think maybe able to be done via something else, but I am not sure what. Basically they want is for anonymous users to only access a few bits of content, and nothing else, where as authenticated users will see everything there, which hasn't been restricted by organic groups. Any ideas will be much appreciated. Gordon.

4 7

DEP - cascading variable system
by Adrian Rossouw 03 Dec '05

03 Dec '05

This is my first attempt at writing something like this, to me it's not as much about creating a useful DEP out of this, as it is getting a working format going. I apologise for having left this so long, it's been in my drafts for week now. I am just sending the incomplete document to see where we can go from. If you have any other suggestions, please make them. I'd like to keep the format as simple as possible. Title : Cascading variable system. Abstract: This DEP defines a method to allow for more flexibility in the way we handle variables. Author: Adrian Rossouw Dependencies: None Repository: Core Status: Suggestion? 1. Introduction : This DEP specifies a way to clear up and centralise the variable system in Drupal, and hopefully abstracts some code that we are often using for specific cases. 2. Motivation : Variables in Drupal ( records in the variable table ), are primarily used to store settings for the site. Frequently however, we have the need to override the specified variable. Two such cases are a. variables that can be overridden for each user (the theme setting, for instance) b. variables that can be overridden for each theme (such as whether or not to display the site title). This is done to allow us to specify defaults for the site, and then allow these defaults to be altered. In the case of (a.), we manually test for the overridden value with each occurence of the variable, and in the case of (b.) we have centralised all the theme variable changes by introducing the theme_get_variable() function. This has the benefit of allowing us to define even more layers of variables, such as .. blog specific settings, node specific settings, locale specific settings (allowing us to localise the variables like site_name etc.) and even install specific settings (so install profiles can provide default values for the system). Additionally, it allows host administrators to specify system wide overrides. variables that an administrator can define as non-changeable (the files path, for instance) 3. Approach : This DEP proposes that we add 2 columns to the variable table, namely 'domain' and 'key'. This extra information would be used to store data against a certain object. In the case of the user system saving variables, it would save the variable with 'user' as the domain and $user->uid as the key. In the case of themes, it would save the variable with 'theme' as the domain and the theme name as the key. By default, only the system wide settings would be loaded, and then each module that wants to add a new layer to the variable stack, can add it's own layer of settings. (ie: user module will load all the settings related to the user). The only variables that will actually be overridden are variables that an explicit interface element is defined for, and the only variables that will be saved are the ones that differ from the original variables. -- Adrian Rossouw Drupal developer and Bryght Guy http://drupal.org | http://bryght.com

12 23

[drupal-devel] performance improvements - avoiding big unserialize()
by Moshe Weitzman 01 Dec '05

01 Dec '05

In speaking with Rasmus last night, he discouraged us from using unserialize() on large arrays when possible. This eats memory and speed. We do this at lkeast twice on evrry page view: the menu cache and the variables cache (i.e. $conf). Rasmus' suggestion was to store these arrays without serialization in a shared memory system like APC. I have a long term plan to add *optional* shared memory storage for these arrays and maybe others as they arise. Perhaps someone else wants to move this forward quickly. Comments welcome.

12 18

Trying to understand hook_nodeapi() invocation sequence from node.module
by Syscrusher 30 Nov '05

30 Nov '05

Good morning! I've run into some odd behavior from node.module in Drupal HEAD (updated as of 2005-11-23 at about 2300 UTC). I'm not reporting an "issue" on the module yet, because I strongly suspect the "problem" is ignorance on my part rather than a bug in the core code. I have a module that adds extra fields to node edit screens, similar to the way the upload.module does. The difference is that my module can have zero, one, or more instances of my custom fields, and I've got code that examines how many are defined and always ensures that all the existing ones are shown plus one blank one, on each preview or initial entry into the Edit page for an existing node. The particulars of my code aren't really relevant, and I'm not asking for help debugging that. Instead, what I'm seeking is to understand why hook_nodeapi() is being invoked in the particular sequence that I'm seeing. I have the following trace enabled in my hook_nodeapi() implementation: function mymodule_nodeapi(&$node, $op, $teaser=NULL, $page=NULL) { // DEBUG: Extra trace logic drupal_set_message("nodeapi $op $node->nid"); switch($op) { // .... the usual stuff here } } Now, on initial entry into the edit form on node with $nid == 8, here is the output of that drupal_set_message(): nodeapi load 8 nodeapi load 8 nodeapi prepare 8 nodeapi form 8 And on a "preview" of the node after editing, I see this: nodeapi load 8 nodeapi load 8 nodeapi prepare 8 nodeapi form 8 nodeapi validate 8 nodeapi view 8 nodeapi validate 8 My question is, why is $op=='load' being always invoked twice, and why is $op=='validate' being invoked twice for previews? The problem I'm having with *my* logic has to do with that second invocation of "load", and I'm finding that I have to build explicit conditional logic in the load function to detect whether I've already done it before. It seems to me that represents extra database hits, if nothing else, and in my particular case a design obstacle as well. I've checked the documentation for hook_nodeapi(), but it doesn't seem to be fully updated for 4.7 yet, and also doesn't really talk about the *sequence* of calls to the function, but rather treats each one individually. Can someone offer some general commentary on how this is supposed to work in 4.7? I suspect I'm not the only one who's confused. Or if I've overlooked it in the docs, please point me to a page, and I'll cheerfully apologize for wasting your time. I did try to RTFM before asking on list, but I may have missed something. Many thanks! Scott -- ------------------------------------------------------------------------------- Scott Courtney Drupal user name: "syscrusher" http://drupal.org/user/9184 scott at 4th dot com Drupal projects: http://drupal.org/project/user/9184 Sandbox: http://cvs.drupal.org/viewcvs/drupal/contributions/sandbox/syscrusher

3 4

404 search
by David K Norman 30 Nov '05

30 Nov '05

I just wanted to make sure this wasn't implemented before I submitted an official feature request issue. Sometimes I remove an attachment from a node, but Google still keeps it indexed for a while. I can see people still trying to access the file in my logs, but they get a 404. I think it would be cool if the 404 message was able to say "Page not found", but then use the missing URL as a search string to display other possible similar pages in the site under the Page not found message. David Norman http://deekayen.net/

6 5

Tip for large site scaling: tracker.module considered harmful
by John Handelaar 30 Nov '05

30 Nov '05

First off, apols for the 'considered harmful' cliché. Description: http://mbr.org Forum posts: 8576, with 63015 comments 1. Some tracker module queries, particularly the one which shows 'all my posts' to individual users, stop functioning on large data sets. In this case, queries ran for >600s 2. An INSERT/UPDATE on {comments} while those queries are trying to complete will be stalled, waiting for a full-table lock (MySQL + MyISAM). 3. While there is an INSERT/UPDATE waiting to execute, *all* other SELECTS on the same table are frozen and marked as 'Locked' in MySQL's process list until 1) and 2) are complete. Add 100 logged-in users, simmer for about 3 minutes -> MySQL server drops dead with 'Too many connections' (whether using mysql_pconnect() or not). Workaround: under *no* circumstances enable tracker.module on a heavily-used site which gives posting rights to visitors. There is, of course, a fix out there waiting to be written in either the query or the schema, but while I'm OK with adding indexes to tables, multi-table indexing is outside my comfort zone. [cross-posted to tracker: http://drupal.org/node/39351 ] The *good* news: disabling the tracker has reverted server load back down to 0.1 - 0.3 (from >30.00) and MBR is once again serving about 60,000 pages per day without complaint. Serious request for Infrastructure people: this sort of thing doesn't affect many of us, but it kills us when bad things happen. I'm not the first person to ask --- can we please have a "Scaling / Large site issues" forum on Drupal.org to collect war stories? jh

8 10

bzr battle plan
by Karoly Negyesi 30 Nov '05

30 Nov '05

bzr and Drupal. We now have a bzr mirror of the core. It's easier for a group of people to work on a bigger change in their own branch and submit a final diff. (hint. bzr diff --diff-options -F^f) Eventually, Dries may use a bzr branch, too and then instead of patches we may submit a link to a bzr branch instead of a patch file but this is not so important. We will provide and advertise a readonly bzr mirror for all contrib directories. If the module maintainer decides that he he does not want to deal with CVS but uses bzr instead, he will have a checkbox in project module to indicate that (or if we do not want project change, just text that). If it's a bzr maintained module then you can contribute two ways: a) send in patches (like now) b) from the bzr mirror, create your branch and submit a link to the maintainer. When the maintainer wants to commit changes back to the repo, he merges changes and pushes that to a private branch. When this push happens, a simple script is run, which commits these changes back to cvs, and from there we mirror to bzr and the cycle is complete. The problem is -- how we avoid conflicts? Simply, if a module is bzr maintained, then we make it clear that you should not commit into the CVS repository dir by the means of CVS. Either submit a patch (as you do now) or use bzr but do not commit into that dir. Ideally, cvs.drupal.org rejects this kind of commit by the means of a pre-commit script. Lacking that, the bzr to cvs script will take care of it. Then this script needs to track of the revision of the cvs revision of the files in the readonly bzr mirror (first ancestor): # check out the first ancestor cvs co foo -r$known_revision # make sure the cvs is in that state cvs commit # bring in the new/changed files bzr pull $maintainers_private_branch # make sure CVS knows about new files cvs add * # put the log into an obscure named file bzr log $maintainers_private_branch -r -1 >__bzr__log # reuse that for cvs log message cvs com -F __bzr__log Yes, that's the actual script. Advanced versions will be able to branch for Drupal versions. Even that should not be too hard, because the bzr mirrors (including the maintainers_private_branch) will have directories modules/eventrepeat/4.6 -- one dir for every Drupal major version. Updating the readonly bzr mirror is easy: cvs up -dP bzr commit Probably the scripts will be written in PHP and linked through the database -- after all, you need store somehwere the known_revision and maintainers_private_branch. The __bzr__log also should be saved and reused when commit in the readonly space. Or to maintain simplicity maybe we could get away even with some bash scripts which store info in text files. Caveat: I am no expert of that. (note. file additions and removals needs to handled. Maybe tailor will be used here.) If the module is a CVS maintained one, then those that are on the bright side can _still_ use the readonly bzr mirror only they are not submitting bzr branch links but patches. Let's see some benefits: The necessary scripts are simple. If someone wants to run cvs update -- (s)he can do that. If a module/theme/translator maintainer is sick of cvs -- there is bzr. Also, a web upload interface is coming for bzr. If a module/theme/translator maintainer wants to use cvs -- (s)he does not need to face bzr yet his/her contributors can use that and submit patches. What's lost? If a module is bzr maintained then you can't just walk in and do an update of the module. The maintainers' responsibility is bigger. Solution? Multiple maintainers with a $maintainers_private_branch for each maintainer. You still can't just walk in, but there always will be someone who can commit your change. Note that this is pretty rare, as most module maintainers do not like if you commit into their directory in the current situation. One more thing: there should be another privilege level, they can commit against any module. Note that we do not need to change _anything_ in the modules currently used Drupal.org! Still CVS is the common denominator This is a plan which needs time. Also, bzr itself needs some time. I do not plan on posting the whole Drupal community through drupal.org until bzr 2.0 is out next spring.

17 37

creating forward compatable modules?
by Tao Starbow 29 Nov '05

29 Nov '05

Hi Drupal devs, First, thank you all for an amazingly useful tool. I am a programmer at the CITRIS center at UC Berkeley. We recently finished moving our own web site (http://www.citris-uc.org) over to Drupal and I am now working on the infrastructure to allow us to start providing Drupal-based sites for more of the campus community. So far I have been able to do everything I need with the available modules and a bit of hacking, but I am now starting to create custom modules. I was wondering if it is possible to create modules that will be compatible with both 4.6 and 4.7? Also, does anyone have advice on how to calculate how many Drupal sites a server can handle? Installing new hardware can take months around here, so I need to stay well ahead of the curve. thanks, Tao Starbow CITRIS

13 22