August 2007 - Development - lists.drupal.org

Seriously disappointed with everyone's involvement
by Karoly Negyesi 29 Aug '07

29 Aug '07

Hi, So I am away from core for like two weeks -- not from Drupal, mind you, I was working on two contrib modules some fierce as part of my dayjob. Now, I am back. There are dozens and dozens of open criticals either with a patch which needs review or even without a patch. Not to mention the incredible pile of normal priority patches needing review. So, I write the menu update. Day and night to make the beta happen as soon as possible. One night I got up at two AM and instead of turning back, I worked four hours on it -- so I could keep in touch with people over the pond. Aside from the people I asked personally on IRC, noone reviewed this. When someone did and the review did not involve actually trying the update itself (I have described the review process in the issue) I got really angry and vent off him -- again, let me apologize to Chris Kennedy. And do you know what the development list is busy with ? Whether uppercase TRUE or lowercase true is faster. Shame on everyone who was on that thread and not doing anything productive. Hint. Drupal releases do not happen out of blue. It involves people working on it. Here is something. Take just 30 minutes out of your day and test one patch. Everyone who reads this. One patch. Not the end of the world. Maybe instead of painting the bikeshed. Do not know which one to pick? Ask the channel. Ask me in email. On any IM you can dream of. I am not hard to find and noone ever walked away with empty hands after asking for a patch to review. Do not even think of answering this letter with angry rants about what kind of person I am (which I am sure I will get) before reviewing one patch. If I need to get attacks, at least something good should come out of it. Karoly Negyesi

1 0

Data import and synchronization to nodes
by Larry Garfield 29 Aug '07

29 Aug '07

Hi all. Algorithmic question. Actually, two questions that I feel have related or complementary solutions, so I am tossing them out together. I have two scenarios I am looking at where I need to be able to pull data in from an external source and make a node out of it, *and* be able to fully update the entire dataset at any time from the external source. Scenario 1: Course listings for a school. Courses come in as a CSV file. The CSV includes duplicate data for courses and sections of courses, as well as fields like the instructor, which is an Instructor node type elsewhere in the system. It is not linked by node ID, however, but by instructor name, and then a lookup needs to be done on import. Typically imports are done a few times a year, but at those times will be done easily a dozen times in a few days. Each import is several hundred records. Scenario 2: A DocBook source tree, such as the one sepeck is building for the new Drupal.org documentation. A given book needs to be made available via Drupal pages, with the structure of the pages on the site matching up with the DocBook tree. Whether pages break at chapters/articles, sections, sub-sections, etc. should be admin controlled. Each book could potentially be hundreds of pages (for some admin-defined definition of page). In neither scenario does the incoming data have any knowledge of "nodes". Also, in neither scenario do I need round-tripping, so if the data is not editable via Drupal, or edits are always overwritten by a new import, that is perfectly fine. Now, the non-node solution to both of these is reasonably straightforward. In the first scenario, simply read course data into two separate tables (one for courses, one for sections) with the appropriate foreign keys, then setup custom menu callbacks for courses and sections and lists thereof that display the desired information. When a new file is imported, flush those tables and rebuild. You probably wouldn't even use auto-increment IDs for them. In the second case, have a single menu callback that corresponds to the root of the Docbook tree. Arguments to that callback map to the structure of the tree. Scan the tree once and build a menu based on the outline, then lazy-load page data and cache it in rendered (via XSLT or whatever) form for later display. If the tree structure changes or the admin changes the page-break settings, flush the cache and rebuild the spine menu. The problem with both of those methods is in neither case is the data a node. That means you do not get any of the benefits of data being nodes (comments, CCK capability, nodereference, a node/$nid "permalink" that doesn't change when the book structure is refactored, Views support, etc.) On the other hand, both cases need flush/rebuild ability. That means creating and destroying nodes by the hundreds on a regular basis -- which would be a very slow operation and would result in the loss of any of that additional metadata -- or building some additional mechanism for tracking what part of the original source data maps to what once-created node. The former is quite undesirable, while the latter is potentially quite complex (especially when, of course, you don't have absolute control over the incoming data so can't guarantee that it has a unique ID you can reference). Since I can think of at least two places I would want to use each of those (Drupal.org being one use case for scenario 2), both seem like natural cases for generalized modules. I want to solve the import/sync problem first, however. I also do not believe that the importexportapi module would be useful here. I looked into it last year for a similar task, and from the documentation determined that it was either too over-engineered or too under-documented for my uses. By the time I figured out how to use it, I could probably just have written it myself. :-/ The DocBook scenario I'm assuming is a Drupal 6-based module only, and PHP 5-only. The courses scenario may be Drupal 5 or Drupal 6, depending on scheduling. So, I toss the brain-teaser out there: Is there a good way to have my nodes and import them too, or are these cases where nodes are simply the wrong tool and the direct-import-and-cache mechanisms described above are the optimal solutions? -- Larry Garfield AIM: LOLG42 larry(a)garfieldtech.com ICQ: 6817012 "If nature has made any one thing less susceptible than all others of exclusive property, it is the action of the thinking power called an idea, which an individual may exclusively possess as long as he keeps it to himself; but the moment it is divulged, it forces itself into the possession of every one, and the receiver cannot dispossess himself of it." -- Thomas Jefferson

3 2

08/28
by Karoly Negyesi 29 Aug '07

29 Aug '07

http://www.nowpublic.com/again-it-seems-some-dmoz-editors-are-corrupt With quite some background info.

2 2

Deleting Cached Permissions
by Ron Parker 29 Aug '07

29 Aug '07

I wrote this list before on this issue: http://drupal.org/node/166557; but, received no response. Decided to re-pose the question. The problem is that I'm getting the "access denied" error on file upload even though the file is uploaded. This is because user_access('upload files') is failing for a user who, in fact, has this permission in this particular group. I did the following: 1. Modified user_access() function to NOT cache the $perm permissions. 2. Deleted records from cache_menu. I don't know which (if not both) of the above did the trick, but it resolved the issue. That means (I think) it's a cache problem. I know I can use cache_clear_all() to delete items from cache_menu. However, I would like know if it's possible to delete/modify the static $perm variable that is set in the user_access() function OUTSIDE of that function? That is: The user_access() function sets a static variable $perm = array(). Permissions are cached in this variable. Can I create a function in my module that can update the $perm variable that is set in the user_access() function of the node.module? Thanks for any assistance. -ron -- Ron Parker Software Creations http://www.scbbs.com Self-Administration Web Site http://saw.scbbs.com SDSS Subscription Mgmt Service http://sdss.scbbs.com Central Ave Dance Ensemble http://www.centralavedance.com R & B Salsa http://www.randbsalsa.com

8 7

Re: [development] Changing Roles (was Deleting Cached Permissions)
by Ron Parker 29 Aug '07

29 Aug '07

On Aug 28, 2007, at 4:04 AM, Karoly Negyesi wrote: > Changing user_access could lead to very obscure and hard to debug > priviledge escalation holes: some code may make presumptions about > if a page is allowed by menu then certain permissions are set which > might not be true if you fiddle with roles on the fly. Saying that > this does not happen currently won't change the fact that it could. Let me again make it crystal clear: The user_access() patch I submitted here http://drupal.org/node/170524 does NOT change user roles or alter current user_access functionality or permissions in any way. It simply resets the permissions calculated by user_access. It clears the cache. AND, it's optional! You have to send the command: user_access('', NULL, TRUE) in order for it to execute, otherwise, user_access is not affected at all. So, if you don't need it, you don't use it. > Just overdefine the menu item that you want to change and define > your access mechanism. Because node/add is defined as cached you > can put your menu definiton in !$may_cache with the same path -- it > will overwrite the original definition. In Drupal 6, you want to do a The OG User Roles (OGR) module basically performs most of the functionality most users need it to do now, and with no patches to the core module, and no "overdefinitions" in hook_menu. It utilizes $user->roles because that's what $user->roles was designed to do: Tell you what permissions a user has. This whole current issue arose because I discovered user_access() was caching permissions and thus causing a problem with OGR and the upload.module. And, there was no way to clear this cache like there is practically everywhere else in Drupal. I further realized that this same issue had caused compatibility problems with the BuddyList, OG Forum and Relativity modules which, as you suggested, I wrote around to resolve. However, at some point, it seems ridiculous to keep trying to write customized code for every module where I have this problem when the simple solution is to reset the cache, a rather harmless option that should be available everywhere that Drupal creates a cache (including user_access()). I know you don't want to support role changes on the fly, but killing Peter to spite Paul doesn't seem like a logical way to do it. -ron -- Ron Parker Software Creations http://www.scbbs.com Self-Administration Web Site http://saw.scbbs.com SDSS Subscription Mgmt Service http://sdss.scbbs.com Central Ave Dance Ensemble http://www.centralavedance.com R & B Salsa http://www.randbsalsa.com

2 1

Re: [development] Changing Roles (was Deleting Cached Permissions)
by Ron Parker 28 Aug '07

28 Aug '07

Let me say it again: OG User Roles does not CHANGE roles. It adds roles based upon the group a user and node is in. For those who can't think of any scenario where a temporary role addition is a good idea, what about this: User 1 belongs to Group A and Group B. In Group A, User 1 is a contributor and can add content. In Group B, User 1 is a member, and can only read content. The way Drupal works now, if you give User 1 "create content" role in Group A, the user will also have the same role and capability in Group B. For that matter, the user will have the "create content" role across the entire site. What OG User Roles (OGR) does is give the user the "create content" role in Group A, but NOT Group B. So, I wouldn't give User 1 a sitewide "create content" role, but a group "create content" role, and only in Group A. OGR doesn't alter sitewide roles, but adds temporary roles based upon group context. In the bigger Drupal scheme, this apparently has no value. But, with respect to Organic Groups and totally autonomous communities within communities, this is the only secure way of achieving that. I can have 100's of groups and subgroups, one or two node types, and use only 4 roles: founder, admin, contributor, member; and, not one member of any group will be able to create or see content in any other group that he/she shouldn't. This is NOT possible now, not without having 100's of roles and 100's of node types. If the Drupal development team doesn't think this is a good idea, so be it. Deleting cached permissions, however, is a separate issue. I still don't see why being able to delete cached permissions is NOT a good idea. -ron Date: Tue, 28 Aug 2007 07:04:57 +0200 (CEST) From: "Karoly Negyesi" <karoly(a)negyesi.net> Subject: Re: [development] Deleting Cached Permissions To: development(a)drupal.org Message-ID: <E1IPtG5-0006zX-9v(a)garm.runbox.com> Content-Type: text/plain; charset="iso-8859-1" I can not imagine any scenario where a temporary role change is a good idea .If it's permanent you can reload Drupal. What are your goals? "use LDAP groups" is a task not a goal. "I want to herd a tribe of gutsy gibbons" now, that's a goal. -- Ron Parker Software Creations http://www.scbbs.com Self-Administration Web Site http://saw.scbbs.com SDSS Subscription Mgmt Service http://sdss.scbbs.com Central Ave Dance Ensemble http://www.centralavedance.com R & B Salsa http://www.randbsalsa.com

7 8

Re: [development] Deleting Cached Permissions
by Ron Parker 28 Aug '07

28 Aug '07

Then, could someone help me with some process or procedure by which I could prevent user_access() from caching $perm or override the results of this caching? Even if I turn Drupal caching off, this function's caching of $perm still occurs, causing my module problems where it shouldn't have them. Thanks! -ron Quoting Ron Parker <sysop(a)scbbs.com>: >> >> That is: The user_access() function sets a static variable $perm = >> array(). Permissions are cached in this variable. Can I create a >> function in my module that can update the $perm variable that is set >> in the user_access() function of the node.module? >> > > A variable declared within a function is private to the function unless the function declares it as global. Earnie -- Ron Parker Software Creations http://www.scbbs.com Self-Administration Web Site http://saw.scbbs.com SDSS Subscription Mgmt Service http://sdss.scbbs.com Central Ave Dance Ensemble http://www.centralavedance.com R & B Salsa http://www.randbsalsa.com

5 7

Re: [development] Deleting Cached Permissions - Why not?
by Ron Parker 28 Aug '07

28 Aug '07

OK, I understand that Drupal doesn't support changing user roles on the fly. Technically, OG User Roles does not change any sitewide roles; it only adds roles based upon the group that an individual node belongs to. But, I won't quibble with details. What I don't understand is why my request to add the ability to reset the user_access() permissions cache was rejected. http://drupal.org/node/170524 The requested code addition does not alter existing permissions in any way. It simply provides a method whereby one can send a command to user_access() to reset the static variable. Can someone tell me why this is not a reasonable thing to be able to do? Thanks! -ron Message: 2 Date: Sat, 25 Aug 2007 10:52:43 +0200 From: Gerhard Killesreiter <gerhard(a)killesreiter.de> Subject: Re: [development] Deleting Cached Permissions To: development(a)drupal.org David Metzler schrieb: >> The permissions (user roles) are being altered temporarily. The reasons >> are documented in the issue Ron has referenced. > > I've said it once and I say it again since apparently nobody noticed: Temporarily changing user roles (per page request) is (currently) unsupported by Drupal. I'd even argue that it shouldn't be supported and that what Ron is doing should be achieved in another way. Cheers, Gerhard -- Ron Parker Software Creations http://www.scbbs.com Self-Administration Web Site http://saw.scbbs.com SDSS Subscription Mgmt Service http://sdss.scbbs.com Central Ave Dance Ensemble http://www.centralavedance.com R & B Salsa http://www.randbsalsa.com

4 3

Re: [development] coding standard question
by andy＠spiders-lair.com 28 Aug '07

28 Aug '07

>>> 1326ms vs. 1382ms is 60ms, and when normal page load times are >>> somewhere around 200ms, it makes a substantial difference. >> >> Even if we disregard the fact that microtime tests have a big stddev, > >We shouldn't do that. The posted results are totally worthless without >an error estimate. > >People, stop wasting everybody's time, read http://drupal.org/node/79237, or preferably an introduction to >statistics and a howto on doing experiments in first year phyiscs, then >re-do the tests. imo the tests are pretty pointless. Given that PHP uses the zend hashtable to store all lists of this type (including constants) then this link shows how the table hash key is generated using the DJBX33A algorithm:- http://lxr.php.net/source/ZendEngine2/zend_hash.h#315 Seems to me that upper or lower case won't have an effect on this (not one that microtime is going to perceive anyhow) so this hardly seems worth following up. If you really feel the "need for speed" then the only suggestion is to keep all constant definitions to 8 characters or less. Hardly practical in the real world. --Andy

4 6

patch needs review
by mark burdett 28 Aug '07

28 Aug '07

Hi, here's a drupal-5 patch that could use a quick review, if anyone has a moment: http://drupal.org/node/170310 quick summary of the patch: If and only if session.cookie_secure is enabled, unique session names will be generated for HTTP and HTTPS sites sharing the same cookie domain, e.g. http://site.example.com/ and https://site.example.com/. For most sites, which don't have session.cookie_secure enabled, there will be no difference in behavior. Until Drupal 5.2, it was possible to set different session names so separate HTTP and HTTPS sessions could co-exist on the browser. This is the user-friendly way to take advantage of session.cookie_secure, while still offering an HTTP url for anonymous visitors. If you use identical session names, as Drupal 5.2 forces you to do, the authenticated HTTPS session will be wiped out from the browser cookie jar if the user visits an HTTP url during her/his session (which could happen easily if e.g. an administrator views a page via normal URL while doing administration work). --mark

1 0