August 2006 - Development - lists.drupal.org

RFC: project releases as nodes
by Derek Wright 02 Aug '06

02 Aug '06

in my continuing efforts to get real release versions for contrib modules[1] and my work to split all the issue tracking functionality out of the project module[2], i came upon what i believe is a really good idea: http://drupal.org/node/75053 "make releases into real nodes?" this would solve *so* many problems, and provide so much flexibility for how to present releases. a summary of some of the cool stuff we'd basically get for free or for very little work: - RSS feed for new releases (something Heine wants for security announcements in contrib) - ability to unpublish releases - ability to promote a release (w/ title and release notes) to the front page of your site (e.g. when drupal 4.8.0/5.0.0 comes out) - block of "most recent releases" is trivial - releases show up in "recent posts" listing for free - taxonomies that apply to releases - other contrib modules (via hook_form_alter() and hook_nodeapi()) could extend the base release node type to add other cool functionality or site-specific customizations. - could easily enable file attachments on release nodes for sites that want to upload releases that way (instead of the wonky hard- coded "releases" directory behavior now) - comments on releases? - cool themeing stuff? - node privacy/ACLs on releases? - views-based display of releases? ... the only possible downside i can see is that it would potentially generate a lot of nodes, though i'm guessing nothing like the rate of new forum posts, for example. note, there would NOT be a whole new release node for each nightly development tarball "release". see my comments in #75053 for more thoughts on this. anyone have any strong objections? any other opinions or bright ideas? you have about a day or two to comment in http://drupal.org/ node/75053. after that, merlin and i are plowing onward, full steam ahead. ;) thanks, -derek (dww) [1] http://drupal.org/node/58066 [2] http://drupal.org/node/74995

1 0

possible drupal.org optimization?
by sime 02 Aug '06

02 Aug '06

FWIW, I'm pointing this out because the topic of optimization is always popular. I assume the patch is easy for anyone with decent DB skills. http://drupal.org/project/comments/add/76040 sime

1 0

Module: User name based redirection
by Khalid B 01 Aug '06

01 Aug '06

This project http://drupal.org/project/userpath duplicates a portion of what pathauto does. This is not necessarily bad, since someone my want just redirects, but then they can install pathauto and disable the node aliasing.. The user, cmb, did not enable contacts. Hope he is on this list.

4 4

OpenID
by Dries Buytaert 01 Aug '06

01 Aug '06

Hello world, take a look at http://iwantmyopenid.org/. "Soon we will be granting $5,000 USD directly to ten open-source projects that successfully implement OpenID." I talked to Scott Kveten (CEO of Janrain) and Corey Shields (OSL) and they would be willing to wire the $5,000 USD directly to OSL so they can buy us an additional machine, something which we might need in the near future. I think this makes for a good opportunity to implement OpenID into Drupal core, and to get rid of our own distributed authentication framework. We talked about this earlier, and I believe there was a consensus about the fact we should adopt a standard. I believe that OpenID is they way to go so even without a bounty it would be of "strategic importance". Thoughts? -- Dries Buytaert :: http://www.buytaert.net/

10 10

Cpu usage: something is very wrong (caching).
by Peter Wolanin 01 Aug '06

01 Aug '06

In response to Augustin, I don't think there is any way around the complete invalidation of cached pages when a node is added, since a single node may be displayed in part or whole on any or all of the site pages. Seems like object-level caching needs to be the next step: http://drupal.org/node/74020 and having a sensible API would make it easier for modules to appropriately invalidate the cache, set a time limit on the cache, or prevent objects from ever being cached. >Date: Tue, 1 Aug 2006 22:31:14 +0800 >From: "Augustin (Beginner)" <drupal.beginner(a)wechange.org> >Subject: [development] Cpu usage: something is very wrong (caching). > >I just reviewed the Caching, Caching, Caching thread, but I didn't see that it >addressed an issue that I recently learned about and that I find somewhat >shocking: the whole {cache} is being emptied whenever a node is created or >edited, meaning that even a medium-traffic website with many nodes but only >one or two added every day will see a very high cpu usage, because the 100s >of nodes will always be purged from {cache} before Drupal has a chance to >serve the cached page to a second visitor. > >This approach defeats the whole caching purpose.

1 0

Node Rendering: the next generation
by Jeff Eaton 01 Aug '06

01 Aug '06

Last week I blogged about complex node theming, and underlying problems in Drupal's node rendering system (ie, node_view()) that are brought to the surface by CCK. Currently, complex node rendering breaks in the the same ways that our 4.6 era form building mechanism broke. We've talked about it before on this list, though 4.7 was still getting rolled up and no one wanted to tackle another change like that at the time. Our current form rendering system was originally designed to support not JUST forms but any kind of content. I've created a patch with help and advice from dopry, RobRoy, chx, and a few others that leverages the advanced form rendering system to build node bodies. The change would force some (very minor) changes to every module that implements hook_view and nodeapi's view operation, but the payoff for themers and designers of complex modules would be considerable. Reshuffling the contents of a node, inserting content *between* existing node elements, and different theming of different parts of a node's content -- stuff we now take for granted with forms -- would all be possible. http://drupal.org/node/74326 is the location of the patch, and anyone willing to chip in with reviews, testing, or thoughts on this approach... well, the input is greatly appreciated. --Jeff

1 0

Views argument builder
by Sam Tresler 01 Aug '06

01 Aug '06

Hi, My first post to the mailing list here. I'm thinking about starting development on a module that can create forms that in turn create a URL to feed to an existing View. A good example would be a classifieds ad setup where the Content Type is Advertisement with various fields, and the View is Search_Results, referencing those fields as needed. This module would allow an admin to create a form in which he could specify values (via checkboxes or dropdown) that, upon completion by a user, are handled in Views argument handler. So, for example, a Price field, would take the arguments max_price and min_price to return ads in a users price range. As I mentioned, I'm new, so any advice or alternate suggetions are very welcome. Mostly, I want to know if this is something people want? Thanks in advance. --Tresler

2 2

Re: Database queries on drupal.org
by Ken Rickard 01 Aug '06

01 Aug '06

I have rerolled the path_alias lookup patch, this time with a working admin. (I blame last night's failure on free OSCon beer!) This patch allows admins who have enabled the path module to go to admin/path/configure And select paths to allow lookups for from a checkbox list. Any paths not selected will not be invoked during drupal_lookup_path., These requests are intercepted within the url() function. The issue is described here: http://drupal.org/node/75976 Again, this is a proof-of-concept patch, designed to see if it buys us any performance gains. I do need some larger-scale testing, if anyone is willing. - Ken Rickard agentrickard

1 1

Cpu usage: something is very wrong (caching).
by Augustin (Beginner) 01 Aug '06

01 Aug '06

Hello, See below for some data from my live site. I just reviewed the Caching, Caching, Caching thread, but I didn't see that it addressed an issue that I recently learned about and that I find somewhat shocking: the whole {cache} is being emptied whenever a node is created or edited, meaning that even a medium-traffic website with many nodes but only one or two added every day will see a very high cpu usage, because the 100s of nodes will always be purged from {cache} before Drupal has a chance to serve the cached page to a second visitor. This approach defeats the whole caching purpose. I may have missed other threads addressing the issues I mention below. Please accept my apologies if I did. Below is the kind of cpu usage I get for my sites. I must give you some background information first, so that you can understand the figures. I am hosted at a nice little hosting-cooperative (i.e. I am host, and hosted, I "own" part of the hosting coop -- en français: http://ouvaton.coop/) I think there are about 5~6000 sites hosted, and I have two sites there. My sites are very low traffic (~20 pages views a day, only). Almost all CMS are represented in those 4000 sites (phpBB, MediaWiki, phpNuke, you name it...). A large proportion of sites use Spip, though, which has an excellent on-file caching method (bypassing php and SQL). Drupal in under-represented (I wonder if I am the only one using Drupal, there). Now, I have an interesting bit of data in my panel, which is the CPU usage, the Bandwidth, and the number of hits IN PERCENTAGE of the total of all the sites co-hosted there. What I found out is that since the beginning, my CPU usage is way above the average compared to the other sites. Here is what I have today for one of my two sites (I have very similar figures for the other site): (use fixed fonts) --------------------------------------------- | For the week | For the day | | rank - % | rank - % | --------------------------------------------| cpu | 127th - 0.153% | 38th - 0.425% | hit | 538th - 0.032% | 577th - 0.030% | Bandwidth | 449th - 0.041% | 496th - 0.037% | --------------------------------------------- I don't have any gallery nor much graphics, so it could have explained the low Bandwidth ranking, but it is in par with the Hit ranking. The CPU ranking however is constantly one order of magnitude above the average of the other 5-6000 sites. What's worse, the CPU ranking is very low for the last day. There is not new content every day, and I observe such a spike each time content is added. Out of the 576 sites that have a HIGHER hit figure than my site, 539 sites needed LESS CPU power than I did! Now, I understand why the cluster of high-end drupal.org servers is having troubles to keep up when a new node is created every couple of minutes! I have observed that my own stats have been getting worse and worse as the number of nodes increases. At the beginning, with a dozen nodes or so, the CPU usage never got very high, because the ratio of pages served from the cache was very high. Now, with a meager ~150 nodes, I find out that my CPU usage never goes below a certain level because all nodes have to be recomputed every couple of days, whether they have changed or not. While I am looking forward to have more visitors, and especially more contributors, I shudder at the thought of the potential cost in CPU power. The formidable strength of Drupal is its flexibility. Everything (everything!) can be customized, down to form elements, and link attributes! There is no HTML in core and everything is abstracted. Of course, it comes at a price, which is the price of CPU computing power. Instead of printing a link(<a href="">click</a>) directly, Drupal has to parse large arrays, check the hooks, etc... before printing this simple piece of html. Same for forms and everything else... I would have thought that this was acceptable if the computing was done once and the cache served many times, if not from file, at least from the DB. The algorithm can be improved in places, yes. SQL can be streamlined, of course. But all of this still require CPU crunching. So, I would like to officially join the ranks behind Gerhard for an improved caching system. I also think we should seriously consider a file-caching mechanism (there was a patch proposed and used on a site, using mod_rewrite magic... we could think of a per-block file/DB caching for those blocks that are often updated --e.g. active forum topics-- while block rarely changing -- e.g. primary links, syndicate block, etc. -- can be hard coded within the cached page.) The bottom line is that the cache should be more durable, and the ration page-served-from-cache/page-computed-on-the-fly should be much higher than what it is now. What can I do? I am not a coder as experienced as you are, so I don't really know where to start. But if someone makes a start, I can follow the issue and test the patches. Once a fairly complete and stable patch comes to light, I can test it on my live site, and see what difference it makes when comparing my own little Drupal site against the thousands of non-Drupal sites hosted at the same place. Also, why not introduce some amount of caching (at least for some blocks, or the body of old nodes) for registered users? yours, Augustin. P.S. If you think your site's users would be interested, I wouldn't mind a charitable plugin to my site wechange.org :) -- http://www.wechange.org/ Because we and the world need to change. http://www.reuniting.info/ Intimate Relationships, peace and harmony in the couple.

3 3

Administrator locale aware watchdog logs
by Gabor Hojtsy 01 Aug '06

01 Aug '06

Hi, http://drupal.org/node/76588 The problem with the current watchdog() calls is that the messages are stored in the actual locale used to present that page. Ie. if you have a German, French and English section (version) of your site, watchdog logs will be recorded in all three languages intermixed with each other. Administrators reviewing logs are supposed to understand all languages used on the site to understand possible problems. This is not the desired user experience. To make the watchdog logs admin locale aware, we need to postpone the t() call to the time the log is presented to the administrator and should not t() the string on storage time. This also means that we should store the message in English, and the variables separately in a different field. I have chosen serialize()/unserialize() to store the array. We should note that for backward compatibility the previous watchdog messages should have an 'N;' value in the field (the serialization of NULL). This needs an upgrade path, which is not yet implemented in the patch, but is quite simple. Some messages in system.install for example also log stuff which is user defined, so t()-ing it is not desired. With specifying N; in the variables field, it is possible to keep those away from t(). Pros of this approach: * Actual locale used to present messages to the administrator * Messages stored before any locale is available (ie. install time messages) are later translated so easier to review and understand Cons: * Tablesorting of watchdog messages on the admin interface is not directly done using the displayed value. So eg. 404 messages with different file names will not get sorted by the filename, which is only added after the tablesort is done. * Slightly more storage requirement for the watchdog table. In my patch, I have updated the watchdog() function used to store messages and the two watchdog review interfaces in watchdog.module. As an example, I also updated the user.module (also fixing some missing theme placeholder calls). This is not a complete patch, updating of other watchdog calls and the upgrade path is also required. /Since this change requires all watchdog() calls changed to run without errors, I have not tested the change in runtime. First I would like to gather opinions on the viability of the patch to put more work into it./ Please review and comment on the issue! Gabor

1 0