- Development - lists.drupal.org

"filebrowser", "s5 player", "series" and "taxonomy html" for adoption
by Gabor Hojtsy 07 Apr '07

07 Apr '07

Hi, To get a hold of the modules I maintain, and also keep up with my changed focus, I am announcing the following modules being up for adoption: http://drupal.org/project/filebrowser A nice FTP like interface for file browsing (originally developed for Drupal.hu and used as a translation browser) Hint: also look into the "filebrowser addons" project which is separately maintained, there are cool ideas around here for an "asset manager" kind of functionality. http://drupal.org/project/s5 Possibly a module suite, but now only one module to export book pages as S5 presentations (it is cool, I did present with this once :) http://drupal.org/project/series A very simple one-purpose module to display a list of node links (possibly an article list) as being in a serial. http://drupal.org/project/taxonomy_html Yes, this is it! Originaly by Moshe Weitzman for Drupal 4.0, this was started nearly 5 (yes, five) years go. A note from the original commit message: "Think of taxonomy.module as the API or engine and this module as a UI generator." This means that taxonomy_html actually displays lists of terms in various ways. I know, I know Views does this and a lot more... Taxonomy_html is a simple, small footprint module with a great history ;) Most of these modules are quite simple, have only a small number of issues attached in their queue, so further maintenance for interested parties should not be a hard task. Unfortunately I am not going to use any of these modules in production in the foreseeable future, so I would better hand them on to someone else, instead of orphaning them right away. Gabor

1 0

PDO and database limitations
by Larry Garfield 06 Apr '07

06 Apr '07

This is an RFC of sorts. :-) At Drupalcon, I was shopping around the idea of a PDO[1] backend for Drupal. For brevity, I'll just reference the blog entry I made on why I think PDO is a good idea[2]. The goal is that PHP 5 users get a nice speed bump and we get a start on eventually shifting Drupal's database layer from the 1997 APIs to the more robust 21st century PHP standards. Most people I talked to liked the idea, so after getting a nod from Dries I've been working on it since the Hackfest and made decent progress. However, I've run into a few snags I want to get a 2nd opinion on (or 3rd, or 4th, or whatever). 1) It turns out that PDO from PECL running under PHP 5.1.6, at least, has issues. Specifically, segfault issues on otherwise perfectly sane queries. This is apparently a known issue with 5.1.6, and one of the reasons 5.2 exists. :-) At present, I have no solution for it other than saying "well, the PDO support is only if you're running 5.2, otherwise use the existing MySQL/PostgreSQL drivers, deal". Does anyone have a problem with that, and if so, an alternate solution? 2) Like any wrapper, PDO, while it offers some really nice features with a common API (like C-level prepared statements, which are what I'm mainly after), has some "lowest common denominator" issues. The main one I've run into so far is that there is no reliable equivalent of mysql_num_rows() for SELECT statements, only for data-changing statements[3]. In testing it doesn't look like the MySQL PDO driver returns anything useful for rowCount() on SELECT. That gives us 3 options. A) Stop using db_num_rows() on result sets. (It's a database-specific feature in the first place.) In places where we use it, use a separate count(*) query instead. That's the more database-agnostic method, and is what the PHP manual recommends[3]. If we go this route, it will involve removing the db_num_rows() function from the existing mysql and postgres drivers and refactoring core queries accordingly. B) Instead, have the PDO wrapper do a full ->fetchAll() on the result set. That gives an array of array or object records (specified in the fetchAll()), which can then be simply sizeof()ed in the PDO/db_num_rows() implementation. The downside here is that you need to specify in the fetchAll() whether you want objects or arrays. To not horribly break any existing APIs (I'm trying to minimize the footprint for now; we can break everything later), we'd need to therefore fetchAll() as an array in the PDO driver and then cast back to an object in db_fetch_object() (or vice versa). That feels quite nasty to me, honestly, and is probably a not-unnoticeable performance hit. C) Drop the PDO idea. It should come as no surprise that this is my least favorite option. :-( As my goal is that in some mythical future when we can require PHP 5 PDO becomes our main database backend (and we can leverage it even more for a cleaner API, faster/more stable prepared statements, etc.), I would favor slimming down and generalizing our database calls now (option A) so that we're better prepared in the future. (It would also make adding support for other database engines like Oracle or MS SQL easier.) I am, however, open to input and suggestions. I am hoping I can get this into Drupal 6, but time will tell if I can get it ready in time. [1] http://www.php.net/pdo [2] http://www.garfieldtech.com/blog/drupalcon-php5 [3] http://us.php.net/manual/en/function.PDOStatement-rowCount.php -- Larry Garfield AIM: LOLG42 larry(a)garfieldtech.com ICQ: 6817012 "If nature has made any one thing less susceptible than all others of exclusive property, it is the action of the thinking power called an idea, which an individual may exclusively possess as long as he keeps it to himself; but the moment it is divulged, it forces itself into the possession of every one, and the receiver cannot dispossess himself of it." -- Thomas Jefferson

9 11

Theme registry
by Dries Buytaert 06 Apr '07

06 Apr '07

Hello world, the theme registry patch has been committed to CVS HEAD. This will make it a lot easier to theme Drupal 6 websites. Woot! Woot! :-) Thanks Earl et al, -- Dries Buytaert :: http://www.buytaert.net/

1 0

drupal hook overrides
by Ashraf Amayreh 06 Apr '07

06 Apr '07

Hello everyone, I've just posted a patch for a hook override feature that I'd like to take your feedback on. You can find the issue at this link http://drupal.org/node/134148 Thanks!

1 0

Re: [development] PDO and database limitations
by Larry Garfield 04 Apr '07

04 Apr '07

On Wed, 4 Apr 2007 11:45:13 -0400, "Khalid Baheyeldin" <kb(a)2bits.com> wrote: > On 4/4/07, Larry Garfield <larry(a)garfieldtech.com> wrote: >> >> This is an RFC of sorts. :-) >> >> 1) It turns out that PDO from PECL running under PHP 5.1.6, at least, > has >> issues. Specifically, segfault issues on otherwise perfectly sane >> queries. >> This is apparently a known issue with 5.1.6, and one of the reasons 5.2 >> exists. :-) At present, I have no solution for it other than saying >> "well, >> the PDO support is only if you're running 5.2, otherwise use the > existing >> MySQL/PostgreSQL drivers, deal". Does anyone have a problem with that, >> and >> if so, an alternate solution? > > > Which distro? > > I use Ubuntu 6.10 and if you install PHP 5, 5.1.6 is the default you get. > This makes > it harder to use 5.2 (custom compile, ...etc.) > > If you have working code, I can verify whether I have the same problem or > not on > my test server. I also run Ubuntu Edgy (6.10), which is how I ran into this problem. After spending 4 hours trying to figure out why I was segfaulting only sometimes on a session-load query, I checked with the php-general development list and was told that 5.1.6 had a lot of segfault issues. I found a 3rd party repository and installed 5.2.1 and the segfaults all went away. > 2) Like any wrapper, PDO, while it offers some really nice features with a >> common API (like C-level prepared statements, which are what I'm mainly >> after), has some "lowest common denominator" issues. The main one I've >> run >> into so far is that there is no reliable equivalent of mysql_num_rows() >> for >> SELECT statements, only for data-changing statements[3]. In testing it >> doesn't look like the MySQL PDO driver returns anything useful for >> rowCount() >> on SELECT. That gives us 3 options. > > > In core HEAD, there are 43 occurrences in 25 files. Haven't checked > contrib. > > includes/common.inc: > includes/database.mysqli.inc: > includes/database.mysql.inc: > includes/database.pgsql.inc: > includes/locale.inc: > includes/session.inc: > modules/aggregator/aggregator.module: > modules/block/block.module: > modules/blog/blog.module: > modules/book/book.module: > modules/comment/comment.module: > modules/drupal/drupal.module: > modules/forum/forum.module: > modules/node/content_types.inc: > modules/node/node.module: > modules/path/path.module: > modules/ping/ping.module: > modules/statistics/statistics.module: > modules/system/system.install: > modules/system/system.module: > modules/taxonomy/taxonomy.module: > modules/user/user.module: > > A) Stop using db_num_rows() on result sets. (It's a database-specific >> feature >> in the first place.) In places where we use it, use a separate count(*) >> query instead. That's the more database-agnostic method, and is what > the >> PHP >> manual recommends[3]. If we go this route, it will involve removing the >> db_num_rows() function from the existing mysql and postgres drivers and >> refactoring core queries accordingly. > > > My concern here is that all db_num_rows() cases become two queries for > people using MySQL (the vast majority) as well as PostgreSQL (it has > pg_num_rows()). Not all, actually. At least one (in sess_write()) really should have been a count query in the first place as it's never iterated. I'm not sure how many cases will simply become 2 queries and how many will be able to stay as one with some logic refactoring. I don't think I'll know that until I start doing so. How do the MS SQL and Oracle implementations deal with these issues? Another goal of the PDO implementation is to make adding support for more databases easier, since there's a smaller API footprint to deal with. > If these are index only queries, then it may not be a big deal, but > anything > more > (which we have a lot of), would incur a significant performance penalty. > It > also > affects large sites using InnoDB which is notorious for very slow SELECT > COUNT(*). > > See a benchmark here: > > http://2bits.com/articles/mysql-innodb-performance-gains-as-well-as-some-pi… Wouldn't InnoDB be just as slow running a full select statement and then doing a num_rows() on it as it would be doing a COUNT() query in the first place? It still has all the WHERE-ing to do. More specifically, MyISAM is a special case where count(*) is faster than it otherwise would be in any other situation. It's not that InnoDB is slow, it's that MyISAM is fast. :-) Still, though, I suspect that at least some cases will not get hosed if we refactor properly; some don't need to be full normal queries in the first place, and I am not opposed to adding additional indexes where appropriate. > Not sure what else can be done here. Is there a way to sneek in a pass > thru > function > (mysql_num_rows() in this case) like ODBC for example allows? I'm not sure what ODBC does. Can you elaborate? I'm pretty sure one can't call mysql_* functions on a PDOStatement object. > B) Instead, have the PDO wrapper do a full ->fetchAll() on the result set. >> That gives an array of array or object records (specified in the >> fetchAll()), >> which can then be simply sizeof()ed in the PDO/db_num_rows() >> implementation. >> The downside here is that you need to specify in the fetchAll() whether >> you >> want objects or arrays. To not horribly break any existing APIs (I'm >> trying >> to minimize the footprint for now; we can break everything later), we'd >> need >> to therefore fetchAll() as an array in the PDO driver and then cast back >> to >> an object in db_fetch_object() (or vice versa). That feels quite nasty > to >> me, honestly, and is probably a not-unnoticeable performance hit. > > > I don't think we can do that safely at all. The result set may exhaust > available > memory, or tax other resources if it is rather large. Yeah, I don't like this one either. > C) Drop the PDO idea. It should come as no surprise that this is my least >> favorite option. :-( > > > Option D) : File a feature request against PDO to include a numRows() > method, > but that is not a quick solution. That's the same as C for the forseeable future. Such an addition wouldn't be added until at least PHP 5.3, I imagine, which is who knows when. And even at that, it would require a lot of C-level implementation to emulate num_rows on all databases, whether they natively support it or not. > I am hoping I can get this into Drupal 6, but time will tell if I can get > it >> ready in time. > > > Ubuntu Feisty (due later in April) will have PHP 5.2.1. > http://packages.ubuntu.com/feisty/web/php5 > > But, what about other distros? If they are on PHP4 or 5.1, then this will > be > a show stopper. I'm not deprecating the existing mysql and pgsql drivers at this point. This is strictly a "if you're using a stable version of PHP 5, then you get this extra, faster option" feature. Eventually I hope to deprecate the legacy database drivers and just use PDO with thin wrappers (for db_query_range() and such) for all database stuff, but that's not something we can even consider until 2008 at the earliest. Several web hosts already offer 5.2, though, as do some distros (Debian Etch, Ubuntu Feisty, etc.), so for those people the PDO option should offer a performance boost on prepared statements. --Larry Garfield

2 1

Module maintainer with contact tab off (directory module).
by Augustin (Beginner) 04 Apr '07

04 Apr '07

Hello, I believe it is good practice for module maintainers to have their contact tab ON. I wish to contact the maintainer of directory.module: http://drupal.org/project/directory http://drupal.org/user/60090 Thanks for your help. Augustin. -- http://www.wechange.org/ Because we and the world need to change. http://www.reuniting.info/ Intimate Relationships, peace and harmony in the couple.

3 3

how to intercept user login with cookie
by vitamona 04 Apr '07

04 Apr '07

hi, is there a way for intercept a user when login via cookie? For example, when a user login with a form (no cookie stored) is possible to intercept them with hook user: function hook_user($type, &$edit, &$user, $category = FALSE) { if ($type == 'login') { Thanks a lot Vita

3 3

mysterious search issue
by Alan Dixon 03 Apr '07

03 Apr '07

I think i've just figured out a problem with a site I'm working on and wanted some wisdom from ths list. The site is http://community.telecentre.org/ (not that it matters). The problem: The problem was that the search module's database stopped getting updated (i.e., new material wasn't showing up in searches). I looked at the search_dataset table and discovered that the biggest nid (i.e., sid) was from a node that was published about 9 months ago (hmm, seems like most folks don't believe a site search anyway?). The diagnosis: So, I ran some debugging and discovered that the sql in node_update_index (the one that tells search whether there are any new nodes to spider) was returning no rows all the time, even though there was lots of new content. After struggling with the logic in the SQL, I think I figured out that the problem was a single node which had gotten a date of May 2007 in the created field. I don't think that's normally a problem, but the node_update_shutdown function (which is invoked in case search gets aborted because it runs out of time) saves the system variables node_cron_last and node_cron_last_nid as the current node's created and nid values. Conclusion: I think what happened was that the search indexer got aborted while processing a node with a future date. That inserted a future value into node_cron_last, which means that nodes don't get spidered again until that date. Question: (multiple choice to make it easy ...) 1. is this a problem with the node_update_shutdown logic (or the point in node_update_index when the last_change global gets set for it)? 2. Or is it a bug in the aggregator2 module that creates nodes with 'created' set in the future? 3. Or have i misdiagnosed the problem? 4. All of the above ... Comments: I've heard of other mysterious search indexing failures like this. It took me quite a while to figure out what was going on - the logic in what nodes get spidered is pretty complex. Does anyone have any handy tools for such search problem diagnosis? Sounds like a useful addition to the devel module or as a separate one. Something that can explain how many and which nodes will be spidered by the next cron perhaps ... -- Alan Dixon, Web Developer http://alan.g.dixon.googlepages.com/

5 6

Re: [development] development Digest, Vol 52, Issue 3
by Ken Rickard 03 Apr '07

03 Apr '07

Rob- The question is: "So do you think extending MySite to have a friends' recent activity would make sense? I haven't used MySite at all so don't really know if this would fit or not as I'll want to track other actions like friendships, donations, etc." The answer is yes, but it's the How that we'd need to explore. You might want to send me a note off-list, or open an issue against the MySite module. Relevant questions: - How are you tracking "friendship" now? - What friend actions need to be reported back to the MySite panel? If you go down the wathdog_user, path, MySite integration might actually become easier, since we'd simply write an include to handle the reporting of that data. - Ken Rickard agentrickard

1 0

module announcement: og_sites, providing multisite features
by Nedjo Rogers 03 Apr '07

03 Apr '07

An early preview release of a module that may be of interest to developers and to anyone using multisites. OG Sites enables a multisite configuration in which a central 'hub' site has a number of associated sites. The hub site features content and users from all sites, while each associated site can access only its own data (plus content explicitly shared from the hub site). This is accomplished through organic groups. Each associated site gets an organic group on the hub site. When content is created on an associated site, it gets assigned to that site's organic group on the hub site (and ditto for users). This association is the basis for determining access. Some features that may interest developers, and may be applicable/adaptible elsewhere: * The code is structured to have plugins by module. For example, node support is in node.inc, user support in user.inc. New module support can be added through a new plugin, which will be automatically detected and loaded if the module it implements is available. * Tables are shared dynamically (using hook_init()) rather than through configuration in the settings.php file. * The install functions automate node type and node creation. The og_sites_hub install function creates a new content type, assigns it as an organic group type, and then creates a node for the group. The og_sites install function creates an organic group node on the hub site. * Access is limited through various means: hook_menu() for menu item overrides, hook_db_rewrite_sql() for node access, hook_form_alter() to reset the rows in admin tables. For more information, see the README: http://cvs.drupal.org/viewcvs/drupal/contributions/modules/og_sites/README.… Any reviews or feedback through the issue queue or privately by email would also be greatly appreciated. OG Sites is produced by CivicSpace and development is sponsored by Alpha, alpha.org.

2 1