July 2006 - Development - lists.drupal.org

Re: [development] serialized data in the DB Re: [support] signup_form_data
by Chad Phillips -- Apartment Lines 13 Jul '06

13 Jul '06

> Database *friendly* would be making a column for Name and a column for > Phone so that other tables in the database could do things like joins > on the data. unless you don't know what the column names are, or how many columns there will be, which is the case with the example you're using. > My perspective is clouded by being more comfortable on a sql> prompt > than in a PHP file, but serialized data in the DB makes me shudder > nearly every time I see it. i've never understood why serialization gets such a bad rap. it's just merely one technique that has it's own particular advantages and disadvantages--retrieval of data via sql queries being a weakness in this case... :) but it also allows for a certain on-the-fly flexibliliy and ease of use that trying to table-ize everything doesn't. > Sorry for the relatively OT rant, but while there are shortcut > benefits on the PHP side to doing things like serializing data and > storing it in the DB it can lead to inefficiencies and handcuffs in > the long run. Databases tend to go really slowly for queries that > look like... agreed. but also, some implementations (like the signup form when it was designed) aren't focused on that kind of data retrieval, and have their own needs. in the case of the signup form, i needed a straightforward way to store themable form data for later display, so serialization was a logical choice. my view is that every tool probably has an appropriate situation where it's useful, including the much-maligned serialization of data... :)

1 0

Question about Drupal ACLs
by Nick Whalen 13 Jul '06

13 Jul '06

Hello, I'm new to the mailing list so I'm sorry if this has been covered before. I'm curious to see if there are built in mechanisms or maybe other modules that will allow me to assign ACLs to specific nodes. In particular I want to show or hide menu items and restrict access to their linked nodes based on a user's role. So far I haven't found anything (that works) fitting this description in Drupal or it's modules. So my question is: Did I miss it, and if I didn't would people in the community be interested in a module that provided this level of access control? Thanks for your time! -- Nick Whalen <nick(a)gottathink.com> Gottathink, LLC

8 14

serialized data in the DB Re: [support] signup_form_data
by Greg Knaddison - GVS 13 Jul '06

13 Jul '06

On 7/13/06, Morbus Iff <morbus(a)disobey.com> wrote: > > a:2:{s:4:"Name";s:6:"brenda";s:5:"Phone";s:10:"5555551234";} > > That is what is called "serialized" data, generated by serialize(). More > documentation at http://us2.php.net/serialize (read the comments for a > more verbose explanation). The above is equivalent to this PHP code: > > array('Name' => 'brenda', 'Phone' => '5555551234'); > > only in a database-friendly format. It's in a format that can be stored in a database, retrieved, unserialized, and used in PHP, but I wouldn't call it "database friendly". Maybe database storable. Database *friendly* would be making a column for Name and a column for Phone so that other tables in the database could do things like joins on the data. My perspective is clouded by being more comfortable on a sql> prompt than in a PHP file, but serialized data in the DB makes me shudder nearly every time I see it. Sorry for the relatively OT rant, but while there are shortcut benefits on the PHP side to doing things like serializing data and storing it in the DB it can lead to inefficiencies and handcuffs in the long run. Databases tend to go really slowly for queries that look like WHERE field LIKE '%Name%\'Brenda\'%' Which is both slow and can generate false matches if the serialized array is sufficiently large. Or a clause like some_table INNER JOIN signup_form_data on some_table.Name = RIGHT(LEFT(field,index(field,'\',\'Phone),len(LEFT(field,index(field,'\',\'Phone))-index(LEFT(field,index(field,'\',\'Phone),'Name\'=>')) Which is slow, can generate false matches, and is relatively obfuscated for what I'm trying to do when compared with a query like: some_table INNER JOIN signup_form_data ON some_table.Name = signup_form_data.Name These query-clauses probably fail miserably when handed to a sql parser, but the idea is clear: serialized arrays are expedient for PHP programmers, but they are database unfriendly. And I'm not calling you out on this, Morbus - you gave a perfect and helpful explanation for the question. I just want to point this out as a sort of motivation to everyone to strive to use better data storage than serialize($foo). Regards, Greg -- Greg Knaddison | Growing Venture Solutions Denver, CO | http://growingventuresolutions.com Technology Solutions for Communities, Individuals, and Small Businesses

1 0

Re: [development] Caching, caching, caching...
by Ken Rickard 12 Jul '06

12 Jul '06

All- In our large Savannah project (http://new.savannahnow.com) we have been running some tests with both Jeremy's file-based cache and with object-based caching. Some notes. 1) I like the filecache patch and have privately sent Jeremy some 4.6 code that remove tthe *wildcard delete barrier to its usage. The solution is to create unique folders for each cache type (variable, filter, page, et. al.) and then purge that folder when sending cache_clear_all('filter', TRUE). Happy to share the code with others. 2) I was under some internal pressure to reduce the sheer volume of db queries on a page. Object caching allows that. In production right now, we cache the following objects: $node $user $block $profile In the cases of $node, $profile $user, we intercept the _load() function and check to see if a cached version of the object exists. It it does, we load it and skip entirely the hook_user hook_nodeapi hook_load steps of the code. If it doesn't exist, we load the object normally and then cache the result. In order to make this work, we also have to know when to flush the individual object cache when a user makes a change. We flush on node_edit, block_save and user_edit, for example. (Apologies for not having good benchmarks, but if memory serves, it increased our page load speed about 15-20% and took 60-70% of the load off MySQL). In the case of a user page, it cuts our queries from around 140 to 25 or fewer after the initial page view. Some problems / issues / goals going forward: 1) Definitely a cache API. Many contrib modules (and some 4.6 core) use cache_clear_all() far too loosely. As a result, we flush the cache far too often. 1a) In fact, the only way to clear a cache of any sort is with cache_clear_all($cid) or cache_clear_all($type, $wildcard = TRUE) or cache_clear_all(). We need more than one function in order to be more clear about what the code needs to accomplish. The block.module, for instance, runs cache_clear_all() on 3 occassions. But it never really intends to clear the entire cache, does it? 2) I'm not sure that I have properly identified all the times to clear the object cache. 3) Some of this is made unnecessary by the fastcache or page cache, but an object cache applies to logged in users as well. For that reason, even caching the $user object would be a benefit. 4) I am more than happy to contribute notes and code to getting this into 4.8. The work I've done is all in 4.6.x, but the approach can apply to CVS. -- Ken Rickard agentrickard Message: 5 > Date: Wed, 12 Jul 2006 10:21:49 +0200 > From: Dries Buytaert <dries.buytaert(a)gmail.com> > Subject: Re: [development] Caching, caching, caching... > To: development(a)drupal.org > Message-ID: <DB3DEF7E-FA41-4AD3-A210-C177CA4B2853(a)gmail.com> > Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed > > > On 12 Jul 2006, at 00:01, Gerhard Killesreiter wrote: > > Now the question is if we want to go down this road. To properly > > invalidate the cache requires to implement all sorts of hooks (forum > > module needs _comment and _user for example) which you would not need > > otherwise. This complicates the code quite a bit. > > Personally, I'd not optimize with MySQL's query cache in mind. If > the folks at MySQL improve their query cache implementation, our > efforts might have been moot. I'd focus my energy on high-level > improvements, rather than low-level improvements. That is, I'd focus > architectural changes that reduce the number of SQL queries or that > simplify complex queries (eg. less joins). For example, the node > table and node_comment_statistics table have a one-on-one relation > and could potentially be merged. I'm not sure what it would buy us, > but it might be worth it. > > Maybe we should run the modified devel.module on drupal.org to see > what the most expensive queries are? You know, accumulate results > over multiple page views. Do we have recent data query data from > drupal.org? Let's start by identifying some key bottlenecks. > > -- > Dries Buytaert :: http://www.buytaert.net/ > > > > ------------------------------ > > -- > [ Drupal development list | http://lists.drupal.org/ ] > > End of development Digest, Vol 43, Issue 38 > ******************************************* >

1 0

Re: [development] Question about Drupal ACLs
by Nick Whalen 12 Jul '06

12 Jul '06

On Mon, 2006-07-10 at 21:31 -0400, Khalid B wrote: > Did you try simple_access? > > It allows access by role on the node level, via one level of indirection. Yes I did, however it didn't meet my primary criteria of being able to hide/show certain items on the primary nav menu based on what role of user was logged in. Any other suggestions? (Sorry it took so long to reply, been busy ;))

3 2

modules into separate directories
by Dries Buytaert 12 Jul '06

12 Jul '06

Hello world, I just moved all core modules into their own directory (using Derek's excellent scripts). This will help pave the path for the install system, as well as a number of other changes. If you run into problems, please report them at http://drupal.org/ node/64280. Thanks! -- Dries Buytaert :: http://www.buytaert.net/

1 0

Patches
by Dries Buytaert 12 Jul '06

12 Jul '06

Hello, here is a list of the top-5 patches I'd like us to see focus on: 1. Simplified CCK into core: http://drupal.org/user/15277 2. Clean up settings page: http://drupal.org/node/72284 3. Reorganize administration pages: http://drupal.org/node/72079 4. The template work that Adrian and Vlado have been working on: no patch yet 5. A new core theme: no patch yet More or less 1.5 months to go until the code freeze. In practice this will be less, because it is impossible to commit 5 big patches during the last week. And even if it is possible, I'm not likely to do that. -- Dries Buytaert :: http://www.buytaert.net/

23 38

SQLite backend
by Sumit Datta 11 Jul '06

11 Jul '06

Hi there, I have been trying to work on an SQLite backend for the last 6 hours or so ... I have made database.sqlite.inc It should work all OK ( except of maybe minor syntax errors ) but i dont know what to do after this ? SQLite is embedded so doesnt need user/pass type login... neither does it need connect to a database... its embedded in PHP. the Admin simply provides a file path which contains the db. So simple. But the existing Drupal db system is with a user/pass login style... I need some help in this spot... -- Thanks =Code is Poetry= Sumit Datta

2 1

[develoment]help required
by dhanesh andurlekar 11 Jul '06

11 Jul '06

hi all i am new to drupal and currently working on to develop a registration module for an online university sort of thing.. achievements till now: i have been able to put in the fields and all required to register online. providing the administrator with the flexibility to choose the fields that he wants. problem statement: i want to link the registration module with the user module so that when the user registers with a username and password he should be able to log in using the same username and password... i am attatching the code which i have written till now...criticism is welcomed as i am very new to all of this...kindly put in your comments as well as suggestions the code is pretty bad as of now as it is in a very initial stage....sorry.... i would also like to release this module to the community if at all i get it working thanking you in anticipation dhanesh p.s the attatched file just contains the code and not the database schema....

2 1

Your dreams coming true
by Karoly Negyesi 11 Jul '06

11 Jul '06

There was so much talk about what can be achieved with install profiles. I need actual bug reviews in http://drupal.org/node/68926 constantly. As I have written in the issue, I am putting an hour every day into the patch and there will be a new version pretty often. Ride with me and there will be a Drupal installer version sooner than you think. Regards NK

1 0