February 2005 - Development - lists.drupal.org

[drupal-devel] New image module
by Chris Cook 16 Feb '05

16 Feb '05

Is there an updated image module floating around that leverages the new image.inc contributed by walkah? The image module in walkah's sandbox still uses the local image includes and would take some manipulation to get working with the new file and Drupal cvs.

3 2

[drupal-devel] Bug#295496: drupal: postgresql interface can't connect to a UNIX domain socket
by Devin Carraway 16 Feb '05

16 Feb '05

Package: drupal Version: 4.5.2-1 Severity: normal Drupal ostensibly supports postgresql, but its use of the PEAR::DB DSN scheme for configuration the database connection is incomplete, and cannot support connection to a pgsql server via a UNIX domain socket, since the DSN scheme for specifying one is not supported by PHP's parse_url() function used by drupal to connect. The PEAR::DB DSN scheme Drupal claims to support is described here: http://pear.php.net/manual/en/package.database.db.intro-dsn.php In this scheme, the syntax for specifying a connection to a pgsql domain socket should be (I have no idea who thought this syntax was a good idea): pgsql://username:password@unix(/var/run/postgresql)/dbname Which is parsed by parse_sql as: % php -r 'print_r(parse_url("pgsql://username:password@unix(/var/run/postgresql)/dbname"));' Array ( [scheme] => pgsql [host] => unix( [user] => username [pass] => password [path] => /var/run/postgresql)/dbname ) However, Drupal's actual implementation is thus: function db_connect($url) { $url = parse_url($url); $conn_string = ' user='. $url['user'] .' dbname='. substr($url['path'], 1) .' password='. $url['pass'] . ' host=' . $url['host']; $conn_string .= ($url['port']) ? ' port=' . $url['port'] : ''; $connection = pg_connect($conn_string) or die(pg_last_error()); return $connection; } Which leads to a $conn_string of: " user=username dbname=var/run/postgresql)/dbname password=password host=unix(" -- System Information: Debian Release: 3.1 APT prefers testing APT policy: (990, 'testing'), (500, 'unstable') Architecture: sparc (sparc64) Kernel: Linux 2.6.6 Locale: LANG=C, LC_CTYPE=C (charmap=ANSI_X3.4-1968) Versions of packages drupal depends on: ii apache2-mpm-prefork [apache2] 2.0.52-3 Traditional model for Apache2 ii debconf 1.4.30.11 Debian configuration management sy ii makepasswd 1.10-2 Generate and encrypt passwords ii php4-cli 4:4.3.10-2 command-line interpreter for the p ii php4-pgsql 3:4.3.9-1 PostgreSQL module for php4 ii postgresql-client 7.4.7-2 front-end programs for PostgreSQL ii qmail [mail-transport-agent] 1.03-31 Secure, reliable, efficient, simpl ii wwwconfig-common 0.0.42 Debian web auto configuration -- debconf information: * drupal/remove_backups: false drupal/createuser_failed: * drupal/db_auto_update: false drupal/dropdb_failed: drupal/upgradedb_impossible: * drupal/dbgeneration: false * drupal/dbtype: PostgreSQL * drupal/database_doremove: false drupal/createdb_failed: * drupal/dbserver: localhost * drupal/webserver: apache2 drupal/upgradedb_failed: * drupal/dbname: drupal * drupal/dbuser: drupal drupal/dbadmin: root drupal/initdb_failed: drupal/conffile_failed:

1 0

[drupal-devel] Suggestion for dates on Drupal web site
by Scott Courtney 15 Feb '05

15 Feb '05

Good morning! I would like to suggest/request a small change in settings on the Drupal web site. Currently, dates in most sections are displaying as the European DD/MM/YYYY format. I would like to suggest that, since this site has a worldwide readership, the dates be formatted in ISO-standard YYYY-MM-DD (or YYYY/MM/DD) order. As a citizen of the U.S., I realize that a lot of U.S. sites are really bad about using the U.S.-specific date format (MM/DD/YYYY), and I also feel that this is a bad practice. My sites always use ISO format, because I'm trying to be a good world citizen, and when I teach web design here in the U.S., I make a point of stressing this to my students. :-) This is a small matter, I realize, and I hope I haven't offended anyone by raising the issue, but I think it would be worth considering so that Drupal users all around the world have equal convenience in reading the dates. Kind regards, Scott -- -----------------------+------------------------------------------------------ Scott Courtney | "I don't mind Microsoft making money. I mind them scott(a)4th.com | having a bad operating system." -- Linus Torvalds http://4th.com/ | ("The Rebel Code," NY Times, 21 February 1999) | PGP Public Key at http://4th.com/keys/scott.pubkey

5 5

[drupal-devel] call for arms: The Final URL regexp [tm]
by Bèr Kessels 15 Feb '05

15 Feb '05

Hi, We currently have loads of web-url rexexping scattered over the place. A quick list: check_url($value) (core) search ranking (core) field_url.inc (flexinode) weblink.module (various places) url_filter.module (filters) probalby more, like events. I would really like it if some regexp-guru can give me a hand with creating a single regexp that can be used drupalwide. What i want is either one API finction that handles all [1] or two API functions [2]. I am working on the weblink module, but plan to make it much better useable by drupal and its modules (an API). [1] all : check_url($text = NULL, $full_text = NULL) $text, if given, check if $text is a valid URL, return the valid url if TRUE, otherwise return FALSE $full_text, if given, check the fulltext for occurring URLS, return an array with all found URLS: $return[X] = array('text'=>$url_text, //optional text found in HTML <a>text</a> 'domain'=>$base_domain, 'local'=>$local_flag //if $base_domain == $base_url 'url'=>$validated_url) [2] two functions: check_url($text = NULL, $full_text = NULL) as above. drupal_url_regexp($type) where $type is 'url', 'mailto' or 'html' (other suggestions?) returns a string that can be used as regular expression in preg_replace to find a plain URL, mailto:url or an a href= URL. Any takers? Bèr -- | Bèr Kessels | webschuur.com | website development | | Turnhoutsebaan 34/3 | 2140 Antwerpen | België | | IM: ber(a)jabber.org.uk | MSN: berkessels(a)gmx.net | | pers: bler.webschuur.com | prof: www.webschuur.com |

2 1

Re: [drupal-devel] Release schedule
by Boris Mann 14 Feb '05

14 Feb '05

On Fri, 11 Feb 2005 01:06:29 +0200, Adrian Rossouw <adrian(a)daemon.co.za> wrote: > > On 11 Feb 2005, at 12:20 AM, Dries Buytaert wrote: > > > > 1. We need an upgrade path for PostgreSQL! > I spent today working on it. The patches will start flowing in the next > day or two. And...if the Drupal community really intends to keep supporting PostgreSQL, it needs to be more than just Adrian doing the work around this. Bryght doesn't use PostgreSQL for it's Drupal instances, so pretty much everything Adrian does here he has to fit in around his work schedule (i.e. in his own time). So...anyone else want to volunteer to help out with PostgreSQL? Dries: you asked about the help texts on the drupal-docs list. For some of the people there (me included), even just installing a CVS build is a pain in the ass. If you really want this done, someone needs to post the help texts, and then the doc team (among others) can help update/rewrite. -- Boris Mann http://www.bryght.com

5 5

[drupal-devel] [feature] Adjustable username crop
by com2 14 Feb '05

14 Feb '05

Project: Drupal Version: 4.5.2 Component: node system Category: feature requests Priority: normal Assigned to: com2 Reported by: com2 Updated by: com2 Status: patch According to that reasoning, the username crop should be taken out of the code, because the problem can be solved through CSS. Whatever the way to solve this problem, it should be configurable by the user. At the moment the code decides it for me and I cannot show the letters with CSS were code cropped it away for me. com2 Previous comments: ------------------------------------------------------------------------ January 27, 2005 - 17:11 : com2 Attachment: http://drupal.org/files/issues/common_0.diff (546 bytes) Some people might not like it when long usernames are cropped in the node haeder. This patch can be applied to includes/common.inc version 4.5.2 and creates a new variable 'user_name_crop' to adjust the crop length. ------------------------------------------------------------------------ January 27, 2005 - 21:32 : com2 Attachment: http://drupal.org/files/issues/common_1.diff (544 bytes) I had the numbers wrong. In the original code it was cropped at 15, not at 10 like in my diff. I also changed it a bit to avoid negative argaments. Attached the new diff from version 4.5.2. if ($object->uid && $object->name) { // Shorten the name when it is too long or it will break many tables. if (strlen($object->name) > variable_get('user_name_crop', 15)+5) { $name = truncate_utf8($object->name, variable_get('user_name_crop', 15)) .'...'; } else { $name = $object->name; ------------------------------------------------------------------------ January 27, 2005 - 22:00 : Bèr Kessels IMO this should be done in a theme, not with yet another config option. what about theme_username() ? ------------------------------------------------------------------------ January 27, 2005 - 22:07 : killes(a)www.drop.org theme_username++ format_name is one of the functions I always need to modify. theme_date would be nice, too. ------------------------------------------------------------------------ January 27, 2005 - 22:27 : Dries This should be done in the theme. ------------------------------------------------------------------------ January 31, 2005 - 12:34 : tangent In some cases there may not be a need to truncate the output at all. We should consider using CSS to adjust the display of text rather than truncating strings to arbitrary lengths that may be less than can be viewed in a given resolution. Given the following markup <div class="container"> This is a string which may be displayed in a small area and may be truncated if the container is too small to contain it. </div> the following CSS may be used to control the overflow of the container. .container { display: block; overflow: hidden; white-space: nowrap; text-overflow: ellipsis; } Use of the proprietary (MS Internet Explorer) "text-overflow" property may be used to show ellipsis on a container with hidden content. Javascript could be used to display ellipsis on these containers for other browsers as well, though it may well not be worth doing so. Using CSS to control display in this manner gives greater control for supporting variable environments (graphical browser, wap browser, text browser) without limiting the content that I user needs to see. -- View: http://drupal.org/node/16299 Edit: http://drupal.org/project/comments/add/16299

2 2

[drupal-devel] [feature] http proxy support patch
by Anonymous 14 Feb '05

14 Feb '05

Project: Drupal Version: 4.4.2 Component: base system Category: feature requests Priority: minor Assigned to: Anonymous Reported by: liquidx Updated by: Anonymous Status: patch Here's teh error I get when I try this "Patch - drupal-cvs-20040802-proxy-v2.diff " Aggregator: failed to parse RSS feed Microsoft Security Info: invalid schema . any idea's? Anonymous Previous comments: ------------------------------------------------------------------------ August 3, 2004 - 03:39 : liquidx Attachment: http://drupal.org/files/issues/drupal-cvs-20040802-proxy.diff (1.96 KB) My drupal site is behind a firewall which doesn't allow outgoing port 80 connections except if they go through an http proxy. normally that doesn't really harm anything, except when i start using XML-RPC or try to get RSS/Atom feed from other blogs. So I've written a quick patch to add http_proxy support in conf.php. I'm not sure whether that is the best place to put it, but if you think people might want a UI to change that, I could do that. But I suspect that proxy settings don't change much (its not like you migrate a drupal web site to different isps everyday :), so conf.php seems like a good choice. Cheers, Alastair Tse ------------------------------------------------------------------------ August 3, 2004 - 03:42 : liquidx Attachment: http://drupal.org/files/issues/drupal-cvs-20040802-proxy-v2.diff (1.96 KB) slight typo in the comment, i never get things right the first time round :( ------------------------------------------------------------------------ August 3, 2004 - 04:13 : Anonymous HOW TO MAKE 1000'S FROM 6$! READ PLEASE! How to afford anything you want, no joke! Read, it cant hurt! READING THIS COULD CHANGE YOUR LIFE! AND RICH! I found this on a bulletin board and decided to try it. A little while back, I was browsing through newsgroups and came across an article similar to this that said you could make thousands of dollars within weeks with only an initial investment of $6.00! So I thought, "Yeah right, this must be a scam", but like most of us, I was curious, so I kept reading. Anyway, it said that you send $1.00 to each of the 6 names and addresses stated in the article. You then place your name and address in the bottom of the list at #6, and post the article in at least 200 newsgroups. (There are thousands) No catch, that was it. So after thinking it over, and talking to a few people first, I thought about trying it. I figured: "what have I got to lose except 6 stamps and $6.00, right?" Then I invested the measly $6.00. Well GUESS WHAT!!... within 7 days, I started getting money in the mail! I was shocked? I figured it would end soon, but the money just kept coming in. In my first week, I made about $25.00. By the end of the second week I had made a total of over $1,000.00! In the third week I had over $10,000.00 and it's still growing. This is now my fourth week and I have made a total of just over $42,000.00 and it's still coming in rapidly. It's certainly worth $6.00, and 6 stamps, I have spent more than that on the lottery!! Let me tell you how this works and most importantly, why it works... Also, make sure you print a copy of this article NOW, so you can get the information off of it as you need it. I promise you that if you follow the directions exactly, that you will start making more money that you thought possible by doing something so easy! Suggestion: Read this entire message carefully! (print it out or download it.) Follow the simple directions and watch the money come in! It's easy. It's legal. And, your investment is only $6.00 (plus postage). IMPORTANT: This is not a rip-off; it is not indecent; it is not illegal; and it is virtually no risk - it really works!!! IMPORTANT2: The $1.00 for each person on the list is for keeping you up to date and for putting you in all our newsgroups. So you will make just as much as we do!!! If all of the following instructions are adhered to, you will receive extraordinary dividends. PLEASE NOTE: Please follow these directions EXACTLY, and $50,000 or more can be yours in 20 to 60 days. This program remains successful because of the honesty and integrity of the participants. Please continue its success by carefully adhering to the instructions. You will now become part of the Mail Order business. In this business your product is not solid and tangible, it's a service. You are in the business of developing Mailing Lists. Many large corporations are happy to pay big bucks for quality lists. However, the money made from the mailing lists is secondary to the income which is made from people like you and me asking to be included in that list. Here are the 4 easy steps to success: STEP 1: Get 6 separate pieces of paper and write the following on each piece of paper "PLEASE PUT ME ON YOUR MAILING LIST." Now get 6 US $1.00 bills and place ONE inside EACH of the 6 pieces of paper so the bill will not be seen through the envelope (to prevent thievery). Next, place one paper in each of the 6 envelopes and seal them. You should now have 6 sealed envelopes, each with a piece of paper stating the above phrase, your name and address, and a $1.00 bill. What you are doing is creating a service. This is absolutely legal? You are requesting a legitimate service and you are paying for it! Like most of us I was a little skeptical and a little worried about the legal aspects of it all. So I checked it out with the U.S. Post Office (1-800-725-2161) and they confirmed that it is indeed legal! Mail the 6 envelopes to the following addresses: 1.) GLEN RAY CAMPBELL Jr. 919 Virginia Ct. Panama City,Fl 32404,usa 2.)Heather Angel 4244 46th St. San Diego, Ca. 92115 3.) Alexander Dolstra Frederik Hendrik straat 37 3143 LB Maassluis the Netherlands 4.) J. Kirk 2360 Forbes Ave. Santa Clara, CA 95050 5.) N. Hernandez PO Box 712 Crystal Beach, Fl. 34681, usa 6.)D.Herela 849 La Fayette apt.6-D El Paso, TX. 79907, usa STEP 2: Now take the #1 name off the list that you see above, move the other names up (6 becomes 5, 5 becomes 4, etc. ...) and add YOUR name as number 6 on the list. STEP 3: Change anything you need to, but try to keep this article as close to the original as possible. Now, post your amended article to at least 200 newsgroups. (I think there are close to 24,000 groups.) All you need is 200, but remember, the more you post, the more money you make! This is perfectly legal! If you have any doubts, refer to Title 18 Sec. 1302 & 1241 of the Postal lottery laws. Keep a copy of these steps for yourself and, whenever you need money, you can use it again, and again. PLEASE REMEMBER that this program remains successful because of the honesty and integrity of the participants and by their carefully adhering to the directions. Look at it this way, if you are of integrity, the program will continue and the money that so many others have received will come your way, too. NOTE: You may want to retain every name and address sent to you, either on a computer or hard copy and keep the notes people send you. This VERIFIES that you are truly providing a service. (Also, it might be a good idea to wrap the $1 bill in dark paper to reduce the risk of mail theft.) So, as each post is downloaded and the directions carefully followed, six members will be reimbursed for their participation as a List Developer with one dollar each. Your name will move up the list geometrically so that when your name reached the #1 position you will be receiving thousands of dollars in CASH!!! What an opportunity for only $6.00 ($1.00 for each of the first six people listed above). Send it now, add your own name to the list and you're in business! ---DIRECTIONS----FOR HOW TO POST TO NEWSGROUPS------------------ STEP 1: You do not need to re-type letter to your own posting. Simply put your cursor at the beginning of this letter and drag your cursor to the bottom of this document, and select 'copy' from the edit menu. This will copy the entire letter into the computer's memory. STEP 2: Open a blank 'notepad' file and place your cursor at the top of the blank page. From the 'edit' menu select 'PASTE'. This will paste a copy of the letter into notepad so that you can add your name to the bottom of the list and to change the numbers of the list. STEP 3: Save your new notepad file as a '.txt' file. If you want to do your postings in different settings, you'll always have this file to go back to. STEP 4: Use Netscape or Internet Explorer and try search for various newsgroups (on-line forums, message boards, chat sites, discussions). STEP 5: Visit these message boards and post this article as a new message by highlighting the text of this letter and selecting 'PASTE' from the edit menu. Fill in the Subject, this will be the header that everyone sees as they scroll through the list of postings in a particular group, click the post message button. You're done with your first one! Congratulations... That is it! All you have to do is jump to different newsgroups and post away, after you get the hang of it, it will take about 30 seconds for each newsgroup! ** REMEMBER, THE MORE NEWSGROUPS YOU POST IN, THE MORE MONEY YOU WILL MAKE!! BUT YOU HAVE TO POST A MINIMUM OF 200** That is it! You will begin receiving money form around the world within days! You may eventually want to rent a P.O. Box due to the large amount of mail you will receive. If you wish to stay anonymous, you can invent a name to use, as long as the postman will deliver it. **JUST MAKE SURE ALL THE ADDRESSES ARE CORRECT.** Now the WHY part: Out of 200 postings, say I receive only 5 replies (a very low example). So then I made $5.00 with my name at #6 on the letter. Now, each of the 5 persons who just sent me $1.00 make the MINIMUM 200 postings, each with my name at #5 and only 5 persons respond to each of the original 5, this is an additional $25.00 for me. Now those 25 each make 200 MININUM posts with my name at #4 and only 5 replies each. This brings in an additional $125.00. Now, those 125 persons turn around and post the MINIMUM 200 with my name at #3 and receive 5 replies each, I will make an additional $625.00. Ok, now here is the fun part, each of those 625 people post a MINIMUM 200 letters with my name at #2 and they receive 5 replies each. That just made me $3,125.00!!! Those 3,125 persons will all deliver this message to 200 newsgroups with my name at #1 and if still 5 persons per 200 react, I will receive an additional $15,625.00!! With an investment of only $6.00! AMAZING! When your name is no longer on the list, you just take the latest posting in the newsgroups, and send out another $6.00 to names on the list, putting your name at number 6 again. And start posting again. The thing to remeber is: do you realize that thousands of people all over the world are joining the internet and reading these articles everyday? JUST LIKE YOU are now!! So, can you afford $6.00 and see if it really works?? I think so... People have said, "what if the plan is played out and no one sends you the money? So what! What are the chances of that Happening when there are tons of new honest users and new honest people who are joining the internet and newsgroups everyday and are willing to give it a try? Anyway, it is only $6.00 for a chance at thousands. Estimates are at 20,000 to 50,000 new users, every day, with thousands of those joining the actual internet. Remember, play FAIRLY and HONESTLY and this will really work. You wouldn't want someone to cheat you the same way you may be cheating! HOW TO MAKE 1000'S FROM 6$! READ PLEASE! ------------------------------------------------------------------------ August 4, 2004 - 08:54 : TDobes This seems like a possibly useful patch for those stuck in an environment where they must use a proxy. (intranets?) I have no proxy set up at the moment with which to test, but +1 to the functionality. ------------------------------------------------------------------------ December 3, 2004 - 11:47 : Steven If we use HTTP/1.1 (using the Host header), I think we can send exactly the same request to both a proxy and the real server. It might make the code a bit cleaner. ------------------------------------------------------------------------ January 26, 2005 - 04:01 : Steve Stock That's correct, HTTP/1.1 explicitly requires servers to accept an absolute URI in the request-line (which of course proxies already require). See http://www.w3.org/Protocols/rfc2616/rfc2616-sec5.html#sec5.1.2 for details. Note: a host header is still required. The drawback is all HTTP/1.1 applications must be able to accept the chunked transfer coding, see http://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html#sec4.4 (third paragraph above section 4.5). I don't know if Drupal supports this (doesn't look like it). While not as nice, for now it appears simpler to implement a proxy along the lines of this patch. My only suggestion would be to add a proxy section to the settings page instead of hardcoding the proxy host/port. ------------------------------------------------------------------------ February 4, 2005 - 04:23 : jhefmv24 Attachment: http://drupal.org/files/issues/common.inc_3.patch (1.37 KB) +1 on having a facility to modify proxy settings... here's a patch of /includes/common.inc for drupal-4.4.2 version ------------------------------------------------------------------------ February 4, 2005 - 08:18 : Bèr Kessels Hi, you use variable_get('proxy_name', '') , but that is not defined anywhere. I would plead against another config option, or conf.php options. I am no proxy guru, so the following suggestion might be impossible: * add the proxy port to the $base_url and try to re-arrange your logic in such a way ,that it will detect the port from the $base_url. If found the variabale will be set, and userd. * or use the default system, but add some logic to the fsockopen, that, if it fails, it will try again, with proxy, if that was sucessfull, set the variable. Bèr ------------------------------------------------------------------------ February 4, 2005 - 08:53 : jhefmv24 see this for complete patches: http://drupal.org/node/7881 -- View: http://drupal.org/node/9706 Edit: http://drupal.org/project/comments/add/9706

1 0

[drupal-devel] [feature] "remember me" doesn't work
by kbahey 14 Feb '05

14 Feb '05

Project: Drupal Version: cvs Component: user.module Category: feature requests Priority: normal Assigned to: kbahey Reported by: marco Updated by: kbahey Status: patch A related patch that moves the session.cookie_lifetime value to settings.php is available. Please see http://drupal.org/node/17303 kbahey Previous comments: ------------------------------------------------------------------------ September 22, 2003 - 08:37 : marco "remember me" checkbox in the login box doesn't work; even if the checkbox is left unchecked the user is NOT forgotten when he quits the browser. Try logging in w/o "remember me", then quit the browser and open it again: you should be still logged in. What happens: when you login w/o checkbox user.module outputs a cookie with lifetime = 0 ("until session ends"); but user.module calls session_start() at the beginning, which outputs a cookie too, with the lifetime specified in .htaccess; and session_start() outputs this cookie always, so on the next page the cookie from user_login() will be overwritten. I run Mozilla 1.4; I can replicate with Drupal 4.0 and Drupal 4.2 on PHP 4.3.3, and I can replicate this on drupal.org which also runs PHP 4.3.3; OTOH I can't replicate on a site running Drupal 4.2 with PHP 4.2.2, which may mean session_start() changed with PHP 4.3.x; I looked in the changelog of PHP but couldn't find anything. I didn't have any report about this before upgrading to PHP 4.3.3, which also seems to strengthen the hypothesis of a changed behaviour in PHP. Another test I did also showed that with PHP 4.2.2 no cookie is printed by session_start() if a session cookie is found, while it is always printed in PHP 4.3.3; I double checked the configurations and didn't find anything which may cause this. If you want to investigate this, I suggest you to use Mozilla and Live HTTP Headers plugin. ------------------------------------------------------------------------ October 10, 2003 - 19:37 : moshe weitzman Can anyone confirm this? Also, how to fix? ------------------------------------------------------------------------ October 12, 2003 - 12:45 : axel(a)debian.linuxrulez.ru I agree it for Mozilla 1.0. On my site running on FreeBSD 4.7, PHP/4.3.0, Drupal CVS (Oct 3) this function also don't work. Though, with Galeon 1.2.5 cookie works ok. On localhost (Debian GNU/Linux 3.0, PHP 4.1.2, same Drupal cvs version) it works ok with Mozilla & Galeon. ------------------------------------------------------------------------ October 12, 2003 - 13:34 : al The original bug report is surely due to Drupal needing to unset the cookie that it originally stored? To fix this bug, we therefore need a check on the user login/validation stage which forcibly unsets the cookie if you don't do "remember me". I suspect Axel's problems with one of his sites and not the other are due to him blocking a cookie originally and having that site on his Mozilla's list of sites to ban cookies from, or similar. Axel - if you are genuinely having issues with remember me not working at all (and not the fault originally described in this report by Ax) then please open a different bug report. Please make sure it's a genuine problem first - i.e. clear your blocked cookies sites list in Mozilla. ------------------------------------------------------------------------ October 12, 2003 - 18:24 : axel(a)debian.linuxrulez.ru Well. I don't sure what is a bug, therefore first post the question about it to forum [1]. Answer to that question point me to this bug report. Already several users of my site [2] report me about problem with "remember me" (I don't know which browsers they use). And there are not any blocked sites in my Mozilla cookies list - from site I receive only cookie PHPSESSID that expire time shows "at end of session". [1] http://drupal.org/node/view/3601 [2] http://debian.linuxrulez.ru ------------------------------------------------------------------------ October 17, 2003 - 15:36 : dmo Expect "remember me" problems for users of Internet Explorer 6. Depending on the privacy settings, IE6 may automatically expire all cookies at the end of the browser session if your site doesn't have a compact P3P policy. See http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnpriv/htm… and http://www.oreillynet.com/pub/a/javascript/2002/10/04/p3p.html for further details. ------------------------------------------------------------------------ October 17, 2003 - 21:53 : moshe weitzman since noone can reproduce this, i am marking as 'by design' ------------------------------------------------------------------------ November 25, 2003 - 02:41 : junyoung This is not a IE6 specific problem. I have seen the same symptom with IE5.5/6.0, Opera 7.0/7.1, and Konqueror 3.1.x so far. FWIW, my blog site is running with Drupal 4.2.0 + PHP 4.3.3. ------------------------------------------------------------------------ November 25, 2003 - 09:06 : remco(a)rc6.org Same problem on http://rc6.org, though the other way around. No matter what I do, my session will time out eventually. Tested using Opera 7.x, IE, Mozilla and Epiphany. ------------------------------------------------------------------------ November 25, 2003 - 10:00 : moshe weitzman reopening this case. i confirm the behavior that marco describes using rc6.org and drupal.org. i find it easiest to just turn cookie prompting on in order to watch what is happenning. what i am seeing, like marco described, is that our 'permanent cookie' which is supposed to last for a year is being overwritten in the next request with a standard session cookie that expires in the time frame specified in .htaccess. for drupal.org, standard sesson cookies last 1 month whereas the permanent cookie lasts for a year. i don't know how to fix this from within drupal. the cookie that we lay down for 'remember me' is working fine. the problem is the later overwrite which is caused by PHP's session handing, not drupal. ------------------------------------------------------------------------ November 26, 2003 - 07:35 : moshe weitzman To make matters more complicated, I cannot reproduce this using PHP as an ISAPI module on IIS ------------------------------------------------------------------------ November 26, 2003 - 14:07 : Dries Maybe we can set a "remember" bit in the session table and periodically wipe users who don't have the "remember"-bit set. The wiping part could be added to sess_gc() ... ------------------------------------------------------------------------ December 3, 2003 - 20:30 : joshk I have this problem w/musicforamerica.org The really maddening thing is that I have another install of drupal 4.3 on the same webserver and it works just fine. If the problem is with drupal's cookie being overwritten by a PHP session cookie, can this be fixed by giving the cookies different names? Sounds too simple to be the solution... ------------------------------------------------------------------------ December 12, 2003 - 11:48 : ykoehler http://ca.php.net/manual/en/function.session-set-cookie-params.php Even though drupal is sending a cookie, it should always set this PHP parameter so that the session_start() call will use the same value, or not send any cookie at all by itself and let session_start() do it with, again, a call to this function to set the correct parameter. The reason why you don't get the same on a site basis is probably due to the different php.ini used for those sites as the default depends on the installation and not drupal which is probably why only some get the bug if there's such a thing. ------------------------------------------------------------------------ December 30, 2003 - 15:50 : paul(a)murphymaphia.com I am having the opposite problem. Even if I check the "remember me" box my session ends when the browser closes and I'm forced to log in the next time I return to the site. No cookie is EVER set by my site. http://www.murphymaphia.com ------------------------------------------------------------------------ January 20, 2004 - 12:31 : mathias Charles Miller has written a persistent login cookie best practices [3] i feel is worth reading. [3] http://fishbowl.pastiche.org/2004/01/19/persistent_login_cookie_best_practi… ------------------------------------------------------------------------ February 11, 2004 - 13:20 : paul(a)murphymaphia.com Has any progress been made on this? I have spent a lot of time in the code and can't manage to track this problem down. If anyone has any ideas, thoughts, etc to share, post them here so we can get this solved. ------------------------------------------------------------------------ February 12, 2004 - 00:07 : dmjossel I have this problem (remember me feature not working) in Drupal 4.3.x on PHP 4.3.2. I do NOT have it on Drupal 4.2 on PHP 4.3.2, in exactly the same environment. So perhaps sessions have changed in PHP 4.3.x, but this still didn't break Drupal 4.2, only 4.3.x. ------------------------------------------------------------------------ February 26, 2004 - 08:47 : moshe weitzman Attachment: http://drupal.org/files/issues/_4drupal (5.3 KB) Here is a patch which attempts to resolve this problem. I took Josh's suggestion - just rename the permanent cookie so it get overwritten by the PHP session cookie. So this patch names our permanent cookie 'remember_me'. The value of this cookie is the current sessionID. This cookie is checked in sess_read(). It is set just as before, in user_login(). I refactored sess_read() a bit for cleaner flow. It uses a new helper function called sess_construct_user(). Feedback welcome. Since not everyone experienced a problem with remember me, I'm particlarly interested in feedback from those who did. ------------------------------------------------------------------------ March 2, 2004 - 05:43 : dmjossel No observable effect. Users who click "remember me" are still logged out after a period of inactivity or a browser restart, as is true in 4.3.2 (but not true in 4.2 on the same machine). Just out of curiosity I made similar changes in a copy of 4.3.2, renaming the cookie-- also with no effect. What *is* odd is that I used to see this problem using Drupal.org itself. But I seem now to have been logged in for quite awhile (several days) without having to reauthenticate, despite long periods of inactivity and browser restarts. Has some change other than this patch been made to CVS running on the live site? ------------------------------------------------------------------------ March 2, 2004 - 06:21 : dmjossel One more follow up: the patch re-enables the "remember me" button on logon and the preferences for the user.module, but not the "logout" button in the user info block. ------------------------------------------------------------------------ March 6, 2004 - 11:33 : moshe weitzman The logout link never left, so naturally this patch does not re-instate it. By "user-info" block, i'm guessing that you mean the 'Navigation' block which holds the menu(). Drupal.org issues session cookies for a month so you can remain logged in for a long time. Anyone else have feedback on this patch. It is working for me. ------------------------------------------------------------------------ March 6, 2004 - 12:01 : Jeremy(a)kerneltrap.org Your patch works perfectly for me. I tested with three browsers (from Linux): Mozilla Firefox, Konquerer, Internet Explorer (via Wine) I did the following with each: 1) with latest CVS installation, I log in then restart my browser. I have to relogin. 2) I applied your patch to my CVS installation. 3) I logged in w/o checking "remember me". I then restart my browser, and I'm not logged in. 4) I logged in w/ "remember me" checked. I restart my browser, and I'm still logged in. 5) I logged in w/ "remember me" checked. I click logout, then restart my browser. I have to login again. ------------------------------------------------------------------------ April 8, 2004 - 20:09 : Brian(a)brianpuccio.net Using weitzmann's patch works for me. I had no problem when running 4.3.0, 4.3.1, or 4.3.2 on my site, but adding 4.4.0 I had to log in each time. Applying the patch to my (next to stock, but ever so slightly modified) install of 4.4.0 fixes the problem and I am remembered across browser tab closings, browser closings and system restarts. Thank you greatly. ------------------------------------------------------------------------ April 11, 2004 - 12:18 : Jeremy(a)kerneltrap.org Attachment: http://drupal.org/files/issues/session.inc.patch (651 bytes) I believe there is a small bug in weitzman's patch. I've been running with it applied on KernelTrap.org for some time, with much positive feedback from users. However, I noticed that occasionally I become a phantom surfer -- I'm logged in and accessing pages, but I don't show up in the "Who's online" block. I suspect the online guests number is low, too. After a little digging, I came up with the attached simple patch. I have yet to confirm if it solves the phantom surfer problem. Essentially, when we log in with a cookie we also update the sessions table timestamp. To use, first apply weitzman's patch (earlier in this thread), then apply this one. ------------------------------------------------------------------------ April 11, 2004 - 16:28 : Jeremy(a)kerneltrap.org Attachment: http://drupal.org/files/issues/session.inc_0.patch (584 bytes) My earlier patch didn't work. It was redundant with sess_write(), and added nothing new. Hence, it didn't fix the "phantom user" that I've been experiencing. Now, however, I believe I fully understand what's going wrong. Essentially what's happening is a user is being loaded via the remember_me cookie, but their session id is different. Thus, the db_query in sess_write() fails to update a user session in the database. This patch detects that failure and acts accordingly by creating a new entry. This patch applies against drupal 4.4 with weitzman's patch [4] applied. [4] http://drupal.org/files/issues/_4drupal ------------------------------------------------------------------------ April 15, 2004 - 02:39 : mabster Still seems to be a bit flakey. I get logged out every once in a while at work for some reason (during the day). And every time I move from my work to my home machines (or vice versa) I need to login again. Does the cookie base itself on my IP address or something? Could it be that because I'm switching between two different PCs that it can't remember my session details? ------------------------------------------------------------------------ April 21, 2004 - 21:19 : dmjossel I've upgraded the site Rampancy.net to Drupal 4.4 and applied Weitzman's patch. The behavior I'm seeing now I don't completely understand. Logging in to the site, you can quit the browser and stay logged in. But after a period of inactivity, somewhere between 30 and 60 minutes, the session expires and you have to log in again. I don't see this behavior on drupal.org. php.ini on the server is set for session cookies to last for 30 days. Drupal, by default, is supposed to give cookies that last 1 year. Is there anything else I should be looking at? Apparently some admins have this problem and others don't. Server info: FreeBSD synfibers.com 4.7-RELEASE-p22 FreeBSD 4.7-RELEASE-p22 #29: Tu i386 Apache/1.3.27 OpenSSL/0.9.7 (Unix) PHP/4.3.4 registered session save handlers are files and user. (local value is user) ------------------------------------------------------------------------ April 21, 2004 - 22:20 : mabster Yes, dmjossel's description matches the behaviour on my site (www.madprops.org) We're a hosted site, so I'm not exactly sure of the server's details. I'm fairly certain it's Windows2003. Definitely a Windows flavour, anyway, which means that this problem (with the patch applied) is not restricted to Linux. ------------------------------------------------------------------------ April 23, 2004 - 05:26 : dmjossel Apologies first for using this like a forum... mabster, upload a file to the server with the phpinfo(); in it (inside PHP tags of course). What values does it show for cookie-related functions? For session_save_handler? Perhaps there is a common element in server configuration in these two cases, despite the different operating systems. ------------------------------------------------------------------------ April 23, 2004 - 05:48 : mabster session.save_handler = files session.cookie_domain = no value session.cookie_lifetime = 0 session.cookie_path = / session.use_cookies = On session.use_only_cookie = Off Other info (at least until someone tells me that publishing this info is dangerous and I remove the file) here [5]. [5] http://www.madprops.org/info.php ------------------------------------------------------------------------ April 23, 2004 - 11:12 : dmjossel Suggestions... try setting session.use_only_cookies =1 either in php.ini, or if you don't have access to that, by using ini_set in drupal's php.conf and see if that has any effect. Next change Drupal's site configuration for the user module, and toggle whether you allow users to choose whether or not to be logged out (this produces the "remember me" cookie if you've installed weitzman's patch) or choose "users are never logged out". Neither of these seem to help here, but perhaps they'd help you. I noticed that until I set my browser to "accept all cookies automatically" and set session.use_only_cookies =1, I was getting the PHPSESSID cookie, but not the remember_me cookie. Now I get them both... but still logged out after 30 minutes of idle time, no matter what. ------------------------------------------------------------------------ April 23, 2004 - 11:26 : moshe weitzman those are not the session values that ship with drupal (see .htaccess file). for example, session_save_handler has to be 'user'. otherwise, you are not using drupal's session table and your 'remember me' will fail. since you are using IIS, you have to set your php.ini with values like those found in .htaccess. .htaccess files are only respected by Apache (and even then, not always. depends on your web host). ------------------------------------------------------------------------ April 23, 2004 - 19:39 : mabster Just out of interest, weitzman: How does drupal.org's 'remember me' work? I installed a cookie inspector on my machine at work, and didn't see any 'remember me' values in drupal.org's cookie. Does it use a different system? I will see what our host can do about changing the php.ini values. Since there are many sites hosted on that one box, it'll be a long shot to get them to change it. ------------------------------------------------------------------------ April 24, 2004 - 12:21 : dmjossel Try tweaking the value of session.gc_maxlifetime either in the .htaccess file or in php.ini. The default value in the .htaccess file seems to be 1440 seconds-- or 24 minutes, almost exactly the amount of time my sessions were lasting before being forced to log in again. I've increased this value, and now I'm seeing the remember_me cookie stay with the same value, while the PHPSESSID cookie is constantly being replaced by a new one-- but I don't have to log in again. ------------------------------------------------------------------------ April 24, 2004 - 20:23 : dmjossel Well, it's the next morning here, and I'm still logged in at Rampancy.net and the other Drupal 4.3 sites that are hosted on the same machine. I think at least this might be a workaround for those who have this problem and Weitzman's patch alone doesn't solve it. ------------------------------------------------------------------------ April 25, 2004 - 20:02 : dmjossel After a longer period of inactivity (equal to the value of session.gc_maxlifetime) the sessions ALWAYS seem to log the user out, regardless of the remember_me cookie value, the PHPSESSID value, or the configuration settings for the user module. ------------------------------------------------------------------------ May 6, 2004 - 06:34 : Robert L I can confirm #41, have tried changing some of the sessions vars in .htaccess, but this did not solve the problem. Sessions are destroyed after some time of browser inactivity (some hours). At the browser side things look oke, it happens with Mozilla and IE. Have not tested the patch yet, I am not using the remember me cookie option, and this looks another unrelated (?) problem to me. I use PHP 4.3.6 with Apache 2. ------------------------------------------------------------------------ June 5, 2004 - 21:41 : Anonymous I fixed it very simply by adding the line: session_set_cookie_params(0, "/dev/null"); immediately before the call to session_start(). This causes the cookie that PHP sets to only apply to the path "/dev/null" which, of course, doesn't exist. Thus, the cookie that Drupal's setcookie() call sets, with a path of "/", is held on to by the browser, and Drupal goes back to its old behaviour. It's a hack, but it works. A better long-term solution would be for Drupal to use the session_set_cookie_params() call to set the expiry time, rather than its own call to setcookie(). Tim Bates ------------------------------------------------------------------------ June 6, 2004 - 12:37 : bertboerland(a)www.drop.org I am sorry to say, that adding a line like that in user.module doesnt do the trick for me (running php 4.2.3/drupal 4.). Can it be that this might work under certain conditions which differ from mine? Anyone else still working on this *critical* bug? I am not changing the status but I think it should still read "active"? ------------------------------------------------------------------------ June 8, 2004 - 03:58 : bertboerland(a)www.drop.org never mind my previous remark... ------------------------------------------------------------------------ August 9, 2004 - 20:53 : moshe weitzman perhaps someone will take interest in this bug. i hope not to release another drupal version without 'remember me'. ------------------------------------------------------------------------ August 13, 2004 - 16:49 : seanr This used to work for me. Now I hear it's been completely disabled. Why? I'd rather have a feature that works for most users and not some than not have the feature at all. ------------------------------------------------------------------------ September 22, 2004 - 11:39 : Anonymous We dropped the whole 'remember me' posiibility in drupal 4.5. If it is - strategically seen - a good move, is another story, but now marked this as "Won't fix". ------------------------------------------------------------------------ September 22, 2004 - 15:35 : moshe weitzman umm - thanks 'anonymous'. this is a valid feature request, even if it isn't a valid bug anymore. marking as such. ------------------------------------------------------------------------ October 13, 2004 - 22:45 : jasper this is a bug actually. If you don't think so, you really aren't thinking about user experience. ------------------------------------------------------------------------ October 19, 2004 - 13:48 : jasper I am having this problem with 4.5. After you login, the cookie that is set to expire in a month is overwritten by a session cookie, so remember me doesn;t work. I "fixed" this by hacking user.module to set its cookie to a non-existant path. Maybe that will mess something up. I notice that drupal.org overcomes this by setting the initial cookie to "host: drupal.org" and the second cookie to "domain: .drupal.org" Where is this documented? How do you do that? ------------------------------------------------------------------------ October 21, 2004 - 11:43 : Anonymous I was having the same problem. I changed user.module's call to setcookie() so that the cookie would expire after 2000000 seconds. That fixed it for me. Now I don't have to login every time I visit. I hope I didn't break anything else, though. setcookie(session_name(), session_id(), FALSE, $path); becomes: setcookie(session_name(), session_id(), time() + 2000000, $path); ------------------------------------------------------------------------ October 21, 2004 - 22:03 : dtan Ya, suddenly noticed that I had to login everytime . . . not a huge deal, but is a bit of a pain. Should this be a configurable option for the site?? Obviously from the length of this issue, people feel it is a useful feature. . . dtan ------------------------------------------------------------------------ October 26, 2004 - 23:32 : tvst I WANT THIS FEATURE BACK! Sure, there were problems with it -- but let's fix the bug instead of killing a nice feature. ------------------------------------------------------------------------ November 3, 2004 - 18:02 : Goba Dries just committed a patch, which removes that superflous cookie setting, so session time in upcoming versions will be determined by the PHP settings, and short lived cookies will not be set. Until new versions are released, it is completely safe to remove the setcookie line in user.module (and the preg line above that), and it solves the problem of users needing to login everytime. ------------------------------------------------------------------------ November 5, 2004 - 01:02 : Anonymous wow. thanks. this was breaking my sites ------------------------------------------------------------------------ December 14, 2004 - 12:06 : kbahey This way, the PHP settings are the ones that determine the session length. Wouldn't it be better to have an admin option that allows users to set their session length? with a global default for sessions (e.g. 1 week, 2 weeks, 1 month, ...etc.) If this option is enabled, then it can be overriden by the user individually too. This is a better scheme than having a forced global setting outside Drupal, on the PHP level for all users. What do others think? ------------------------------------------------------------------------ February 3, 2005 - 13:50 : kbahey Reviving this issue. The session length should be an option that the administrator, or the end user can set. We should not rely on the PHP settings alone. ------------------------------------------------------------------------ February 11, 2005 - 19:54 : kbahey Attachment: http://drupal.org/files/issues/use-module-session.patch (1.92 KB) I feel that removing the setcookie() call was not the right thing to do. Session length should not be relegated to .htaccess or php.ini, but should be an application feature. I am attaching a patch that reinstates this functionality via a configuration option. The sysadmin determines how long visitors should have their session for, and a cookie is set for them with the appropriate lifetime. Please look at this rationally and objectively and do not dismiss it offhand because 'too many options are bad'. I had to go through php.ini and .htaccess hell for the last few months because my hosting company has implemented PHP suExec (hence .htaccess cannot set php parameters), then moved to another machine with .htaccess reinstated. This also allows people (like me) who serve several domains from a single Drupal installation to set the session length differently for different domains. Moreover, the Drupal end user should not have to go through fiddling with .htaccess values to get this functionality back. ------------------------------------------------------------------------ February 12, 2005 - 06:51 : Bèr Kessels -1 for yet another config option. Thisis a (very) advanced feuature, that will never have to be changed on a site. Therefore we need to give it a good default, and if people really want to chage it, those few can do so in conf.php. Bèr ------------------------------------------------------------------------ February 12, 2005 - 08:57 : Goba Please *do not use setcookie()* to overwrite the session cookie! The problem before was that $path and $domain were not the same as set by the session handler cookie, and so the session handler cookie was not overwritten. This behaviour depends on host configuration... See, you introduce a $path variable in your patch, which is not set, and you don't care about the cookie domain... These were the main problems with the previous aproach, and this is why the setcookie() was removed. You should deal directly with the session ini settings! So please ini_set() the session cookie lifetime, if .htaccess is not enough for you, and let the session handler set the proper cookie. ------------------------------------------------------------------------ February 12, 2005 - 09:59 : kbahey Attachment: http://drupal.org/files/issues/user-module-session-2.patch (8.89 KB) Getting the sysadmin to tweek the .htaccess or php.ini is a tedious thing. If we can provide a better interface for them to do this, then it is better. Every time PHP gets upgraded by my hosting company or move machines, things do not work. With this patch, the sysadmin can set how long sessions are valid for. This version of the patch still overrides the session cookie, but uses the correct path. I am seeing reluctance to putting this in CVS, based on 'yet another option' objection, or it being a cookie. While I can sympathise with the objections, and see their point of view, I do not see another solution being proposed to those who asked for this feature to be brought back. See for example comments 46, 49, and 54. Taking away a feature is not a good thing. I would rather have had this be an end user settable option (a global default for the site, and a per user settable option). Until the feature is back with a technical solution that is acceptable to the powers that be, users can use this patch for CVS. ------------------------------------------------------------------------ February 12, 2005 - 10:14 : Goba kbahey, please, what is the problem with setting the ini setting with ini_set() runtime? It would work without fiddling with the $path and $domain... Also note that your patch contains a lot of unrelated changes... ------------------------------------------------------------------------ February 12, 2005 - 11:06 : kbahey Attachment: http://drupal.org/files/issues/user-module-session-3.patch (2.04 KB) Sorry. The patch was for a lot of things by mistake. Here is the patch only for user.module. As for ini_set(), I can do: ini_set ("session.cookie_lifetime", number_of_seconds); But again, number_of_seconds has to be stored in a varaible, which will be another option. Will we get the objection "no more options" again? However, as I understand it, ini_set() has to be called before session_start() which is the first thing in includes/session.inc. Can variable_get() be called that early in Drupal life span to get the stored value? On a related note, why don't we have the following defaults set in conf.php, so regardless of .htaccess or php.ini, we ensure they are set in Drupal no matter what? ini_set ("session.cache_expire", 200000); ini_set ("session.gc_maxlifetime", 200000); ini_set ("session.auto_start", 0); ini_set ("session.save_handler", "user"); ini_set ("session.cache_limiter", "none"); ini_set ("session.use_trans_sid", 0); ini_set ("session.use_cookies", 1); ini_set ("session.use_only_cookies", 1); This also has the advantage that every domain in a multi-domain installation can have their own settings, if needed, instead of just one .htaccess. ------------------------------------------------------------------------ February 12, 2005 - 17:15 : Abalieno Excuse me, I'm one of the less experienced user of Drupal out there. This problem was HUGE for me because my installation, by default, remembered the sessions only for 20 minutes, then forcing me to relog. I've asked on the mailing list, I had peoples trying to register at my site to figure it out, I asked in other mailing lists. Nothing. At the end I discovered that it wasn't Drupal but a variable into php.ini (session.gc_maxlifetime). Noone was able to point me in the right direction. The last message I received from Mark Quinn is: "As for your PHP session issue, you may want to try and take it up with your hosting service, or wait until drupal properly resolves the issue with an application-managed cookie; I've not yet seen any discussion patches come through on this, myself. Maybe the Usual Suspects may have a better idea of whether someone is working on this part of the code, at present." At the end I was able to fix this by changing the php.ini. But it was a pain. I don't know from where comes the 'design choice' of 'not another setting'. This is a *basic* feature that on EVERY other engine can be set in the option. What is better? Something that you can notice easily in a configuration page or something that was set by someone else and there's no other way to change than to hack it yourself because the engine isn't flexible? "This a (very) advanced feature, that will never have to be changed on a site." No, this is a BASIC feature on EVERY engine. In general the users of a site need to set this by themselves, some want to be anonymous, some want to log it just once and forget, some want to log in once and remain logged forever. Not only this should be an option, but this should be also defined, by default, in the permissions panel. So that each member can set the cookie in the preferred way. ------------------------------------------------------------------------ February 12, 2005 - 18:47 : kbahey This is exactly what I mean: we removed a feature and did not provide a replacement. My attempt to reintroduce the feature, admittedly having some drawbacks, was dimissed with no alternative provided. Can we discuss the alternatives, and implement a solution before 4.6 is out? ------------------------------------------------------------------------ February 12, 2005 - 22:57 : Bèr Kessels I did provide an alternative, only not in so much words: Use variable_set("site_cookie_lifetime",200000) and then ini_set to enforce a correct *default* lifetime. This will do for 99% ofthe drupal sites and users. For those 1% that want to control everything: use a setting in settings.php: # Variable overrides: # # To override specific entries in the 'variable' table for this site, # set them here. You usually don't need to use this feature. This is # useful when used in a configuration file for a vhost or directory, # rather than the default settings.php. Any configuration setting from # the variable table can be given a new value. # # $conf = array( # 'site_name' => 'My Drupal site', # 'theme_default' => 'pushbutton', # 'anonymous' => 'Visitor' # ); Really: I visit a lot of geeky pages everyday, but never, ever have i even there seen an option to say how long a cookie will live on my HDD. Let alone on the joe-average site. We really do not want joe-average to give answers to questions like "how long should you remain logged in". The machines i.c..w admins should make that choice for him. ------------------------------------------------------------------------ February 12, 2005 - 23:19 : Anonymous To start with the last comment about no sites doing this, Google's own Gmail has a checkbox that says : "Keep me logged in for two weeks". Hotmail and Yahoo both have a check box "remember my password on this computer". Even Slashdot has such an option. So, it is a fairly common way to do things, and was present in Drupal but taken away. Now to the solution: You say "Use variable_set("site_cookie_lifetime",200000) and then ini_set to enforce a correct *default* lifetime. " So, we are still going to have an option to set and save the site_cookie_lifetime. So, I assume that part of the user.module patch will remain. Good! Now, for the ini_set() part. I see that it has to be called before session_start() is called. I asked above where it should go. Should it go in session.inc? If so, I can add it there, and that would be that. Regarding the second option: Which variable specifically should be overriden in the $conf array to achieve this? ------------------------------------------------------------------------ February 12, 2005 - 23:20 : kbahey Oops! That was me. Drupal.org logged me out. Sort of relevant to the discussion ;-) ------------------------------------------------------------------------ February 12, 2005 - 23:32 : Bèr Kessels to clarify: an option is: Session lifetime: [400000_________] provide the lifetime of your users sessions in seconds. A user option for this would be: username [__________] password [__________] session lifetime [__________] Those are unwanted. Wanted, however is: * No options in the interface, but in code only! * username [__________] password [__________] remember [x] That is my point, and not that we should not have a lifetime in PHP. ------------------------------------------------------------------------ February 12, 2005 - 23:41 : kbahey No one suggested that we should give the end user, nor the sysadmin the number of seconds. Even the patch as it stands today does not do that, but displays: 1 day, 2 days, 1 week, 2 weeks, 1 month, ....etc. So, on the login screen, there would be a "remember me" checkbox. Then in the admin screens there would be an option like the one above for how long this 'remembering' should be. If someone answers the last two paragraphs in #69, I can get this done soon. ------------------------------------------------------------------------ February 13, 2005 - 00:23 : Abalieno Again I cannot code myself but I think I can provide a good solution. Why not to add options that are configurable, but just not under the eyes of everyone? This system should work this way: - By default Drupal should do what it does already. Users are automatically logged in for one month (about 2000000 seconds of the default option). Not using php.ini or .htaccess but its own cookies. - Then you add an option and link it to the permission pages. - The admin of the site will access a special page where he can set: * The default behaviour of the site for EVERY user. -> One day, one week , two weeks, a month, three moths, forever. Depending on this setting the variable will be reset as the standard option for every new user. Then, in the case the user is in a usergroup flagged to administer this option, each user has a new option in the config page of his account: * For how long you want Drupal to log you in automatically? -> One day, one week , two weeks, a month, three moths, forever. * A checkbox to remain anonymous even when logged in. This makes no option appear on the homepage, it allows Drupal to be set, by default, in an optimal way that doesn't need to be changed. But, in the case it's needed, it provides configurable options that both the admin and each user can set. Imho, the "remember me" box on the homepage isn't needed. Drupal is ok how it is already. What it needs is configurable options and permissions on their own page (the same page where you can set the timezone). ------------------------------------------------------------------------ February 13, 2005 - 06:19 : Bèr Kessels I do no know if it is me that is not clear, or you that cannot understand me: I, and might be only me, do *not* want any new config option in the config screens. Be it textareas where you can add seconds, or dropdowns where you can set weeks, days etc. Why not? I think this config option is 1) confusing and 2) unnessecary for 99% of the drupal cases. On top of that: this is a one-tim setting. On top of thát, it is a setting only advanced users will want to tweak (the 1%). So I am suggesting you code whatever you want to code for it, cookies, init, php.ini settings, whatever. But do /not/ add new config options in the admin or user screens. People (that 1%) can then set the variable in settings.php ------------------------------------------------------------------------ February 13, 2005 - 07:15 : stefan nagtegaal I totally agree with Ber on this one.. We already have to much 'setup-once-and-never-change-again' options inside drupal which are UI-clutter.. This, is in my opinion an option of the same type. You set this once, and never look at it again.. So, absolutely a -1 to put on one of the options screens again.. ------------------------------------------------------------------------ February 13, 2005 - 13:39 : kbahey Attachment: http://drupal.org/files/issues/user-module-session-4.patch (1.27 KB) Here is a new version of the patch. There is no configuration option anymore. The default is set to 30 days where the session cookie is valid for. This value can be overriden by setting a line for it in the $conf variable in the settings.php file under the sites directory. How does that look to others? ------------------------------------------------------------------------ February 13, 2005 - 13:55 : Goba How many times should I ask for ini_set()-ing the session cookie lifetime, and using less code, instead of hacking around with overwritinng the cookie? Please! Yes, the session.inc code is included from bootstrap.inc after variable_get and the settings.php file data is already available! I know this not because I know Drupal code, but because I just looked into index.php, and followed the includes... ------------------------------------------------------------------------ February 13, 2005 - 14:10 : kbahey Attachment: http://drupal.org/files/issues/session-inc.patch (1.11 KB) Here is the patch using ini_set() for the session.cookie_lifetime override. Ber, Stefan, and Goba: do you have any more concerns/objections? ------------------------------------------------------------------------ February 13, 2005 - 14:17 : Goba Yes, you have at least one typo in the comment: "by by", plus you don't need to have one intermediate variable, you can directly include the return value of variable_get() in the ini_set() call. Plus since you override the .htaccess setting here, .htaccess needs a comment in it that the setting is overwritten in the session.inc file. Or better, the .htaccess setting needs to be commented out and it should be noted there that this is set programatically. So someone who changes the .htaccess should not expect the setting to change. This is crucial to make this stuff be easy to handle from the support viewpoint later... ------------------------------------------------------------------------ February 13, 2005 - 14:29 : stefan nagtegaal Maybe it would also be handy - for codewise cosistency - to use Doxygen comments.... ------------------------------------------------------------------------ February 13, 2005 - 15:02 : kbahey Attachment: http://drupal.org/files/issues/session-inc-3.patch (2.25 KB) If it was up to me, I would have moved all the .htaccess php values and flag into Drupal itself using ini_set. This way, it does not matter who takes precedence over whom (.htaccess, global php.ini, php.ini in users' public_html directory, ...etc.). However, different PHP versions have different permissions and defaults, and this would not be possible until everyone settles down on PHP5. Goba, I know that it can all be on one line. My personal preference is to break it down to be more readable. However, for the sake of being consistent with everyones' else code, I will do it your way. Here is a version of the patch that: 1. Makes it all on one line (variable_get is not stored in a temporary variable) 2. Makes the comment doxygen style 3. Comments out the variable in .htaccess 4. Adds the values for ensuring that PHP would not append the PHPSESSID to the url, and only uses cookies to .htaccess ------------------------------------------------------------------------ February 13, 2005 - 16:43 : Bèr Kessels kbahey, Thank you a lot for your work and efforts. Please do not get our (mine, Gobas and Stefans) objections as nitpicking or as criticism. On contrary. The very fact that I am commenting on your code is that I see it valuable. But I saw it going into That Heap Of Patches That Will Never BE Committed [tm]. Simply because of code style, config options and all the likes. I am confident thatboth Goba and Stefan, just like me, liked the fact there is work done on important issues. Thanks again for your time, and most of all for not getting annoyed by people lilke me hammering (but not committing code myself) on issues! we need people like you much more than people like me (only giving critique on ones work). Bèr ------------------------------------------------------------------------ February 13, 2005 - 17:34 : kbahey Ber, Thanks for taking the time to clarify, and for the clarification. I did not take the comments negatively, and they did not upset me (not a lot anyway). I have tested the cookie, and it works well on my test machine. I hope it does the same on my hosting machine, which I cannot upgrade to CVS until 4.6 stabilizes a bit. On that machine, I had troubles with php.ini (in public_html with PHP SuExec) and .htaccess, and confused precedence I hope Dries applies this before 4.6 is released. Thanks again. ------------------------------------------------------------------------ February 13, 2005 - 23:03 : jvandyk Tested and works on my test machine (PHP 4.3.10, MySQL 4.19, Drupal HEAD). -- View: http://drupal.org/node/2974 Edit: http://drupal.org/project/comments/add/2974

1 0

[drupal-devel] [task] Pager Queries, Node Access and Performance
by Anonymous 14 Feb '05

14 Feb '05

Project: Drupal Version: cvs Component: node.module Category: tasks Priority: normal Assigned to: Anonymous Reported by: pyromanfo Updated by: Anonymous Status: patch I haven't run numbers, but subjectively my test site seems much faster with the patch in the case where there is a nid=0 row, a bit faster where there is no such row (but not as obvious), and about the same with node access turned off. Anonymous Previous comments: ------------------------------------------------------------------------ February 1, 2005 - 15:54 : pyromanfo I've got a Drupal 4.5.2 site running with Taxonomy Access Control. When you go to view a taxonomy and it tries to do an access check, the SQL query takes literally minutes to complete. This is on a dual Xeon box, it's rather beefy and runs two other sites (vBulletin and a hacked up Postnuke site) just fine. The problem comes from the taxonomy_select_nodes function $sql_count = 'SELECT COUNT(DISTINCT(n.nid)) FROM {node} n '. node_access_join_sql() .' INNER JOIN {term_node} tn ON n.nid = tn.nid WHERE tn.tid IN ('. $str_tids .') AND n.status = 1 AND '. node_access_where_sql(); Which ends up looking like SELECT COUNT(DISTINCT(n.nid)) FROM node n INNER JOIN node_access na ON (na.nid = 0 OR na.nid = n.nid) INNER JOIN term_node tn0 ON n.nid = tn0.nid WHERE n.status = 1 AND na.grant_view = 1 AND CONCAT(na.realm, na.gid) IN ('all0','term_access1') AND tn0.tid IN (9) This will take several minutes to complete. The query next to it that selected the nodes themselves took 3 minutes to complete. Where is the problem exactly? I've seen elsewhere that this is a problem for node_privacy_byrole as well, which makes sense because really all the various access control modules are contributing here is an extra entry in the concat search. Let me know if you need any details of how the MySQL server is configured. Here's some data from my drupal database mysql> select count(nid) from node; +------------+ | count(nid) | +------------+ | 10376 | +------------+ 1 row in set (0.00 sec) mysql> select count(nid) from node_access; +------------+ | count(nid) | +------------+ | 41508 | +------------+ 1 row in set (0.00 sec) Drupal performs very snappily with the node_access stuff disabled, and otherwise it performs very well. A three minute increase in the execute time of an SQL query is just strange, there's something odd going on here I think. ------------------------------------------------------------------------ February 3, 2005 - 21:02 : pyromanfo Reducing the size of the 'realm' field of node_access to something manageable like 12 instead of the current 255 speeds up the query tremendously, still it's rather slow. I also added a search index on the grant_view permission, since the view permission is the only permission you'd want to check for more than one node usually. It did speed it up, however it still take a minute or so on my little test box. I'm going to tweak with it some more and see if I can get it performing better. ------------------------------------------------------------------------ February 3, 2005 - 23:12 : killes(a)www.drop.org Could you give us an estimate with numbers? Or even better, make available the database for testing? ------------------------------------------------------------------------ February 4, 2005 - 19:59 : pyromanfo I can't really make the database available without going through and setting all the text and titles to something random and the uid's to 2, I may try that. I can give you some more numbers tonight though when I start messing with it again. You want execution times and table sizes? Any other specific numbers? Should I just dump show status? ------------------------------------------------------------------------ February 4, 2005 - 20:21 : killes(a)www.drop.org Averages of execution times before and after would be fine, I think. The db would be great of course You could simple set all text to "foo" and email addresses to "root@localhost". ------------------------------------------------------------------------ February 5, 2005 - 17:31 : pyromanfo I'm running these tests on a similar database I haven't been developing on. Created from the same source data with the same script though, I just haven't been hacking it up and it runs on faster hardware. SELECT COUNT(DISTINCT(n.nid)) FROM node n INNER JOIN node_access na ON (na.nid = 0 OR na.nid = n.nid) INNER JOIN term_node tn0 ON n.nid = tn0.nid WHERE n.status = 1 AND na.grant_view = 1 AND CONCAT(na.realm, na.gid) IN ('all0','term_access1') AND tn0.tid IN (2); +------------------------+ | COUNT(DISTINCT(n.nid)) | +------------------------+ | 2993 | +------------------------+ 1 row in set (7 min 4.00 sec) Then I ran the following queries alter table node_access modify realm varchar(12) NOT NULL default ''; alter table node_access add index view_indx (nid,gid,realm,grant_view); Then I ran mysqlcheck --auto_repair -os Then the query again SELECT COUNT(DISTINCT(n.nid)) FROM node n INNER JOIN node_access na ON (na.nid = 0 OR na.nid = n.nid) INNER JOIN term_node tn0 ON n.nid = tn0.nid WHERE n.status = 1 AND na.grant_view = 1 AND CONCAT(na.realm, na.gid) IN ('all0','term_access1') AND tn0.tid IN (2); +------------------------+ | COUNT(DISTINCT(n.nid)) | +------------------------+ | 2993 | +------------------------+ 1 row in set (6 min 18.62 sec) Not as big a speed boost as I thought I was seeing unfortunately. Here's 'show variables' mysql> show variables; +---------------------------------+---------------------------------------------------+ | Variable_name | Value | +---------------------------------+---------------------------------------------------+ | back_log | 50 | | basedir | /usr/local/mysql/ | | binlog_cache_size | 32768 | | bulk_insert_buffer_size | 8388608 | | character_set_client | latin1 | | character_set_connection | latin1 | | character_set_database | latin1 | | character_set_results | latin1 | | character_set_server | latin1 | | character_set_system | utf8 | | character_sets_dir | /usr/local/mysql/share/mysql/charsets/ | | collation_connection | latin1_swedish_ci | | collation_database | latin1_swedish_ci | | collation_server | latin1_swedish_ci | | concurrent_insert | ON | | connect_timeout | 5 | | datadir | /usr/local/mysql/var/ | | date_format | %Y-%m-%d | | datetime_format | %Y-%m-%d %H:%i:%s | | default_week_format | 0 | | delay_key_write | ON | | delayed_insert_limit | 100 | | delayed_insert_timeout | 300 | | delayed_queue_size | 1000 | | expire_logs_days | 0 | | flush | OFF | | flush_time | 0 | | ft_boolean_syntax | + -><()~*:""&| | | ft_max_word_len | 84 | | ft_min_word_len | 4 | | ft_query_expansion_limit | 20 | | ft_stopword_file | (built-in) | | group_concat_max_len | 1024 | | have_archive | NO | | have_bdb | NO | | have_compress | YES | | have_crypt | YES | | have_csv | NO | | have_example_engine | NO | | have_geometry | YES | | have_innodb | YES | | have_isam | NO | | have_ndbcluster | NO | | have_openssl | NO | | have_query_cache | YES | | have_raid | NO | | have_rtree_keys | YES | | have_symlink | YES | | init_connect | | | init_file | | | init_slave | | | innodb_additional_mem_pool_size | 1048576 | | innodb_autoextend_increment | 8 | | innodb_buffer_pool_awe_mem_mb | 0 | | innodb_buffer_pool_size | 8388608 | | innodb_data_file_path | ibdata1:10M:autoextend | | innodb_data_home_dir | | | innodb_fast_shutdown | ON | | innodb_file_io_threads | 4 | | innodb_file_per_table | OFF | | innodb_locks_unsafe_for_binlog | OFF | | innodb_flush_log_at_trx_commit | 1 | | innodb_flush_method | | | innodb_force_recovery | 0 | | innodb_lock_wait_timeout | 50 | | innodb_log_arch_dir | | | innodb_log_archive | OFF | | innodb_log_buffer_size | 1048576 | | innodb_log_file_size | 5242880 | | innodb_log_files_in_group | 2 | | innodb_log_group_home_dir | ./ | | innodb_max_dirty_pages_pct | 90 | | innodb_table_locks | ON | | innodb_max_purge_lag | 0 | | innodb_mirrored_log_groups | 1 | | innodb_open_files | 300 | | innodb_thread_concurrency | 8 | | interactive_timeout | 28800 | | join_buffer_size | 131072 | | key_buffer_size | 8388600 | | key_cache_age_threshold | 300 | | key_cache_block_size | 1024 | | key_cache_division_limit | 100 | | language | /usr/local/mysql/share/mysql/english/ | | large_files_support | ON | | license | GPL | | local_infile | ON | | locked_in_memory | OFF | | log | OFF | | log_bin | OFF | | log_error | | | log_slave_updates | OFF | | log_slow_queries | OFF | | log_update | OFF | | log_warnings | 1 | | long_query_time | 10 | | low_priority_updates | OFF | | lower_case_file_system | OFF | | lower_case_table_names | 0 | | max_allowed_packet | 1048576 | | max_binlog_cache_size | 4294967295 | | max_binlog_size | 1073741824 | | max_connect_errors | 10 | | max_connections | 100 | | max_delayed_threads | 20 | | max_error_count | 64 | | max_heap_table_size | 16777216 | | max_insert_delayed_threads | 20 | | max_join_size | 4294967295 | | max_length_for_sort_data | 1024 | | max_relay_log_size | 0 | | max_seeks_for_key | 4294967295 | | max_sort_length | 1024 | | max_tmp_tables | 32 | | max_user_connections | 0 | | max_write_lock_count | 4294967295 | | myisam_data_pointer_size | 4 | | myisam_max_extra_sort_file_size | 2147483648 | | myisam_max_sort_file_size | 2147483647 | | myisam_recover_options | OFF | | myisam_repair_threads | 1 | | myisam_sort_buffer_size | 8388608 | | net_buffer_length | 16384 | | net_read_timeout | 30 | | net_retry_count | 10 | | net_write_timeout | 60 | | new | OFF | | old_passwords | ON | | open_files_limit | 1024 | | pid_file | ********** | | port | 3306 | | preload_buffer_size | 32768 | | protocol_version | 10 | | query_alloc_block_size | 8192 | | query_cache_limit | 1048576 | | query_cache_min_res_unit | 4096 | | query_cache_size | 0 | | query_cache_type | ON | | query_cache_wlock_invalidate | OFF | | query_prealloc_size | 8192 | | range_alloc_block_size | 2048 | | read_buffer_size | 131072 | | read_only | OFF | | read_rnd_buffer_size | 262144 | | relay_log_purge | ON | | rpl_recovery_rank | 0 | | secure_auth | OFF | | server_id | 0 | | skip_external_locking | ON | | skip_networking | OFF | | skip_show_database | OFF | | slave_net_timeout | 3600 | | slow_launch_time | 2 | | socket | /tmp/mysql.sock | | sort_buffer_size | 2097144 | | sql_mode | | | storage_engine | MyISAM | | sync_binlog | 0 | | sync_frm | ON | | system_time_zone | EST | | table_cache | 64 | | table_type | MyISAM | | thread_cache_size | 0 | | thread_stack | 196608 | | time_format | %H:%i:%s | | time_zone | SYSTEM | | tmp_table_size | 33554432 | | tmpdir | | | transaction_alloc_block_size | 8192 | | transaction_prealloc_size | 4096 | | tx_isolation | REPEATABLE-READ | | version | 4.1.7 | | version_comment | Source distribution | | version_compile_machine | i686 | | version_compile_os | pc-linux | | wait_timeout | 28800 | +---------------------------------+---------------------------------------------------+ 176 rows in set (0.03 sec) I'm attaching my test database which takes several minutes to execute a similar query If you want me to run the tests on the test database I can do that, it's just on a much skimpier machine and it takes quite a bit longer (Athlon 600, 256MB Ram vs Dual Xeon). ------------------------------------------------------------------------ February 5, 2005 - 17:33 : pyromanfo Oops, preview removed the test db, here it is. ------------------------------------------------------------------------ February 5, 2005 - 17:38 : pyromanfo It's too big, it's 4.47 MB, I'm gonna put it up on GWJ and link to it. Get it here [1] [1] http://www.gamerswithjobs.com/testdb/drupal_test_db.tar.bz2 ------------------------------------------------------------------------ February 5, 2005 - 17:46 : pyromanfo Attachment: http://drupal.org/files/issues/drupal_test_db.tar.bz2 (734.78 KB) Nevermind, I left in some sensitive info and forgot to wipe the search index, now it's much smaller. ------------------------------------------------------------------------ February 7, 2005 - 22:58 : killes(a)www.drop.org Thanks, but I think you stripped the DB a bit too much. If I execute the query it is very fast because it does not return anything... ------------------------------------------------------------------------ February 7, 2005 - 23:12 : killes(a)www.drop.org When I use the query from your original post, I get some result. I guess I will write a script and run some tests over night... ------------------------------------------------------------------------ February 7, 2005 - 23:55 : killes(a)www.drop.org Ok, here are some preliminary results: - The unchanged query from the original post runs nearly 14 minutes on the hardware redLED lent to us. - If I shorten the varchar field to 18 from 255, it gets about halfed. - neither adding an index for the realm nor the grant_view field does make a change - Omitting the "na.nid = 0 OR" part makes the query very fast (quarter of a second) - SELECT COUNT(DISTINCT(n.nid)) FROM node n INNER JOIN node_access na ON ( na.nid = n.nid) left jo in node_access nb ON nb.nid = 0 INNER JOIN term_node tn0 ON n.nid = tn0.nid WHERE n.status = 1 AND ((n a.grant_view = 1 AND CONCAT(na.realm, na.gid) IN ('all0','term_access1')) OR (nb.grant_view = 1 AND CON CAT(nb.realm, nb.gid) IN ('all0','term_access1'))) AND tn0.tid IN (9); This query is still very much faster (about half a second) than the original query and gives the same result for your database. I don't know if it is fully equivalent, though. Note that it is a left join. ------------------------------------------------------------------------ February 8, 2005 - 13:06 : pyromanfo The na.nid = 0 stuff is basically a catchall, that's sort of a default permission that overrides anything else. This is good when you're not using node_access, however when you are the na.nid=0 stuff probably isn't there, I know Node Privacy By Role and Taxonomy Access Control both delete the entries for nid=0. Perhaps it's a good idea to control this another way, by setting a variable and using that to determine whether or not some sort of node_access is installed, as it seems using nid = 0 as a default simply kills performance. ------------------------------------------------------------------------ February 8, 2005 - 13:08 : pyromanfo Oh, and thanks for taking a good look at it killes, I should've explained better. The DB I provided, the test DB, has a large number of nodes in tid=9, the other DB I ran the tests on has alot in tid=2. ------------------------------------------------------------------------ February 8, 2005 - 16:17 : Anonymous The more I think about replacing nid=0 with a configuration setting the more it makes sense. The two major node access modules right now have to have a seperate "enable/disable" setting to deal with nid=0, since it overrides everything. In fact, it seems like it's only purpose it to override all other node access settings with a universal default. Why not instead have a global configuration setting under permissions that enables or disable node level permissions, then the various node access modules wouldn't need a seperate disable/enable setting. Furthermore, this performance problem would disappear, because either the node_access stuff would be there and you aren't looking for nid=0, or it isn't and you don't care what's in node_access. It seems it'd save queries to node access when you're not using node level permissions, and make it cleaner to implement a node access modifying module. ------------------------------------------------------------------------ February 8, 2005 - 16:17 : pyromanfo I hate the site randomly logging you out, that was me above. ------------------------------------------------------------------------ February 8, 2005 - 19:23 : JonBob "The more I think about replacing nid=0 with a configuration setting the more it makes sense. The two major node access modules right now have to have a seperate "enable/disable" setting to deal with nid=0, since it overrides everything. In fact, it seems like it's only purpose it to override all other node access settings with a universal default." This is not the case. I think you're confusing the case of nid=0 with the case of realm=all. As an example of the utility of nid=0, consider the scenario given in the example access module [2]. This module implements a simple system whereby anyone can view "public" nodes, and only users with a specific permission can view "private" nodes. This can be rephrased as follows: anonymous users can only view "public" nodes, while privileged users can view all nodes. A grant with nid=0 takes care of the second case, so that the module only needs to deal with granting permission to view the public nodes as necessary. Gerhard's query works in the case where every node has at least one node_access entry with its nid, but I'm afraid it fails in the above example. A node that is private would not have any corresponding entry, and so the inner join would fail for it, returning no record. The left join doesn't even have a chance to be matched. It would be worth benchmarking the scenario in which the nid=0 possibility is removed from the query, but the node_access table has an entry added for each node instead of a single nid=0 entry. If this is faster than the OR in the join, the extra DB storage is probably worth the change. [2] http://drupaldocs.org/api/head/file/contributions/docs/developer/examples/n… ------------------------------------------------------------------------ February 9, 2005 - 01:08 : pyromanfo "As an example of the utility of nid=0, consider the scenario given in the example access module. This module implements a simple system whereby anyone can view "public" nodes, and only users with a specific permission can view "private" nodes. This can be rephrased as follows: anonymous users can only view "public" nodes, while privileged users can view all nodes. A grant with nid=0 takes care of the second case, so that the module only needs to deal with granting permission to view the public nodes as necessary. " I think there's a bug in the module, in the docs in the top shouldn't the initial SQL statement be adding that permission to gid=1? Though you're right, I'd forgotten because I hadn't seen any of the current modules use it. As for that performance comparison mysql> SELECT COUNT(DISTINCT(n.nid)) FROM node n INNER JOIN term_node tn0 ON n.nid = tn0.nid INNER JOIN node_access na ON (na.nid = n.nid) WHERE n.status = 1 AND na.grant_view = 1 AND CONCAT(na.realm, na.gid) IN ('all0','term_access1') AND tn0.tid IN (2); +------------------------+ | COUNT(DISTINCT(n.nid)) | +------------------------+ | 2993 | +------------------------+ 1 row in set (0.23 sec) mysql> update node_access set realm='blam' where realm='term_access'; Query OK, 41508 rows affected (5.42 sec) Rows matched: 41508 Changed: 41508 Warnings: 0 mysql> insert into node_access (nid,gid,realm,grant_view) values (0,1,'term_access',1); Query OK, 1 row affected (0.00 sec) mysql> SELECT COUNT(DISTINCT(n.nid)) FROM node n INNER JOIN term_node tn0 ON n.nid = tn0.nid INNER JOIN node_access na ON (na.nid = 0 OR na.nid=n.nid) WHERE n.status = 1 AND na.grant_view = 1 AND CONCAT(na.realm, na.gid) IN ('all0','term_access1') AND tn0.tid IN (2); +------------------------+ | COUNT(DISTINCT(n.nid)) | +------------------------+ | 2993 | +------------------------+ 1 row in set (0.16 sec) So I don't think it improves performance for the nid=0 case, though it is suprisingly slow. ------------------------------------------------------------------------ February 9, 2005 - 01:57 : moshe weitzman perhaps we should split the query into 2 queries in order to avoid the 'na.nid = 0' part of the JOIN. Alternatively, perhaps we could inject that clause only when needed using the db_rewrite_sql hook. not sure. ------------------------------------------------------------------------ February 10, 2005 - 04:36 : pyromanfo Splitting it into two queries would make sense I think. Basically you could have it entirely in node_access_join_sql and node_access_where_sql. There we can check for the nid=0 case for any gids and realms. Then if it returns true, we know we don't have to check anything else, so we return null for the SQL to enter. We could even store this in a static variable to avoid calling this twice for join_sql and where_sql So in the nid=0 case, this should be similarly fast as we're removing a table from the JOIN in exchange for checking node_access for one nid. mysql> select count(nid) from node_access where nid=0 and gid=1 and realm in ('all','term_access','blah'); +------------+ | count(nid) | +------------+ | 0 | +------------+ 1 row in set (0.37 sec) SELECT COUNT(DISTINCT(n.nid)) FROM node n INNER JOIN term_node tn0 ON n.nid = tn0.nid WHERE n.status = 1 AND tn0.tid IN (9); +------------------------+ | COUNT(DISTINCT(n.nid)) | +------------------------+ | 4202 | +------------------------+ 1 row in set (0.15 sec) SELECT COUNT(DISTINCT(n.nid)) FROM node n INNER JOIN node_access na ON (na.nid = 0 OR na.nid = n.nid) INNER JOIN term_node tn0 ON n.nid = tn0.nid WHERE n.status = 1 AND na.grant_view = 1 AND CONCAT(na.realm, na.gid) IN ('all0','blah1') AND tn0.tid IN (9); +------------------------+ | COUNT(DISTINCT(n.nid)) | +------------------------+ | 4202 | +------------------------+ 1 row in set (1.97 sec) It looks to be faster, actually. Then, when we don't get a result, we check node access with the 'na.nid=0 OR' removed. This is certain to be faster as the test have shown, taking the query down from 7 minutes to a matter of seconds. In either case, this method would be faster. Plus, it wouldn't require any changes to other modules, simply change the node_access_where_sql and node_access_join_sql functions. JonBob, would you be okay with that? I can code it if you think you'll accept the patch. ------------------------------------------------------------------------ February 10, 2005 - 13:38 : Anonymous Splitting the queries into two is probably the only way to go. However, I'd like to avoid calling the nid = 0 query if possible. My Drupal installations that use node access do not use nid = 0 at all. I think we could store the available (nid = 0) => realm combinations somewhere and only execute the query if a matching condition is found. Gerhard ------------------------------------------------------------------------ February 10, 2005 - 15:14 : moshe weitzman that nid=0 is going to be very fast when done all by itself. no need to introduce any other mechanism IMO. testing will tell us for sure though ------------------------------------------------------------------------ February 10, 2005 - 18:31 : killes(a)www.drop.org The nid = 0 query would be executed for every node query on a page call even if your access module doesn't use nid = 0 at all. Most implementations do not use nid = 0. If we can avoid a query we should do so. ------------------------------------------------------------------------ February 11, 2005 - 20:00 : JonBob Attachment: http://drupal.org/files/issues/node_access.patch (2.91 KB) Here's a patch that splits the nid=0 check off into a separate query. If the user has such a grant, things should be really fast since we don't need any join or even DISTINCT() check. If not, the join is done as before, but without the "OR nid=0" piece. This ideally needs performance checking on systems using node access with nid=0 rows, using it without them, and not using it at all. ------------------------------------------------------------------------ February 14, 2005 - 12:33 : killes(a)www.drop.org Ok, here is a first test result: After installing the node_privacy_by_role module for the drupal.org database, the Drupal install became unusable for all but the No. 1 user and the load went high due to many hanging mysql processes. After applying the patch the site does work again. The module seems to generate one entry per node and role in the node_access table. I did not make any benchmarks, the result is so obvious. ------------------------------------------------------------------------ February 14, 2005 - 14:10 : killes(a)www.drop.org I've used a script provided by JonBob to create entries in the node_access table and installed the node_access_example module. The script generated about 500 node_access entries. I then benchmarked the tracker page. Results: 11.78 +- 0.48 s with patch 17.04 +- 0.70 s without patch The patched version needs only 70% of the time the non patched version needs. -- View: http://drupal.org/node/16558 Edit: http://drupal.org/project/comments/add/16558

1 0

[drupal-devel] [task] Pager Queries, Node Access and Performance
by killes 14 Feb '05

14 Feb '05

Project: Drupal Version: cvs Component: node.module Category: tasks Priority: normal Assigned to: Anonymous Reported by: pyromanfo Updated by: killes(a)www.drop.org Status: patch I've used a script provided by JonBob to create entries in the node_access table and installed the node_access_example module. The script generated about 500 node_access entries. I then benchmarked the tracker page. Results: 11.78 +- 0.48 s with patch 17.04 +- 0.70 s without patch The patched version needs only 70% of the time the non patched version needs. killes(a)www.drop.org Previous comments: ------------------------------------------------------------------------ February 1, 2005 - 16:54 : pyromanfo I've got a Drupal 4.5.2 site running with Taxonomy Access Control. When you go to view a taxonomy and it tries to do an access check, the SQL query takes literally minutes to complete. This is on a dual Xeon box, it's rather beefy and runs two other sites (vBulletin and a hacked up Postnuke site) just fine. The problem comes from the taxonomy_select_nodes function $sql_count = 'SELECT COUNT(DISTINCT(n.nid)) FROM {node} n '. node_access_join_sql() .' INNER JOIN {term_node} tn ON n.nid = tn.nid WHERE tn.tid IN ('. $str_tids .') AND n.status = 1 AND '. node_access_where_sql(); Which ends up looking like SELECT COUNT(DISTINCT(n.nid)) FROM node n INNER JOIN node_access na ON (na.nid = 0 OR na.nid = n.nid) INNER JOIN term_node tn0 ON n.nid = tn0.nid WHERE n.status = 1 AND na.grant_view = 1 AND CONCAT(na.realm, na.gid) IN ('all0','term_access1') AND tn0.tid IN (9) This will take several minutes to complete. The query next to it that selected the nodes themselves took 3 minutes to complete. Where is the problem exactly? I've seen elsewhere that this is a problem for node_privacy_byrole as well, which makes sense because really all the various access control modules are contributing here is an extra entry in the concat search. Let me know if you need any details of how the MySQL server is configured. Here's some data from my drupal database mysql> select count(nid) from node; +------------+ | count(nid) | +------------+ | 10376 | +------------+ 1 row in set (0.00 sec) mysql> select count(nid) from node_access; +------------+ | count(nid) | +------------+ | 41508 | +------------+ 1 row in set (0.00 sec) Drupal performs very snappily with the node_access stuff disabled, and otherwise it performs very well. A three minute increase in the execute time of an SQL query is just strange, there's something odd going on here I think. ------------------------------------------------------------------------ February 3, 2005 - 22:02 : pyromanfo Reducing the size of the 'realm' field of node_access to something manageable like 12 instead of the current 255 speeds up the query tremendously, still it's rather slow. I also added a search index on the grant_view permission, since the view permission is the only permission you'd want to check for more than one node usually. It did speed it up, however it still take a minute or so on my little test box. I'm going to tweak with it some more and see if I can get it performing better. ------------------------------------------------------------------------ February 4, 2005 - 00:12 : killes(a)www.drop.org Could you give us an estimate with numbers? Or even better, make available the database for testing? ------------------------------------------------------------------------ February 4, 2005 - 20:59 : pyromanfo I can't really make the database available without going through and setting all the text and titles to something random and the uid's to 2, I may try that. I can give you some more numbers tonight though when I start messing with it again. You want execution times and table sizes? Any other specific numbers? Should I just dump show status? ------------------------------------------------------------------------ February 4, 2005 - 21:21 : killes(a)www.drop.org Averages of execution times before and after would be fine, I think. The db would be great of course You could simple set all text to "foo" and email addresses to "root@localhost". ------------------------------------------------------------------------ February 5, 2005 - 18:31 : pyromanfo I'm running these tests on a similar database I haven't been developing on. Created from the same source data with the same script though, I just haven't been hacking it up and it runs on faster hardware. SELECT COUNT(DISTINCT(n.nid)) FROM node n INNER JOIN node_access na ON (na.nid = 0 OR na.nid = n.nid) INNER JOIN term_node tn0 ON n.nid = tn0.nid WHERE n.status = 1 AND na.grant_view = 1 AND CONCAT(na.realm, na.gid) IN ('all0','term_access1') AND tn0.tid IN (2); +------------------------+ | COUNT(DISTINCT(n.nid)) | +------------------------+ | 2993 | +------------------------+ 1 row in set (7 min 4.00 sec) Then I ran the following queries alter table node_access modify realm varchar(12) NOT NULL default ''; alter table node_access add index view_indx (nid,gid,realm,grant_view); Then I ran mysqlcheck --auto_repair -os Then the query again SELECT COUNT(DISTINCT(n.nid)) FROM node n INNER JOIN node_access na ON (na.nid = 0 OR na.nid = n.nid) INNER JOIN term_node tn0 ON n.nid = tn0.nid WHERE n.status = 1 AND na.grant_view = 1 AND CONCAT(na.realm, na.gid) IN ('all0','term_access1') AND tn0.tid IN (2); +------------------------+ | COUNT(DISTINCT(n.nid)) | +------------------------+ | 2993 | +------------------------+ 1 row in set (6 min 18.62 sec) Not as big a speed boost as I thought I was seeing unfortunately. Here's 'show variables' mysql> show variables; +---------------------------------+---------------------------------------------------+ | Variable_name | Value | +---------------------------------+---------------------------------------------------+ | back_log | 50 | | basedir | /usr/local/mysql/ | | binlog_cache_size | 32768 | | bulk_insert_buffer_size | 8388608 | | character_set_client | latin1 | | character_set_connection | latin1 | | character_set_database | latin1 | | character_set_results | latin1 | | character_set_server | latin1 | | character_set_system | utf8 | | character_sets_dir | /usr/local/mysql/share/mysql/charsets/ | | collation_connection | latin1_swedish_ci | | collation_database | latin1_swedish_ci | | collation_server | latin1_swedish_ci | | concurrent_insert | ON | | connect_timeout | 5 | | datadir | /usr/local/mysql/var/ | | date_format | %Y-%m-%d | | datetime_format | %Y-%m-%d %H:%i:%s | | default_week_format | 0 | | delay_key_write | ON | | delayed_insert_limit | 100 | | delayed_insert_timeout | 300 | | delayed_queue_size | 1000 | | expire_logs_days | 0 | | flush | OFF | | flush_time | 0 | | ft_boolean_syntax | + -><()~*:""&| | | ft_max_word_len | 84 | | ft_min_word_len | 4 | | ft_query_expansion_limit | 20 | | ft_stopword_file | (built-in) | | group_concat_max_len | 1024 | | have_archive | NO | | have_bdb | NO | | have_compress | YES | | have_crypt | YES | | have_csv | NO | | have_example_engine | NO | | have_geometry | YES | | have_innodb | YES | | have_isam | NO | | have_ndbcluster | NO | | have_openssl | NO | | have_query_cache | YES | | have_raid | NO | | have_rtree_keys | YES | | have_symlink | YES | | init_connect | | | init_file | | | init_slave | | | innodb_additional_mem_pool_size | 1048576 | | innodb_autoextend_increment | 8 | | innodb_buffer_pool_awe_mem_mb | 0 | | innodb_buffer_pool_size | 8388608 | | innodb_data_file_path | ibdata1:10M:autoextend | | innodb_data_home_dir | | | innodb_fast_shutdown | ON | | innodb_file_io_threads | 4 | | innodb_file_per_table | OFF | | innodb_locks_unsafe_for_binlog | OFF | | innodb_flush_log_at_trx_commit | 1 | | innodb_flush_method | | | innodb_force_recovery | 0 | | innodb_lock_wait_timeout | 50 | | innodb_log_arch_dir | | | innodb_log_archive | OFF | | innodb_log_buffer_size | 1048576 | | innodb_log_file_size | 5242880 | | innodb_log_files_in_group | 2 | | innodb_log_group_home_dir | ./ | | innodb_max_dirty_pages_pct | 90 | | innodb_table_locks | ON | | innodb_max_purge_lag | 0 | | innodb_mirrored_log_groups | 1 | | innodb_open_files | 300 | | innodb_thread_concurrency | 8 | | interactive_timeout | 28800 | | join_buffer_size | 131072 | | key_buffer_size | 8388600 | | key_cache_age_threshold | 300 | | key_cache_block_size | 1024 | | key_cache_division_limit | 100 | | language | /usr/local/mysql/share/mysql/english/ | | large_files_support | ON | | license | GPL | | local_infile | ON | | locked_in_memory | OFF | | log | OFF | | log_bin | OFF | | log_error | | | log_slave_updates | OFF | | log_slow_queries | OFF | | log_update | OFF | | log_warnings | 1 | | long_query_time | 10 | | low_priority_updates | OFF | | lower_case_file_system | OFF | | lower_case_table_names | 0 | | max_allowed_packet | 1048576 | | max_binlog_cache_size | 4294967295 | | max_binlog_size | 1073741824 | | max_connect_errors | 10 | | max_connections | 100 | | max_delayed_threads | 20 | | max_error_count | 64 | | max_heap_table_size | 16777216 | | max_insert_delayed_threads | 20 | | max_join_size | 4294967295 | | max_length_for_sort_data | 1024 | | max_relay_log_size | 0 | | max_seeks_for_key | 4294967295 | | max_sort_length | 1024 | | max_tmp_tables | 32 | | max_user_connections | 0 | | max_write_lock_count | 4294967295 | | myisam_data_pointer_size | 4 | | myisam_max_extra_sort_file_size | 2147483648 | | myisam_max_sort_file_size | 2147483647 | | myisam_recover_options | OFF | | myisam_repair_threads | 1 | | myisam_sort_buffer_size | 8388608 | | net_buffer_length | 16384 | | net_read_timeout | 30 | | net_retry_count | 10 | | net_write_timeout | 60 | | new | OFF | | old_passwords | ON | | open_files_limit | 1024 | | pid_file | ********** | | port | 3306 | | preload_buffer_size | 32768 | | protocol_version | 10 | | query_alloc_block_size | 8192 | | query_cache_limit | 1048576 | | query_cache_min_res_unit | 4096 | | query_cache_size | 0 | | query_cache_type | ON | | query_cache_wlock_invalidate | OFF | | query_prealloc_size | 8192 | | range_alloc_block_size | 2048 | | read_buffer_size | 131072 | | read_only | OFF | | read_rnd_buffer_size | 262144 | | relay_log_purge | ON | | rpl_recovery_rank | 0 | | secure_auth | OFF | | server_id | 0 | | skip_external_locking | ON | | skip_networking | OFF | | skip_show_database | OFF | | slave_net_timeout | 3600 | | slow_launch_time | 2 | | socket | /tmp/mysql.sock | | sort_buffer_size | 2097144 | | sql_mode | | | storage_engine | MyISAM | | sync_binlog | 0 | | sync_frm | ON | | system_time_zone | EST | | table_cache | 64 | | table_type | MyISAM | | thread_cache_size | 0 | | thread_stack | 196608 | | time_format | %H:%i:%s | | time_zone | SYSTEM | | tmp_table_size | 33554432 | | tmpdir | | | transaction_alloc_block_size | 8192 | | transaction_prealloc_size | 4096 | | tx_isolation | REPEATABLE-READ | | version | 4.1.7 | | version_comment | Source distribution | | version_compile_machine | i686 | | version_compile_os | pc-linux | | wait_timeout | 28800 | +---------------------------------+---------------------------------------------------+ 176 rows in set (0.03 sec) I'm attaching my test database which takes several minutes to execute a similar query If you want me to run the tests on the test database I can do that, it's just on a much skimpier machine and it takes quite a bit longer (Athlon 600, 256MB Ram vs Dual Xeon). ------------------------------------------------------------------------ February 5, 2005 - 18:33 : pyromanfo Oops, preview removed the test db, here it is. ------------------------------------------------------------------------ February 5, 2005 - 18:38 : pyromanfo It's too big, it's 4.47 MB, I'm gonna put it up on GWJ and link to it. Get it here [1] [1] http://www.gamerswithjobs.com/testdb/drupal_test_db.tar.bz2 ------------------------------------------------------------------------ February 5, 2005 - 18:46 : pyromanfo Attachment: http://drupal.org/files/issues/drupal_test_db.tar.bz2 (734.78 KB) Nevermind, I left in some sensitive info and forgot to wipe the search index, now it's much smaller. ------------------------------------------------------------------------ February 7, 2005 - 23:58 : killes(a)www.drop.org Thanks, but I think you stripped the DB a bit too much. If I execute the query it is very fast because it does not return anything... ------------------------------------------------------------------------ February 8, 2005 - 00:12 : killes(a)www.drop.org When I use the query from your original post, I get some result. I guess I will write a script and run some tests over night... ------------------------------------------------------------------------ February 8, 2005 - 00:55 : killes(a)www.drop.org Ok, here are some preliminary results: - The unchanged query from the original post runs nearly 14 minutes on the hardware redLED lent to us. - If I shorten the varchar field to 18 from 255, it gets about halfed. - neither adding an index for the realm nor the grant_view field does make a change - Omitting the "na.nid = 0 OR" part makes the query very fast (quarter of a second) - SELECT COUNT(DISTINCT(n.nid)) FROM node n INNER JOIN node_access na ON ( na.nid = n.nid) left jo in node_access nb ON nb.nid = 0 INNER JOIN term_node tn0 ON n.nid = tn0.nid WHERE n.status = 1 AND ((n a.grant_view = 1 AND CONCAT(na.realm, na.gid) IN ('all0','term_access1')) OR (nb.grant_view = 1 AND CON CAT(nb.realm, nb.gid) IN ('all0','term_access1'))) AND tn0.tid IN (9); This query is still very much faster (about half a second) than the original query and gives the same result for your database. I don't know if it is fully equivalent, though. Note that it is a left join. ------------------------------------------------------------------------ February 8, 2005 - 14:06 : pyromanfo The na.nid = 0 stuff is basically a catchall, that's sort of a default permission that overrides anything else. This is good when you're not using node_access, however when you are the na.nid=0 stuff probably isn't there, I know Node Privacy By Role and Taxonomy Access Control both delete the entries for nid=0. Perhaps it's a good idea to control this another way, by setting a variable and using that to determine whether or not some sort of node_access is installed, as it seems using nid = 0 as a default simply kills performance. ------------------------------------------------------------------------ February 8, 2005 - 14:08 : pyromanfo Oh, and thanks for taking a good look at it killes, I should've explained better. The DB I provided, the test DB, has a large number of nodes in tid=9, the other DB I ran the tests on has alot in tid=2. ------------------------------------------------------------------------ February 8, 2005 - 17:17 : Anonymous The more I think about replacing nid=0 with a configuration setting the more it makes sense. The two major node access modules right now have to have a seperate "enable/disable" setting to deal with nid=0, since it overrides everything. In fact, it seems like it's only purpose it to override all other node access settings with a universal default. Why not instead have a global configuration setting under permissions that enables or disable node level permissions, then the various node access modules wouldn't need a seperate disable/enable setting. Furthermore, this performance problem would disappear, because either the node_access stuff would be there and you aren't looking for nid=0, or it isn't and you don't care what's in node_access. It seems it'd save queries to node access when you're not using node level permissions, and make it cleaner to implement a node access modifying module. ------------------------------------------------------------------------ February 8, 2005 - 17:17 : pyromanfo I hate the site randomly logging you out, that was me above. ------------------------------------------------------------------------ February 8, 2005 - 20:23 : JonBob "The more I think about replacing nid=0 with a configuration setting the more it makes sense. The two major node access modules right now have to have a seperate "enable/disable" setting to deal with nid=0, since it overrides everything. In fact, it seems like it's only purpose it to override all other node access settings with a universal default." This is not the case. I think you're confusing the case of nid=0 with the case of realm=all. As an example of the utility of nid=0, consider the scenario given in the example access module [2]. This module implements a simple system whereby anyone can view "public" nodes, and only users with a specific permission can view "private" nodes. This can be rephrased as follows: anonymous users can only view "public" nodes, while privileged users can view all nodes. A grant with nid=0 takes care of the second case, so that the module only needs to deal with granting permission to view the public nodes as necessary. Gerhard's query works in the case where every node has at least one node_access entry with its nid, but I'm afraid it fails in the above example. A node that is private would not have any corresponding entry, and so the inner join would fail for it, returning no record. The left join doesn't even have a chance to be matched. It would be worth benchmarking the scenario in which the nid=0 possibility is removed from the query, but the node_access table has an entry added for each node instead of a single nid=0 entry. If this is faster than the OR in the join, the extra DB storage is probably worth the change. [2] http://drupaldocs.org/api/head/file/contributions/docs/developer/examples/n… ------------------------------------------------------------------------ February 9, 2005 - 02:08 : pyromanfo "As an example of the utility of nid=0, consider the scenario given in the example access module. This module implements a simple system whereby anyone can view "public" nodes, and only users with a specific permission can view "private" nodes. This can be rephrased as follows: anonymous users can only view "public" nodes, while privileged users can view all nodes. A grant with nid=0 takes care of the second case, so that the module only needs to deal with granting permission to view the public nodes as necessary. " I think there's a bug in the module, in the docs in the top shouldn't the initial SQL statement be adding that permission to gid=1? Though you're right, I'd forgotten because I hadn't seen any of the current modules use it. As for that performance comparison mysql> SELECT COUNT(DISTINCT(n.nid)) FROM node n INNER JOIN term_node tn0 ON n.nid = tn0.nid INNER JOIN node_access na ON (na.nid = n.nid) WHERE n.status = 1 AND na.grant_view = 1 AND CONCAT(na.realm, na.gid) IN ('all0','term_access1') AND tn0.tid IN (2); +------------------------+ | COUNT(DISTINCT(n.nid)) | +------------------------+ | 2993 | +------------------------+ 1 row in set (0.23 sec) mysql> update node_access set realm='blam' where realm='term_access'; Query OK, 41508 rows affected (5.42 sec) Rows matched: 41508 Changed: 41508 Warnings: 0 mysql> insert into node_access (nid,gid,realm,grant_view) values (0,1,'term_access',1); Query OK, 1 row affected (0.00 sec) mysql> SELECT COUNT(DISTINCT(n.nid)) FROM node n INNER JOIN term_node tn0 ON n.nid = tn0.nid INNER JOIN node_access na ON (na.nid = 0 OR na.nid=n.nid) WHERE n.status = 1 AND na.grant_view = 1 AND CONCAT(na.realm, na.gid) IN ('all0','term_access1') AND tn0.tid IN (2); +------------------------+ | COUNT(DISTINCT(n.nid)) | +------------------------+ | 2993 | +------------------------+ 1 row in set (0.16 sec) So I don't think it improves performance for the nid=0 case, though it is suprisingly slow. ------------------------------------------------------------------------ February 9, 2005 - 02:57 : moshe weitzman perhaps we should split the query into 2 queries in order to avoid the 'na.nid = 0' part of the JOIN. Alternatively, perhaps we could inject that clause only when needed using the db_rewrite_sql hook. not sure. ------------------------------------------------------------------------ February 10, 2005 - 05:36 : pyromanfo Splitting it into two queries would make sense I think. Basically you could have it entirely in node_access_join_sql and node_access_where_sql. There we can check for the nid=0 case for any gids and realms. Then if it returns true, we know we don't have to check anything else, so we return null for the SQL to enter. We could even store this in a static variable to avoid calling this twice for join_sql and where_sql So in the nid=0 case, this should be similarly fast as we're removing a table from the JOIN in exchange for checking node_access for one nid. mysql> select count(nid) from node_access where nid=0 and gid=1 and realm in ('all','term_access','blah'); +------------+ | count(nid) | +------------+ | 0 | +------------+ 1 row in set (0.37 sec) SELECT COUNT(DISTINCT(n.nid)) FROM node n INNER JOIN term_node tn0 ON n.nid = tn0.nid WHERE n.status = 1 AND tn0.tid IN (9); +------------------------+ | COUNT(DISTINCT(n.nid)) | +------------------------+ | 4202 | +------------------------+ 1 row in set (0.15 sec) SELECT COUNT(DISTINCT(n.nid)) FROM node n INNER JOIN node_access na ON (na.nid = 0 OR na.nid = n.nid) INNER JOIN term_node tn0 ON n.nid = tn0.nid WHERE n.status = 1 AND na.grant_view = 1 AND CONCAT(na.realm, na.gid) IN ('all0','blah1') AND tn0.tid IN (9); +------------------------+ | COUNT(DISTINCT(n.nid)) | +------------------------+ | 4202 | +------------------------+ 1 row in set (1.97 sec) It looks to be faster, actually. Then, when we don't get a result, we check node access with the 'na.nid=0 OR' removed. This is certain to be faster as the test have shown, taking the query down from 7 minutes to a matter of seconds. In either case, this method would be faster. Plus, it wouldn't require any changes to other modules, simply change the node_access_where_sql and node_access_join_sql functions. JonBob, would you be okay with that? I can code it if you think you'll accept the patch. ------------------------------------------------------------------------ February 10, 2005 - 14:38 : Anonymous Splitting the queries into two is probably the only way to go. However, I'd like to avoid calling the nid = 0 query if possible. My Drupal installations that use node access do not use nid = 0 at all. I think we could store the available (nid = 0) => realm combinations somewhere and only execute the query if a matching condition is found. Gerhard ------------------------------------------------------------------------ February 10, 2005 - 16:14 : moshe weitzman that nid=0 is going to be very fast when done all by itself. no need to introduce any other mechanism IMO. testing will tell us for sure though ------------------------------------------------------------------------ February 10, 2005 - 19:31 : killes(a)www.drop.org The nid = 0 query would be executed for every node query on a page call even if your access module doesn't use nid = 0 at all. Most implementations do not use nid = 0. If we can avoid a query we should do so. ------------------------------------------------------------------------ February 11, 2005 - 21:00 : JonBob Attachment: http://drupal.org/files/issues/node_access.patch (2.91 KB) Here's a patch that splits the nid=0 check off into a separate query. If the user has such a grant, things should be really fast since we don't need any join or even DISTINCT() check. If not, the join is done as before, but without the "OR nid=0" piece. This ideally needs performance checking on systems using node access with nid=0 rows, using it without them, and not using it at all. ------------------------------------------------------------------------ February 14, 2005 - 13:33 : killes(a)www.drop.org Ok, here is a first test result: After installing the node_privacy_by_role module for the drupal.org database, the Drupal install became unusable for all but the No. 1 user and the load went high due to many hanging mysql processes. After applying the patch the site does work again. The module seems to generate one entry per node and role in the node_access table. I did not make any benchmarks, the result is so obvious. -- View: http://drupal.org/node/16558 Edit: http://drupal.org/project/comments/add/16558

1 0