- Development - lists.drupal.org

[drupal-devel] restricting access to file downloads with flexinode
by David Cohen 05 Aug '05

05 Aug '05

Hey all, I'm using Drupal on a project where some content is available to all, but some content restricted to paid subscribers. To make uploading content easy for the site administrators, I've created a custom node type with the wonderful flexinode module. This allows the admins to upload several files (which belong together) in one easy step. This works great, but I wouldn't be writing if it didn't get more complicated, so please read on. My node type corresponds to a 'lesson'. Each lesson has some public information which everyone can see, such as a title, and a preview image. Other content is restricted to only paid users, this includes flash files and quicktime videos. I'm using templates and php code to ensure users see only the portions they're supposed to see. My problem is the uploaded files. By default, anyone can download the files uploaded via flexinode. I never show an anonymous user the link to a flash file, but if they type "system/files?file=private-file.swf" in the URL, they'll be able to download it anyway. My second problem is that lots of files will be uploaded, and eventually the files directory will be a real mess. So I want to control where each of these files goes. For instance, the previews could all go to 'files/preview_images' and the movies to 'files/videos'. I've address both of these problems in the attached patch to flexinode. I'd appreciate it if some experienced drupal coders would check my work. I'd like to see something like this checked into the code base. I hope this would be useful to others as well. I'm happy to keep working on it if that's what it takes. If you review the code, search for '???' and see if you can answer the questions I left in the comments. Also - yes, I'm aware much of flexinode's functionality will be replaced by CCK, but I need this functionality today. And, I'd hoped to isolate my changes by adding another field type to flexinode. But to make it work I had to change flexinode's file download hook. And since I had to do that, I figured I might as well make the changes to the existing field_file.inc. As far as I know, the changes do not affect behavior for existing flexinode files. Here's how it works.... When adding a file field to a flexinode, you now have the option of specifiying a permission with that file. User's can download only if they have that permission. Note that different files within a node can have different permissions. This allows me to keep the preview images public but restrict access to the other files. Also, you have the option of specifying a subdirectory of files/ where the file will be saved. I plan to make this more dynamic, so that the directory can be a function of the other fields in the node. For instance, instead of just typeing 'private-files', you could type 'private-files/%flexinode_2' and %flexinode_2 would be substituted with a value from the node. This might get a little tricky but it will really help me organize the files. Finally, while I have your attention (if you're still reading) a couple questions about drupal's file handling... How much overhead is involved in using drupal to restrict access to the files? That is, can I serve a lot of large files via the file_transfer method? Is there some better way through Apache? Also, the file.inc seems pretty heavy handed about forcing all files to be in a subdirectory of files/ (the 'file_directory_path' variable). It would be nice to have some public files in files/ but also be able to save some uploads elsewhere (say /var/private-files). file_save_upload currently makes this impossible. If it were possible, some files could be served up efficiently via urls like 'mysite.com/files/foo.swf' while others could be restricted 'mysite.com/system/file?file=/var/private-files/bar.swf'. I'd like to see something like that. This concludes my first post to drupal-devel. I hope I haven't offended anyone ;) Drupal is a really great product. Cheers, -Dave P.S. Also attaching a patch to file.inc. The call to drupal_access_denied when downloading a file should return immediately after that call.

3 3

[drupal-devel] [bug] if postgresql function
by plj 05 Aug '05

05 Aug '05

Issue status update for http://drupal.org/node/23499 Post a follow up: http://drupal.org/project/comments/add/23499 Project: Drupal -Version: 4.6.0 +Version: cvs Component: forum.module Category: bug reports Priority: normal Assigned to: patrickslee Reported by: cchamilt Updated by: plj Status: patch (code needs review) I'm advocating it too. Just tested the fix in comment #13 against HEAD. Works well. I suggest that Dries would just commit it; at least the situation improves dramatically, because as now, the failure is certain. Perhaps there is something wrong in igric's setup, if the patch does not work? What is your Postgres version? plj Previous comments: ------------------------------------------------------------------------ Tue, 24 May 2005 04:32:18 +0000 : cchamilt Postgresql function "if" is defined to return a boolean value, yet forum requests an integer - last_comment_uid. user error: query: SELECT n.nid, l.last_comment_timestamp, IF(l.last_comment_uid, cu.name, l.last_comment_name) as last_comment_name, l.last_comment_uid FROM node n, node_comment_statistics l /*! USE INDEX (node_comment_timestamp) */, users cu, term_node r WHERE n.nid = r.nid AND r.tid = 1 AND n.status = 1 AND n.type = 'forum' AND l.last_comment_uid = cu.uid AND n.nid = l.nid ORDER BY l.last_comment_timestamp DESC LIMIT 1 OFFSET 0 in /var/www/hosts/www.ambigc.com/htdocs/includes/database.pgsql.inc on line 62. warning: pg_query() [function.pg-query]: Query failed: ERROR: function if(integer, character varying, character varying) does not exist HINT: No function matches the given name and argument types. You may need to add explicit type casts. in /var/www/hosts/www.ambigc.com/htdocs/includes/database.pgsql.inc on line 45. ------------------------------------------------------------------------ Tue, 24 May 2005 22:07:49 +0000 : patrickslee Attachment: http://drupal.org/files/issues/forum.module_3.patch (2.69 KB) I have also encountered this bug and made a possible patch by replacing IF statements to CASE. A patch file is attached. ------------------------------------------------------------------------ Tue, 24 May 2005 22:28:20 +0000 : adrian There is an IF function defined in the database.pgsql function. On what page is this happening, as what might be happening is that the type isn't being picked up correctly. It's defined as IF(anyelement, anyelement, text) returns anyelement. IIRC. What I have seen happen is that it wants the text field to return integer, or vice versa. ------------------------------------------------------------------------ Tue, 24 May 2005 22:32:56 +0000 : patrickslee I found the definition in database.pgsql. It is like this: CREATE FUNCTION "if"(boolean, anyelement, anyelement) RETURNS anyelement AS ' SELECT CASE WHEN $1 THEN $2 ELSE $3 END; ' LANGUAGE 'sql'; So it is actually wanting boolean. ------------------------------------------------------------------------ Tue, 24 May 2005 22:43:22 +0000 : patrickslee So my patch will work, and we can make it easier by replacing the IF function to: IF(l.last_comment_uid 0 AND l.last_comment_uid NULL, cu.name, l.last_comment_name) BTW there is another place in forum.module use IF like this. ------------------------------------------------------------------------ Tue, 24 May 2005 22:45:41 +0000 : patrickslee Oops... my <>s are stolen. IF(l.last_comment_uid <> 0 AND l.last_comment_uid <> NULL, cu.name, l.last_comment_name) ------------------------------------------------------------------------ Wed, 25 May 2005 05:20:34 +0000 : cchamilt Yes that line works great. Thanks for your help! ------------------------------------------------------------------------ Wed, 25 May 2005 05:52:24 +0000 : patrickslee This patch shows another problem. The last post user will always show "Anonymous" or whatever you defined. Looking for solution... ------------------------------------------------------------------------ Wed, 25 May 2005 06:00:55 +0000 : patrickslee Use this line instead: IF(l.last_comment_uid <> 0 AND l.last_comment_uid <> NULL, cu.name, l.last_comment_name) Don't forget there are two places in forum.module need to be changed. ------------------------------------------------------------------------ Wed, 25 May 2005 06:10:26 +0000 : patrickslee Oh what have I done! I changed nothing! IF(l.last_comment_uid <> 0 AND l.last_comment_uid IS NOT NULL, cu.name, l.last_comment_name) ------------------------------------------------------------------------ Wed, 25 May 2005 10:12:14 +0000 : adrian perhaps we could also just add more wrapper functions for the specific types ? create or replace function IF (integer, anyelement, anyelement) returns anyelement as ' SELECT if (($1 0) AND NOT nullvalue($1), $2, $3); ' LANGUAGE 'sql'; Or something. ------------------------------------------------------------------------ Wed, 25 May 2005 10:41:18 +0000 : plj '' was missing. So what adrian actually means is this: CREATE OR REPLACE FUNCTION IF (integer, anyelement, anyelement) RETURNS anyelement AS ' SELECT if (($1 <> 0) AND NOT nullvalue($1), $2, $3); ' LANGUAGE 'sql'; I applied that and now forums work -- no module patches. PostgreSQL 7.4.7. ------------------------------------------------------------------------ Wed, 25 May 2005 10:45:07 +0000 : plj Sheesh. I meant that '<>' was missing. Gotta learn to use that preview button... ------------------------------------------------------------------------ Wed, 25 May 2005 21:53:11 +0000 : patrickslee Is nullvalue() a function? I cannot find it in neither postgres manual nor mysql manual. Use "IS NOT NULL" would be better as it is more standard-compatible. SO we have this: CREATE OR REPLACE FUNCTION IF (integer, anyelement, anyelement) RETURNS anyelement AS ' SELECT IF(($1 <> 0) AND $1 IS NOT NULL, $2, $3); ' LANGUAGE 'sql'; Looks we have a satisfying fix here. Is this fix going to be committed in CVS for next release? ------------------------------------------------------------------------ Thu, 26 May 2005 00:26:58 +0000 : adrian hostmaster=# \df nullvalue List of functions Result data type | Schema | Name | Argument data types ------------------+------------+-----------+--------------------- boolean | pg_catalog | nullvalue | "any" But it doesn't matter to me =) ------------------------------------------------------------------------ Thu, 26 May 2005 19:21:57 +0000 : Dries Can you guys provide a patch? I don't have a PostgreSQL setup to test it against. Will commit to DRUPAL-4-6 and HEAD. ------------------------------------------------------------------------ Thu, 26 May 2005 20:29:38 +0000 : patrickslee Attachment: http://drupal.org/files/issues/database.pgsql_3.patch (363 bytes) Here you have it. ------------------------------------------------------------------------ Wed, 15 Jun 2005 10:18:32 +0000 : igrcic Dont know about you guys but that patch doesnt work for me in drupal 4.6.1!? Im using PostgreSQL. Instead im using following function that works just fine if(integer,text,text) BEGIN IF $1 THEN RETURN $2; END IF; IF NOT $1 THEN RETURN $3; END IF; END; ------------------------------------------------------------------------ Sun, 19 Jun 2005 08:55:54 +0000 : Dries Just so you know: I'm not committing this patch yet because one user commented that it didn't work. Waiting for a follow-up from the PostgreSQL front. ------------------------------------------------------------------------ Sun, 19 Jun 2005 11:28:04 +0000 : igrcic Ok, here how it is now. First of all, forget my previous if FUNCTION :) Currenty im using drupal 4.6.1 and I had that problems with forums. Now Im using two if functions, because one is required for forums and I forgot for another another one, but I know that it was called because of different function parameters (bool, int). So one for the forum is : CREATE OR REPLACE FUNCTION "if"(integer, text, text) RETURNS text AS ' BEGIN IF $1 IS NOT NULL AND $1 > 0 THEN RETURN $2; ELSE RETURN $3; END IF; END; ' LANGUAGE 'plpgsql'; The another one is standard function that comes with drupal database.inc.pgsql file CREATE FUNCTION "if"(boolean, anyelement, anyelement) RETURNS anyelement AS ' SELECT CASE WHEN $1 THEN $2 ELSE $3 END; ' LANGUAGE 'sql'; Im using it for a while and everything works nice! Before I posted this topic I replaced function with that one in patch and it worked nice with this one too. But I know that once I had some error with it (didn't write it down grrr), so im stickin with the first function, because it work perfectly for me! And we have to have 2 functions, one for boolean and one for integer! In mysql it isnt so. Hope everything's clear now! ------------------------------------------------------------------------ Sun, 19 Jun 2005 11:30:15 +0000 : igrcic Oops, maybe I screwed up something with formatting my post :( Hope you can fix it, sorry ------------------------------------------------------------------------ Thu, 23 Jun 2005 03:43:59 +0000 : Arto Drupal 4.6.1 on PostgreSQL 7.4.7: The patch (database.pgsql_3.patch) fixes the problem; igrcic's method would certainly fix it as well. Advocate committing the patch.

1 0

[drupal-devel] [bug] Bootstrap totally broken with PostgreSQL
by plj 05 Aug '05

05 Aug '05

Issue status update for http://drupal.org/node/28274 Post a follow up: http://drupal.org/project/comments/add/28274 Project: Drupal Version: cvs Component: base system Category: bug reports Priority: critical Assigned to: Anonymous Reported by: plj Updated by: plj Status: patch (code needs work) Attachment: http://drupal.org/files/issues/bootstrap.inc_4.patch (986 bytes) Just cheked out today's HEAD, installed, and attempted to access main page. Results: [client 10.10.59.23] PHP Notice: Undefined variable: _SESSION in /var/www/drupal_HEAD/includes/bootstrap.inc on line 771 [client 10.10.59.23] PHP Notice: Undefined variable: _SESSION in /var/www/drupal_HEAD/includes/bootstrap.inc on line 771 [client 10.10.59.23] PHP Notice: Undefined variable: _SESSION in /var/www/drupal_HEAD/includes/bootstrap.inc on line 771 [client 10.10.59.23] PHP Warning: pg_query(): Query failed: ERROR: syntax error at or near "cache" at character 13 in /var/www/drupal_HEAD/includes/database.pgsql.inc on line 69 [client 10.10.59.23] PHP Fatal error: ERROR: syntax error at or near "cache" at character 13\nquery: LOCK TABLES cache WRITE in /var/www/drupal_HEAD/includes/database.pgsql.inc on line 86 [client 10.10.59.23] PHP Notice: Undefined property: cache in /var/www/drupal_HEAD/includes/session.inc on line 45 [client 10.10.59.23] PHP Warning: pg_query(): Query failed: ERROR: syntax error at or near "cache" at character 13 in /var/www/drupal_HEAD/includes/database.pgsql.inc on line 69 [client 10.10.59.23] PHP Fatal error: ERROR: syntax error at or near "cache" at character 13\nquery: LOCK TABLES cache WRITE in /var/www/drupal_HEAD/includes/database.pgsql.inc on line 86 [client 10.10.59.23] PHP Warning: pg_query(): Query failed: ERROR: syntax error at or near "cache" at character 13 in /var/www/drupal_HEAD/includes/database.pgsql.inc on line 69 [client 10.10.59.23] PHP Fatal error: ERROR: syntax error at or near "cache" at character 13\nquery: LOCK TABLES cache WRITE in /var/www/drupal_HEAD/includes/database.pgsql.inc on line 86 This is because bootstap.inc uses LOCK/UNLOCK TABLES, but PostgreSQL does not support this syntax. I've attached a patch that fixes the problem on pgsql, but this probably still needs work as it most likely won't work on MySQL as such. plj

1 2

[drupal-devel] [bug] apache and drupal both compressing pages
by fago 05 Aug '05

05 Aug '05

Issue status update for http://drupal.org/node/13224 Post a follow up: http://drupal.org/project/comments/add/13224 Project: Drupal Version: cvs Component: base system Category: bug reports Priority: critical Assigned to: killes(a)www.drop.org Reported by: bertboerland(a)www.drop.org Updated by: fago Status: patch (code needs review) i don't have this issue any more. since i'm using a new server running apache2 with deflate instead of apache1 with mod-gzip everything is fine. unfortunately, i've no access to the old server (for testing) any more. fago Previous comments: ------------------------------------------------------------------------ Sat, 20 Nov 2004 16:22:55 +0000 : bertboerland(a)www.drop.org hi this weekend I went ahead and upgraded drupal from 4.4 to 4.5 on a machine, I dint have time/priority (in Dutch these words are almost the same :-) sooner. The core went fine and as usual lots of unpacking, configuring for all the modules I want to experiment with. After some time I tried to access the site in a -just-to-be-sure- moment from windows machine with an IE browser without being logged in. I found some strange behavior. Sometimes IE wanted to download the file, sometimes it showed the file (being a binary) and sometimes I got a valid page. I never got this from my firefox on both windows and Linux while being logged in. Later I could reproduce this on a firefox browser without being logged in. After some time / thinking I think I found out that both apache (my webserver) and drupal were compressing (gzipping) the page resulting in a browser that was able to uncompress the page once. But if it was a cached page and drupal was zipping this and Apache as well, the browser was just displaying a gzipped page. I have a rather old setup with an ancient version of apache but does this sound possible? Since I am hosting (some) other content outside the drupal dir, I would like to keep apache gzipping pages so is there an option to stop drupal from compressing pages? ------------------------------------------------------------------------ Sat, 20 Nov 2004 16:32:16 +0000 : killes(a)www.drop.org Which method use your Apache to gzip the pages? Which version is it? I find it strange that we get this report only several weeks after the release. I suspect that your particualr set up is borken. ------------------------------------------------------------------------ Sat, 20 Nov 2004 16:45:23 +0000 : bertboerland(a)www.drop.org The reason I waited so long was that I wanted others to find out this :-) Mind you, I *think* it is the fact that they are both compressing the cached pages. You might want to try it out for yourself by going to http://willy.boerland.com/myblog and in case a there is a cached page you will end up by downloading a binary that once renamed to a gz and unpacked, will give you the html file. drupal 4.5 apache: 1.3.26 (with *all* security patches) php: 4.2.3 Content-Encoding: gzip And yes, since I quess 100+ sites are already at 4.5 I think it is related to my setup. However, I understand correctly there i no way of having drupal stop sending out compressed pages? There is no configuration option? ------------------------------------------------------------------------ Sat, 20 Nov 2004 17:06:05 +0000 : killes(a)www.drop.org Yes, I get funny garbage, too. The apache version you run is the one Debian provides. It is still used very often, but still nobody but you seems to experience the problem, so I still think that somehow your setup is broken. Apache should recognize that the page is already gzipped. Do you use mod_gzip or libz? ------------------------------------------------------------------------ Sat, 20 Nov 2004 17:57:14 +0000 : bertboerland(a)www.drop.org Since this one is more or less critical for me but not for the RotW, re-setting priority to "nomal". I am using zlib btw. Did you try to save the file and unzip it? And I am correct that there is no option to disable cached paged from being zipped? ------------------------------------------------------------------------ Sat, 20 Nov 2004 18:30:23 +0000 : bertboerland(a)www.drop.org I think I "solved" this for me by changing the bootstrap.inc setting this to postponed, if no-one else experiences this problem within 3 weeks I will close this issue. ------------------------------------------------------------------------ Sat, 20 Nov 2004 18:31:19 +0000 : killes(a)www.drop.org I tried to download it now, but you must have changed something, the page renders fine. You are right, there is no option to disable gzipped cache. I'd really like to discover the reason for this problem. Normally, zlib should check for gzipped content and do not gzip it again. Drupal sends the appropriate header. ------------------------------------------------------------------------ Sat, 20 Nov 2004 18:32:25 +0000 : killes(a)www.drop.org Locking could have prevented this. ;) ------------------------------------------------------------------------ Sun, 21 Nov 2004 12:36:23 +0000 : axel(a)drupal.ru > I found some strange behavior. Sometimes IE wanted to download the file, sometimes it showed the file (being a binary) and sometimes I got a valid page. Sometimes I got same thing in Mozilla Firefox with differrent sites (not sure, but not only with Drupal) when I work trough my local proxies (I use chain of two proxies on my Debian box - one for banner deleting (privoxy) and second for offline page caching (wwwoffle)). I don't dig to details of such behaviour, but simply switch off proxies and reload browser's page help to solve problem - then I again switch on proxies and page loaded ok. May be you also access site through proxy? Then you need to check this proxy settings, I think. ------------------------------------------------------------------------ Sun, 21 Nov 2004 21:13:26 +0000 : bertboerland(a)www.drop.org Axel, no it was not related to proxys or even the browser. It was related to cached pages being zipped by drupal and sent to apache that was ziping the page as well, resulting in a zipped zipped page for the client that only once unzipped the page and hence displayed a binary ------------------------------------------------------------------------ Tue, 21 Dec 2004 17:10:12 +0000 : bertboerland(a)www.drop.org It seems like I was not the only one [1]. There is not enough information about the others setup (/please post here!/) to find a generic cause, but it is sure that some setups will cause zipped pages to be zipped. [1] http://drupal.org/node/14569 ------------------------------------------------------------------------ Thu, 03 Feb 2005 15:03:17 +0000 : leafish_dylan Has anybody found a better way to fix this problem? Disabling the storage of compressed cache pages in bootstrap.inc works, but it's ugly. Is an uncompressed cache likely to take up a lot of space in the database? If not, perhaps a global "gzip/zlib compression" option for Drupal would be simpler. ------------------------------------------------------------------------ Sat, 05 Feb 2005 14:40:16 +0000 : Anonymous As author of the gzipped cache patch I'd really appreciate if the people who experience this problem could supply detailed(!) info on their setup and the headers sent both by the browser and the server. I'd like to fix the problem (if there is one on Drupal's side) or at least document correct server settings if the problem is there. ------------------------------------------------------------------------ Sat, 05 Feb 2005 16:54:42 +0000 : bertboerland(a)www.drop.org Here some info, please ask if you need more: apache 1.3.26 mmcache (version unknown) drupal 4.5.1 Note: I didnt post (and thought abaout!) the MMcache between drupal and apache. The configuration of mmcache is extension="mmcache.so" mmcache.shm_size="16" mmcache.cache_dir="/tmp/mmcache" mmcache.enable="1" mmcache.optimizer="1" mmcache.check_mtime="1" mmcache.debug="0" mmcache.filter="" mmcache.shm_max="0" mmcache.shm_ttl="0" mmcache.shm_prune_period="0" mmcache.shm_only="0" mmcache.compress="1" headers: http://willy.boerland.com/myblog/ GET /myblog/ HTTP/1.1 Host: willy.boerland.com User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.5) Gecko/20041107 Firefox/1.0 Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5 Accept-Language: en-us,en;q=0.5 Accept-Encoding: gzip,deflate Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Keep-Alive: 300 Connection: keep-alive Cookie: PHPSESSID=307f07c169e3bbcbe92d150026c97583 HTTP/1.x 200 OK Date: Sat, 05 Feb 2005 16:45:49 GMT Server: Apache-AdvancedExtranetServer/1.3.26 X-Powered-By: PHP/4.2.3 Content-Encoding: gzip Keep-Alive: timeout=15, max=100 Connection: Keep-Alive Transfer-Encoding: chunked Content-Type: text/html; charset=utf-8 ---------------------------------------------------------- ------------------------------------------------------------------------ Mon, 21 Feb 2005 10:53:02 +0000 : coma I have exactly the same problem my setup it's currently apache 2.0.52, php 4.3.10 and drupal 4.5.2. The relevant info it's: from apache: <Location /> # Insert filter SetOutputFilter DEFLATE # Netscape 4.x has some problems... BrowserMatch ^Mozilla/4 gzip-only-text/html # Netscape 4.06-4.08 have some more problems BrowserMatch ^Mozilla/4\.0[678] no-gzip # MSIE masquerades as Netscape, but it is fine BrowserMatch \bMSIE !no-gzip !gzip-only-text/html # Don't compress images SetEnvIfNoCase Request_URI \ \.(?:gif|jpe?g|png)$ no-gzip dont-vary # Make sure proxies don't deliver the wrong content Header append Vary User-Agent env=!dont-vary </Location> In php.ini: zlib.output_compression = On And I can assure you that my problem was that last element, changing zlib.output_compression to off solved the problem, and the output of all php scripts (not only drupal) it's compressed by either drupal or apache. You can test this things with wget: wget --header "Accept-Encoding: gzip,deflate" http://localhost/ The file wget drops should be a gzippped html, if you do a zcat and get a plain html it's all good, if you get another gzipped file it has been double compressed. ------------------------------------------------------------------------ Sat, 02 Apr 2005 19:38:42 +0000 : moshe weitzman i too am seeing strange behavior here. specifically, php is simply dieing after printing out the $cache->data in drupal_page_header(). truly, i don't think drupal should be storing gzipped cache, nor do i think it should perform gzip at all. these are better handled at php or apache layers. ------------------------------------------------------------------------ Sat, 23 Apr 2005 14:06:14 +0000 : mjr Apparently i am observing a new variant of that problem in one of the subsites of my 4.6 test installation: after enebaling cacheing, lynx, w3m and the gecko based browsers display garbage. lynx tells me it wants to load a file named index.html.gz. wget http://beate/drupal46/ on that site gives a readable html page. w3m -dump_head tells me: Received cookie: PHPSESSID=a6916d57d08727ab8ed945ddac4ecf2c HTTP/1.1 200 OK Date: Sat, 23 Apr 2005 13:56:19 GMT Server: Apache/2.0.53 (Debian GNU/Linux) PHP/4.3.10-10 mod_ssl/2.0.53 OpenSSL/0.9.7e mod_perl/1.999.20 Perl/v5.8.4 X-Powered-By: PHP/4.3.10-10 Set-Cookie: PHPSESSID=a6916d57d08727ab8ed945ddac4ecf2c; expires=Mon, 16 May 2005 17:29:39 GMT; path=/ Expires: Sun, 19 Nov 1978 05:00:00 GMT Last-Modified: Sat, 23 Apr 2005 13:56:19 GMT Cache-Control: no-store, no-cache, must-revalidate Cache-Control: post-check=0, pre-check=0 Pragma: no-cache Content-Encoding: gzip Vary: Accept-Encoding Connection: close Content-Type: text/html; charset=utf-8 Apparently, the site sends uncompressed pages but tells the browser it is compressed. BTW, links2 and dillo do disply the page but will not allow me to log in and disable cacheing. Any clues? Thanks Michael ------------------------------------------------------------------------ Sat, 23 Apr 2005 14:08:42 +0000 : mjr Perhaps I should mention that zlib.output_compression = Off in my php.ini ------------------------------------------------------------------------ Sat, 23 Apr 2005 18:43:30 +0000 : bertboerland(a)www.drop.org /Apparently i am observing a new variant of that problem / Please open a new bug report, since your problem is likely unrelated to this problem, open an new problem and you might want to use a link to this problem, but dont use this problem for other than descibed. ------------------------------------------------------------------------ Sat, 04 Jun 2005 23:53:22 +0000 : mjr no, it is not unrelated. After some investigation, i found out that the garbage is being stored in drupals cache. This has been reported in this thread. My present workaround is to completly disable drupals caching (and to manually delete the compressed pages from the database.) Whats the use of letting drupal compress pages and store these int its cache table? Just saving a few cycles of CPU time? On the cost of letting the web server and its client decide wether to transfer compressed or uncompressed data - they have the protocol to do that, drupal has not. Michael ------------------------------------------------------------------------ Mon, 06 Jun 2005 16:48:30 +0000 : killes(a)www.drop.org We save cpu cycles and storage space. Apache only needs to check the Encoding header which will be easy. I do not know why zlib output compression doesn't check those headers for some (or all?) people. ------------------------------------------------------------------------ Fri, 10 Jun 2005 23:32:40 +0000 : bluec I have the same problem with Drupal 4.6.1: If the site delivers a cached page it is gziped twice and the result is garbage in my browser window. I tried two differnet browsers: Firefox and Opera and it's the same problem with both of them. I recorded the HTTP headers: - Cache turned on - zlib-compression turned on - Result: Garbage http://kim.sshtun.v63.org:22003/index.html GET /index.html HTTP/1.1 Host: kim.sshtun.v63.org:22003 User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.8) Gecko/20050511 Firefox/1.0.4 Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5 Accept-Language: de-de,en;q=0.5 Accept-Encoding: gzip,deflate Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Keep-Alive: 300 Connection: keep-alive Referer: http://kim.sshtun.v63.org:22003/fachschaft/ Cookie: PHPSESSID=6ddbf3d5a32d-CHANGED-BY-CHRIS HTTP/1.x 200 OK Date: Fri, 10 Jun 2005 23:16:08 GMT Server: Apache/2.0.54 (Gentoo/Linux) mod_ssl/2.0.54 OpenSSL/0.9.7e PHP/5.0.4 X-Powered-By: PHP/5.0.4 Expires: Mon, 26 Jul 1997 05:00:00 GMT Last-Modified: Fri, 10 Jun 2005 23:11:59 GMT Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0 Pragma: no-cache Set-Cookie: PHPSESSID=6ddbf3d5a32d20-CHANGED-BY-CHRIS; expires=Mon, 04 Jul 2005 02:49:28 GMT; path=/ Etag: "03cff5fcc105e72dfc2671d3ffdb285c" Content-Encoding: gzip Content-Length: 5024 Keep-Alive: timeout=15, max=95 Connection: Keep-Alive Content-Type: text/html; charset=utf-8 Now I turned off compression by setting in my .htaccess php_flag zlib.output_compression On and the result is OK: http://kim.sshtun.v63.org:22003/index.html GET /index.html HTTP/1.1 Host: kim.sshtun.v63.org:22003 User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.8) Gecko/20050511 Firefox/1.0.4 Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5 Accept-Language: de-de,en;q=0.5 Accept-Encoding: gzip,deflate Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Keep-Alive: 300 Connection: keep-alive Referer: http://kim.sshtun.v63.org:22003/index.html Cookie: PHPSESSID=6ddbf3d5a32d20-CHANGED-BY-CHRIS HTTP/1.x 200 OK Date: Fri, 10 Jun 2005 23:17:08 GMT Server: Apache/2.0.54 (Gentoo/Linux) mod_ssl/2.0.54 OpenSSL/0.9.7e PHP/5.0.4 X-Powered-By: PHP/5.0.4 Expires: Mon, 26 Jul 1997 05:00:00 GMT Last-Modified: Fri, 10 Jun 2005 23:11:59 GMT Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0 Pragma: no-cache Set-Cookie: PHPSESSID=6ddbf3d5a32d208-CHANGED-BY-CHRIS; expires=Mon, 04 Jul 2005 02:50:28 GMT; path=/ Etag: "03cff5fcc105e72dfc2671d3ffdb285c" Content-Encoding: gzip Vary: Accept-Encoding Content-Length: 5054 Keep-Alive: timeout=15, max=100 Connection: Keep-Alive Content-Type: text/html; charset=utf-8 Turning off the cache is a solution as well. As soon as cache and zlib are operating the result is garbage. If you need any other information just let me know. I can turn on compression at any time and test other solutions. Chris ------------------------------------------------------------------------ Mon, 27 Jun 2005 18:10:12 +0000 : aries The same. Interesting, only one page doing this, all others are not. -- Aries http://aries.mindworks.hu ------------------------------------------------------------------------ Thu, 07 Jul 2005 14:46:34 +0000 : fago quite strange. i never had problems with this, then i upgraded from 4.6.1 to drupal 4.6.2... now i must turn off the cache, or i've the troubles mentioned here... (i haven't touched the apache or php config in the meanwhile) ------------------------------------------------------------------------ Thu, 14 Jul 2005 09:47:32 +0000 : bertboerland(a)www.drop.org yet another one [2], closed than one, raised importance here. [2] http://drupal.org/node/25326 ------------------------------------------------------------------------ Sat, 16 Jul 2005 19:52:26 +0000 : bertboerland(a)www.drop.org killes sigested that is want apache that compressed the page but drupal and php was the first one. after some testing, i found out he was correct! to solve this 1) either comment out the gzip lines in bootstrap.inc 2) put /etc/php.ini "zlib.output_compression = On" to OFF (real) Fix needed. Drupal should look if the page was zipped in the first place.... ------------------------------------------------------------------------ Fri, 29 Jul 2005 12:17:46 +0000 : killes(a)www.drop.org Attachment: http://drupal.org/files/issues/gzip.patch (2.45 KB) Here's a patch that needs some testing. ------------------------------------------------------------------------ Fri, 29 Jul 2005 13:18:51 +0000 : Dries Why do we set $do_cache to TRUE in the nested if's? $do_cache is already set to TRUE higher up so we're just overwriting TRUE with TRUE? Also, can we rename $do_cache to $cache? That is more Drupal-ish. Does that solve all apache-Drupal-caching problems or only some? I vaguely remember we agreed to add a setting, because that was perceived the only True Solution. ------------------------------------------------------------------------ Fri, 29 Jul 2005 18:22:04 +0000 : killes(a)www.drop.org Attachment: http://drupal.org/files/issues/gzip_0.patch (2.46 KB) I've updated the patch. The patch is intended to cure all the problems we had. It needs testing, though. I don't recall to agreeing that a setting would be needed. Just for the record: Steven measured the ratio of serving gzipped cached pages to re-unzipped pages on drupal.org to be about 5.1:1. That is from 6 page requests which we serve from the cache only one needs the cache to be unzipped. This is probably due to crawlers who only speak http 1.0. ------------------------------------------------------------------------ Sat, 30 Jul 2005 14:33:42 +0000 : bertboerland(a)www.drop.org people who expierenced this problem, please try the patch of killes and report here if successful or not. i will patch as well ------------------------------------------------------------------------ Mon, 01 Aug 2005 14:55:47 +0000 : moshe weitzman In my opinion, this is a case of misplaced optimization. Very little CPU is required to gzip a document. Why else would Apache and PHP provide an option to do this *for every page*. PHP knows that all its pages will be dynamic yet it still offers this option. My recommendation is this feature entirely, and take the win in simplicity/code reduction. Furthermore, we won't have any more issues like this one, which has lingered unfixed for 9 months. Admins who want gzip will elect to do so in their php.ini or .htaccess. ------------------------------------------------------------------------ Fri, 05 Aug 2005 13:46:03 +0000 : Arto If the caching mechanism is to be patched, that'd be a good time to also tackle the problem that it's currently unusable on PostgreSQL: http://drupal.org/node/26369

1 0

[drupal-devel] [bug] apache and drupal both compressing pages
by Arto 05 Aug '05

05 Aug '05

Issue status update for http://drupal.org/node/13224 Post a follow up: http://drupal.org/project/comments/add/13224 Project: Drupal Version: cvs Component: base system Category: bug reports Priority: critical Assigned to: killes(a)www.drop.org Reported by: bertboerland(a)www.drop.org Updated by: Arto Status: patch (code needs review) If the caching mechanism is to be patched, that'd be a good time to also tackle the problem that it's currently unusable on PostgreSQL: http://drupal.org/node/26369 Arto Previous comments: ------------------------------------------------------------------------ Sat, 20 Nov 2004 16:22:55 +0000 : bertboerland(a)www.drop.org hi this weekend I went ahead and upgraded drupal from 4.4 to 4.5 on a machine, I dint have time/priority (in Dutch these words are almost the same :-) sooner. The core went fine and as usual lots of unpacking, configuring for all the modules I want to experiment with. After some time I tried to access the site in a -just-to-be-sure- moment from windows machine with an IE browser without being logged in. I found some strange behavior. Sometimes IE wanted to download the file, sometimes it showed the file (being a binary) and sometimes I got a valid page. I never got this from my firefox on both windows and Linux while being logged in. Later I could reproduce this on a firefox browser without being logged in. After some time / thinking I think I found out that both apache (my webserver) and drupal were compressing (gzipping) the page resulting in a browser that was able to uncompress the page once. But if it was a cached page and drupal was zipping this and Apache as well, the browser was just displaying a gzipped page. I have a rather old setup with an ancient version of apache but does this sound possible? Since I am hosting (some) other content outside the drupal dir, I would like to keep apache gzipping pages so is there an option to stop drupal from compressing pages? ------------------------------------------------------------------------ Sat, 20 Nov 2004 16:32:16 +0000 : killes(a)www.drop.org Which method use your Apache to gzip the pages? Which version is it? I find it strange that we get this report only several weeks after the release. I suspect that your particualr set up is borken. ------------------------------------------------------------------------ Sat, 20 Nov 2004 16:45:23 +0000 : bertboerland(a)www.drop.org The reason I waited so long was that I wanted others to find out this :-) Mind you, I *think* it is the fact that they are both compressing the cached pages. You might want to try it out for yourself by going to http://willy.boerland.com/myblog and in case a there is a cached page you will end up by downloading a binary that once renamed to a gz and unpacked, will give you the html file. drupal 4.5 apache: 1.3.26 (with *all* security patches) php: 4.2.3 Content-Encoding: gzip And yes, since I quess 100+ sites are already at 4.5 I think it is related to my setup. However, I understand correctly there i no way of having drupal stop sending out compressed pages? There is no configuration option? ------------------------------------------------------------------------ Sat, 20 Nov 2004 17:06:05 +0000 : killes(a)www.drop.org Yes, I get funny garbage, too. The apache version you run is the one Debian provides. It is still used very often, but still nobody but you seems to experience the problem, so I still think that somehow your setup is broken. Apache should recognize that the page is already gzipped. Do you use mod_gzip or libz? ------------------------------------------------------------------------ Sat, 20 Nov 2004 17:57:14 +0000 : bertboerland(a)www.drop.org Since this one is more or less critical for me but not for the RotW, re-setting priority to "nomal". I am using zlib btw. Did you try to save the file and unzip it? And I am correct that there is no option to disable cached paged from being zipped? ------------------------------------------------------------------------ Sat, 20 Nov 2004 18:30:23 +0000 : bertboerland(a)www.drop.org I think I "solved" this for me by changing the bootstrap.inc setting this to postponed, if no-one else experiences this problem within 3 weeks I will close this issue. ------------------------------------------------------------------------ Sat, 20 Nov 2004 18:31:19 +0000 : killes(a)www.drop.org I tried to download it now, but you must have changed something, the page renders fine. You are right, there is no option to disable gzipped cache. I'd really like to discover the reason for this problem. Normally, zlib should check for gzipped content and do not gzip it again. Drupal sends the appropriate header. ------------------------------------------------------------------------ Sat, 20 Nov 2004 18:32:25 +0000 : killes(a)www.drop.org Locking could have prevented this. ;) ------------------------------------------------------------------------ Sun, 21 Nov 2004 12:36:23 +0000 : axel(a)drupal.ru > I found some strange behavior. Sometimes IE wanted to download the file, sometimes it showed the file (being a binary) and sometimes I got a valid page. Sometimes I got same thing in Mozilla Firefox with differrent sites (not sure, but not only with Drupal) when I work trough my local proxies (I use chain of two proxies on my Debian box - one for banner deleting (privoxy) and second for offline page caching (wwwoffle)). I don't dig to details of such behaviour, but simply switch off proxies and reload browser's page help to solve problem - then I again switch on proxies and page loaded ok. May be you also access site through proxy? Then you need to check this proxy settings, I think. ------------------------------------------------------------------------ Sun, 21 Nov 2004 21:13:26 +0000 : bertboerland(a)www.drop.org Axel, no it was not related to proxys or even the browser. It was related to cached pages being zipped by drupal and sent to apache that was ziping the page as well, resulting in a zipped zipped page for the client that only once unzipped the page and hence displayed a binary ------------------------------------------------------------------------ Tue, 21 Dec 2004 17:10:12 +0000 : bertboerland(a)www.drop.org It seems like I was not the only one [1]. There is not enough information about the others setup (/please post here!/) to find a generic cause, but it is sure that some setups will cause zipped pages to be zipped. [1] http://drupal.org/node/14569 ------------------------------------------------------------------------ Thu, 03 Feb 2005 15:03:17 +0000 : leafish_dylan Has anybody found a better way to fix this problem? Disabling the storage of compressed cache pages in bootstrap.inc works, but it's ugly. Is an uncompressed cache likely to take up a lot of space in the database? If not, perhaps a global "gzip/zlib compression" option for Drupal would be simpler. ------------------------------------------------------------------------ Sat, 05 Feb 2005 14:40:16 +0000 : Anonymous As author of the gzipped cache patch I'd really appreciate if the people who experience this problem could supply detailed(!) info on their setup and the headers sent both by the browser and the server. I'd like to fix the problem (if there is one on Drupal's side) or at least document correct server settings if the problem is there. ------------------------------------------------------------------------ Sat, 05 Feb 2005 16:54:42 +0000 : bertboerland(a)www.drop.org Here some info, please ask if you need more: apache 1.3.26 mmcache (version unknown) drupal 4.5.1 Note: I didnt post (and thought abaout!) the MMcache between drupal and apache. The configuration of mmcache is extension="mmcache.so" mmcache.shm_size="16" mmcache.cache_dir="/tmp/mmcache" mmcache.enable="1" mmcache.optimizer="1" mmcache.check_mtime="1" mmcache.debug="0" mmcache.filter="" mmcache.shm_max="0" mmcache.shm_ttl="0" mmcache.shm_prune_period="0" mmcache.shm_only="0" mmcache.compress="1" headers: http://willy.boerland.com/myblog/ GET /myblog/ HTTP/1.1 Host: willy.boerland.com User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.5) Gecko/20041107 Firefox/1.0 Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5 Accept-Language: en-us,en;q=0.5 Accept-Encoding: gzip,deflate Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Keep-Alive: 300 Connection: keep-alive Cookie: PHPSESSID=307f07c169e3bbcbe92d150026c97583 HTTP/1.x 200 OK Date: Sat, 05 Feb 2005 16:45:49 GMT Server: Apache-AdvancedExtranetServer/1.3.26 X-Powered-By: PHP/4.2.3 Content-Encoding: gzip Keep-Alive: timeout=15, max=100 Connection: Keep-Alive Transfer-Encoding: chunked Content-Type: text/html; charset=utf-8 ---------------------------------------------------------- ------------------------------------------------------------------------ Mon, 21 Feb 2005 10:53:02 +0000 : coma I have exactly the same problem my setup it's currently apache 2.0.52, php 4.3.10 and drupal 4.5.2. The relevant info it's: from apache: <Location /> # Insert filter SetOutputFilter DEFLATE # Netscape 4.x has some problems... BrowserMatch ^Mozilla/4 gzip-only-text/html # Netscape 4.06-4.08 have some more problems BrowserMatch ^Mozilla/4\.0[678] no-gzip # MSIE masquerades as Netscape, but it is fine BrowserMatch \bMSIE !no-gzip !gzip-only-text/html # Don't compress images SetEnvIfNoCase Request_URI \ \.(?:gif|jpe?g|png)$ no-gzip dont-vary # Make sure proxies don't deliver the wrong content Header append Vary User-Agent env=!dont-vary </Location> In php.ini: zlib.output_compression = On And I can assure you that my problem was that last element, changing zlib.output_compression to off solved the problem, and the output of all php scripts (not only drupal) it's compressed by either drupal or apache. You can test this things with wget: wget --header "Accept-Encoding: gzip,deflate" http://localhost/ The file wget drops should be a gzippped html, if you do a zcat and get a plain html it's all good, if you get another gzipped file it has been double compressed. ------------------------------------------------------------------------ Sat, 02 Apr 2005 19:38:42 +0000 : moshe weitzman i too am seeing strange behavior here. specifically, php is simply dieing after printing out the $cache->data in drupal_page_header(). truly, i don't think drupal should be storing gzipped cache, nor do i think it should perform gzip at all. these are better handled at php or apache layers. ------------------------------------------------------------------------ Sat, 23 Apr 2005 14:06:14 +0000 : mjr Apparently i am observing a new variant of that problem in one of the subsites of my 4.6 test installation: after enebaling cacheing, lynx, w3m and the gecko based browsers display garbage. lynx tells me it wants to load a file named index.html.gz. wget http://beate/drupal46/ on that site gives a readable html page. w3m -dump_head tells me: Received cookie: PHPSESSID=a6916d57d08727ab8ed945ddac4ecf2c HTTP/1.1 200 OK Date: Sat, 23 Apr 2005 13:56:19 GMT Server: Apache/2.0.53 (Debian GNU/Linux) PHP/4.3.10-10 mod_ssl/2.0.53 OpenSSL/0.9.7e mod_perl/1.999.20 Perl/v5.8.4 X-Powered-By: PHP/4.3.10-10 Set-Cookie: PHPSESSID=a6916d57d08727ab8ed945ddac4ecf2c; expires=Mon, 16 May 2005 17:29:39 GMT; path=/ Expires: Sun, 19 Nov 1978 05:00:00 GMT Last-Modified: Sat, 23 Apr 2005 13:56:19 GMT Cache-Control: no-store, no-cache, must-revalidate Cache-Control: post-check=0, pre-check=0 Pragma: no-cache Content-Encoding: gzip Vary: Accept-Encoding Connection: close Content-Type: text/html; charset=utf-8 Apparently, the site sends uncompressed pages but tells the browser it is compressed. BTW, links2 and dillo do disply the page but will not allow me to log in and disable cacheing. Any clues? Thanks Michael ------------------------------------------------------------------------ Sat, 23 Apr 2005 14:08:42 +0000 : mjr Perhaps I should mention that zlib.output_compression = Off in my php.ini ------------------------------------------------------------------------ Sat, 23 Apr 2005 18:43:30 +0000 : bertboerland(a)www.drop.org /Apparently i am observing a new variant of that problem / Please open a new bug report, since your problem is likely unrelated to this problem, open an new problem and you might want to use a link to this problem, but dont use this problem for other than descibed. ------------------------------------------------------------------------ Sat, 04 Jun 2005 23:53:22 +0000 : mjr no, it is not unrelated. After some investigation, i found out that the garbage is being stored in drupals cache. This has been reported in this thread. My present workaround is to completly disable drupals caching (and to manually delete the compressed pages from the database.) Whats the use of letting drupal compress pages and store these int its cache table? Just saving a few cycles of CPU time? On the cost of letting the web server and its client decide wether to transfer compressed or uncompressed data - they have the protocol to do that, drupal has not. Michael ------------------------------------------------------------------------ Mon, 06 Jun 2005 16:48:30 +0000 : killes(a)www.drop.org We save cpu cycles and storage space. Apache only needs to check the Encoding header which will be easy. I do not know why zlib output compression doesn't check those headers for some (or all?) people. ------------------------------------------------------------------------ Fri, 10 Jun 2005 23:32:40 +0000 : bluec I have the same problem with Drupal 4.6.1: If the site delivers a cached page it is gziped twice and the result is garbage in my browser window. I tried two differnet browsers: Firefox and Opera and it's the same problem with both of them. I recorded the HTTP headers: - Cache turned on - zlib-compression turned on - Result: Garbage http://kim.sshtun.v63.org:22003/index.html GET /index.html HTTP/1.1 Host: kim.sshtun.v63.org:22003 User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.8) Gecko/20050511 Firefox/1.0.4 Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5 Accept-Language: de-de,en;q=0.5 Accept-Encoding: gzip,deflate Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Keep-Alive: 300 Connection: keep-alive Referer: http://kim.sshtun.v63.org:22003/fachschaft/ Cookie: PHPSESSID=6ddbf3d5a32d-CHANGED-BY-CHRIS HTTP/1.x 200 OK Date: Fri, 10 Jun 2005 23:16:08 GMT Server: Apache/2.0.54 (Gentoo/Linux) mod_ssl/2.0.54 OpenSSL/0.9.7e PHP/5.0.4 X-Powered-By: PHP/5.0.4 Expires: Mon, 26 Jul 1997 05:00:00 GMT Last-Modified: Fri, 10 Jun 2005 23:11:59 GMT Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0 Pragma: no-cache Set-Cookie: PHPSESSID=6ddbf3d5a32d20-CHANGED-BY-CHRIS; expires=Mon, 04 Jul 2005 02:49:28 GMT; path=/ Etag: "03cff5fcc105e72dfc2671d3ffdb285c" Content-Encoding: gzip Content-Length: 5024 Keep-Alive: timeout=15, max=95 Connection: Keep-Alive Content-Type: text/html; charset=utf-8 Now I turned off compression by setting in my .htaccess php_flag zlib.output_compression On and the result is OK: http://kim.sshtun.v63.org:22003/index.html GET /index.html HTTP/1.1 Host: kim.sshtun.v63.org:22003 User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.8) Gecko/20050511 Firefox/1.0.4 Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5 Accept-Language: de-de,en;q=0.5 Accept-Encoding: gzip,deflate Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Keep-Alive: 300 Connection: keep-alive Referer: http://kim.sshtun.v63.org:22003/index.html Cookie: PHPSESSID=6ddbf3d5a32d20-CHANGED-BY-CHRIS HTTP/1.x 200 OK Date: Fri, 10 Jun 2005 23:17:08 GMT Server: Apache/2.0.54 (Gentoo/Linux) mod_ssl/2.0.54 OpenSSL/0.9.7e PHP/5.0.4 X-Powered-By: PHP/5.0.4 Expires: Mon, 26 Jul 1997 05:00:00 GMT Last-Modified: Fri, 10 Jun 2005 23:11:59 GMT Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0 Pragma: no-cache Set-Cookie: PHPSESSID=6ddbf3d5a32d208-CHANGED-BY-CHRIS; expires=Mon, 04 Jul 2005 02:50:28 GMT; path=/ Etag: "03cff5fcc105e72dfc2671d3ffdb285c" Content-Encoding: gzip Vary: Accept-Encoding Content-Length: 5054 Keep-Alive: timeout=15, max=100 Connection: Keep-Alive Content-Type: text/html; charset=utf-8 Turning off the cache is a solution as well. As soon as cache and zlib are operating the result is garbage. If you need any other information just let me know. I can turn on compression at any time and test other solutions. Chris ------------------------------------------------------------------------ Mon, 27 Jun 2005 18:10:12 +0000 : aries The same. Interesting, only one page doing this, all others are not. -- Aries http://aries.mindworks.hu ------------------------------------------------------------------------ Thu, 07 Jul 2005 14:46:34 +0000 : fago quite strange. i never had problems with this, then i upgraded from 4.6.1 to drupal 4.6.2... now i must turn off the cache, or i've the troubles mentioned here... (i haven't touched the apache or php config in the meanwhile) ------------------------------------------------------------------------ Thu, 14 Jul 2005 09:47:32 +0000 : bertboerland(a)www.drop.org yet another one [2], closed than one, raised importance here. [2] http://drupal.org/node/25326 ------------------------------------------------------------------------ Sat, 16 Jul 2005 19:52:26 +0000 : bertboerland(a)www.drop.org killes sigested that is want apache that compressed the page but drupal and php was the first one. after some testing, i found out he was correct! to solve this 1) either comment out the gzip lines in bootstrap.inc 2) put /etc/php.ini "zlib.output_compression = On" to OFF (real) Fix needed. Drupal should look if the page was zipped in the first place.... ------------------------------------------------------------------------ Fri, 29 Jul 2005 12:17:46 +0000 : killes(a)www.drop.org Attachment: http://drupal.org/files/issues/gzip.patch (2.45 KB) Here's a patch that needs some testing. ------------------------------------------------------------------------ Fri, 29 Jul 2005 13:18:51 +0000 : Dries Why do we set $do_cache to TRUE in the nested if's? $do_cache is already set to TRUE higher up so we're just overwriting TRUE with TRUE? Also, can we rename $do_cache to $cache? That is more Drupal-ish. Does that solve all apache-Drupal-caching problems or only some? I vaguely remember we agreed to add a setting, because that was perceived the only True Solution. ------------------------------------------------------------------------ Fri, 29 Jul 2005 18:22:04 +0000 : killes(a)www.drop.org Attachment: http://drupal.org/files/issues/gzip_0.patch (2.46 KB) I've updated the patch. The patch is intended to cure all the problems we had. It needs testing, though. I don't recall to agreeing that a setting would be needed. Just for the record: Steven measured the ratio of serving gzipped cached pages to re-unzipped pages on drupal.org to be about 5.1:1. That is from 6 page requests which we serve from the cache only one needs the cache to be unzipped. This is probably due to crawlers who only speak http 1.0. ------------------------------------------------------------------------ Sat, 30 Jul 2005 14:33:42 +0000 : bertboerland(a)www.drop.org people who expierenced this problem, please try the patch of killes and report here if successful or not. i will patch as well ------------------------------------------------------------------------ Mon, 01 Aug 2005 14:55:47 +0000 : moshe weitzman In my opinion, this is a case of misplaced optimization. Very little CPU is required to gzip a document. Why else would Apache and PHP provide an option to do this *for every page*. PHP knows that all its pages will be dynamic yet it still offers this option. My recommendation is this feature entirely, and take the win in simplicity/code reduction. Furthermore, we won't have any more issues like this one, which has lingered unfixed for 9 months. Admins who want gzip will elect to do so in their php.ini or .htaccess.

1 0

[drupal-devel] Vanilla: the forum software to look for
by Kristjan Jansen 05 Aug '05

05 Aug '05

I posted this a week ago to this list but have not seen it since -- hope this is no dupe ------------------------------------------------------------------------------------------------------------------------- http://getvanilla.com/ http://lussumo.com/community/ Comparing their approach to Drupal's we do not fare bad at all. Some tiny details I like (to rip from them ;): Not logged in: * Overall look. yes, perhaps too Typepad but nice and smooth netherless (perhaps too low-contrast, I have missed many vital UI elements) * "edited" mark next to a comment * Informative login / register dialogs Logged in: http://lussumo.com/community/signin.php guest:guest * switchable small / big textarea (but ihmo it should work vice versa, small -> big) * ajax-based (un)bookmarking * ajax-based (un)blocking topics (taxonomy) * navigation history block * search history block

2 1

[drupal-devel] [feature] Allow for more integers in node_access grant_x columns
by moshe weitzman 05 Aug '05

05 Aug '05

Issue status update for http://drupal.org/node/25395 Project: Drupal Version: cvs Component: node.module Category: feature requests Priority: normal Assigned to: moshe weitzman Reported by: moshe weitzman Updated by: moshe weitzman Status: patch Attachment: http://drupal.org/files/issues/node_access_0.patch (620 bytes) It is useful for modules to store more information in the grant_view, grant_update, Grant_delete columns than just 1 or 0. This patch treats 0 and 1 just as before, and any value greater than 1 is assumed TRUE (just like PHP). Organic groups already has 2 needs for this. One would be to store the 'group role' of a member in this table. Total characters changed: 1 moshe weitzman

2 2

[drupal-devel] [bug] Patch review queue
by robertDouglass 04 Aug '05

04 Aug '05

Issue status update for http://drupal.org/node/28210 Post a follow up: http://drupal.org/project/comments/add/28210 Project: Drupal Version: cvs Component: base system -Category: feature requests +Category: bug reports Priority: normal Assigned to: Anonymous Reported by: robertDouglass Updated by: robertDouglass -Status: active +Status: patch (code needs review) [dirty trick] I'd like to have better ways of making issues like this show up on the devel-list. robertDouglass Previous comments: ------------------------------------------------------------------------ Thu, 04 Aug 2005 17:48:46 +0000 : robertDouglass I've had some fun clicking the "Play patch bingo!" link repeatedly and think it is a very fine addition to the Contributor links block. One thing that really impressed me was the huge number of patches that seem like a good idea, whose authors assert that the patch is ready to commit, and that have several "+1 I reviewed and liked" comments. All of this made me think a little more about the undead issue of whether we need more core committers and how patches can get reviewed and processed in a more organized manner. Then I had an idea. Here it is. In order to 1) motivate the masses of developers to review patches, 2) guarantee that patches don't get buried in the queue and never get read, 3) have a way to *measure* whether Dries and Steven can keep up on their own, why don't we implement a patch review schedule? I imagine it working something like this: * New patches are put into a first-in-first-out queue. This queue is the order in which committers will review the patch. * Committers must (must is a very flexible word here... more about expectations than rules) review the top patch before reviewing the one right in line behind it, kind of like a judge has to review court cases in a fixed order. * The "up next" patches get well advertised so that developers and those interested in a particular patch know when to do their lobbying and when to make sure that things get reviewed by a critical mass of developers so that the committer has enough support to make a decision. * Every patch review must result in a status change or update comment. This is the history or review trail of a patch. This occurs now anyway, so this is no big change but rather an acknowledgment of the expectation that the committers never "silently" look at a patch and then move it to the back of the queue. * Every patch review results in either a committ, having the patch removed from the queue (won't fix), or being sent to the back of the queue with concrete criteria and expectations for when it gets reviewed next. (I'm not suggesting that if Dries writes "remove this space and send a new patch" and someone does this 5 minutes later that the patch has to wait two weeks until it gets to the top of the list) That's the extent of what I propose to add/change. Here's what I hope that we would gain from it. I expect that the testing efforts of the developer community would become more focused, meaning we would be more likely to be testing the same patch at the same time. Why? Let's say that I have a patch in the queue and it is in place 5. That means very soon, Dries or Steven will be reviewing it. My big chance! Since I know that the opportunity is at hand and that the price of not getting it committed this time around will be high (put back to the end of the queue), I will make sure the patch still applies and then get on IRC and say "hey chx, walkah, killes and amazon, please test this patch today... it is up for review and I need some critical mass". The result will be that more eyes and intentions will be focused on the patch at that moment, which hopefully will give Dries the info he needs and has asked for when it comes to review time. Please note that I am not suggesting that committers should be forbidden from following a patch that they're interested in and committing patches from the middle of the queue. I'm proposing a system for addressing those days when they wake up in the morning and say "well, let's see.... maybe I'll review patches today. Let's take a look at that patch queue." The value of the list is that the whole world will know that very soon, the patch on the top of it will be reviewed and when it is, that is the biggest chance for a while that it will get committed. Furthermore, nobody submitting an issue will have the voice in their head saying "hope Dries is online today so he sees this patch before it gets burried in the queue". Instead you'll know that your patch *is* burried at the end of the queue, but that it will definitely, 100% guaranteed, get reviewed eventually. We can also train the masses to participate more in the review process. If the "up to bat" and "on deck" and "in the hole" issues are visible on drupal.org, and if we provide nice instructions for reviewing patches in such a way that people get the idea "oh, this is something I'm expected to do", I think we'll have a higher review rate. Another thing that such a queue would open the door for is an issue freeze that comes before a code freeze in the development cycle (of course critical bugs will always be handled ASAP and not be put in any queue). Maybe two weeks before a code freeze we can decide that all new issues will go into the queue for the next cycle, and the timing of the code freeze will depend on working through the current queue. As it is, there are dozens of small bugfixes in the queue which will likely not see the light of day before the code freeze or release. This method of organizing the queue might help give us more concrete goals for each stage in the cycle. Finally, and this is the part that is hardest to say since I love Dries and Steven and think they do a bang-up job, a system like this would make it very clear if the system isn't keeping up with demand. If we can't even make it through the whole queue once per development cycle, there is a bottleneck that is restraining our progress. Perhaps we'd decide, for example, to not increase the number of committers, but rather that we need a pre-commit team (5-10 people) who are the ones who review every patch, and only the ones they approve get sent to Dries and Steven. A solution like that would make two similar queues; the one I just described with everything in it which will get reviewed in order by memebers of the pre-committ team, and a queue for Dries and Steven that functions the same way but only contains patches that have passed the pre-commit review phase and was promoted by a member of the pre-commit team (perhaps members of the pre-commit team cannot promote their own patches). Ok - there it is. Flame me! cheers, Robert ------------------------------------------------------------------------ Thu, 04 Aug 2005 19:11:09 +0000 : robertDouglass Another benefit that I forgot to mention is that the price of ignoring crappy issues is higher. Take the case where a really important and really anticipated patch is in place 12 and the 11 issues before it are crap, meaning submitted by bad coders, not addressing important issues and generally unlikely to make core. Anyone interested in the good issue in place 12 will be saying "hey cats, can't we clean the litter box here, I mean, what is all this crap keeping us from getting to the real deal". And *whooosh*, 11 crappy issues are marked to "get a life" or at least "won't fix" and removed from the queue. The price of ignoring issues is high already; our good issues get lost in the crap. The real price we pay for this is delayed and comes in the form of a bloated queue that a core committer won't touch with a ten foot clue-bat. By requiring that we clear the crap to get to the meat the price is moved upfront, which might seem more expensive, but the slim and trim patch queue that will result will be worth it and the overall price of crappy issues will be lower.

1 0

[drupal-devel] [feature] Sort logs by ID instead of timestamp
by njivy 04 Aug '05

04 Aug '05

Issue status update for http://drupal.org/node/23560 Project: Drupal Version: cvs Component: watchdog.module Category: feature requests Priority: normal Assigned to: Anonymous Reported by: njivy Updated by: njivy Status: patch In watchdog.module I propose changing the key used to sort logs chronologically. Instead of using the timestamp, which is accurate only to the nearest second, we could use the log entry id ("wid" in the database). As I understand it, the IDs are assigned sequentially, so this is a good approximation to the correct order in which the events are recorded. Also, I don't know of any events that are recorded after a delay. The problem with the current sorting key arises when logged events come within 1 second of each other. The order of events is not always preserved when viewing the log because the timestamp is identical for several events. In "modules/watchdog.module" neard line 91, the word "timestamp" is replaced with "wid": $header = array( ' ', array('data' => t('Type'), 'field' => 'w.type'), array('data' => t('Date'), 'field' => 'w.wid', 'sort' => 'desc'), array('data' => t('Message'), 'field' => 'w.message'), array('data' => t('User'), 'field' => 'u.name'), array('data' => t('Operations'), 'colspan' => '2') ); Preliminary testing suggests this is a good tweak. njivy

7 7

[drupal-devel] [bug] too much information stored during a profile edit
by killes 04 Aug '05

04 Aug '05

Issue status update for http://drupal.org/node/21515 Post a follow up: http://drupal.org/project/comments/add/21515 Project: Drupal -Version: 4.6.0 +Version: cvs Component: user.module Category: bug reports Priority: minor Assigned to: Anonymous Reported by: fago Updated by: killes(a)www.drop.org Status: patch (code needs review) We are currently addressing this issue through a more general rewrite of the form subsystem. For discussion about it see here: http://lists.drupal.org/archives/drupal-devel/2005-06/msg00091.html killes(a)www.drop.org Previous comments: ------------------------------------------------------------------------ Thu, 28 Apr 2005 16:07:26 +0000 : fago as i already mentioned in this post [1] some time ago, arbitrary data can be stored in the serialized data field of the user table. example: go to my account, save the page local, edit it to send the "post" data to the website and edit an form name to edit[somethingnew] und press save -> $somethingnew will be saved serialized in the data field i think the fault is in user_edit (on line 1145) where user_save() is called without checking $edit completely. however, i 've felt like that's known and tolerated, so i did, the a little bit missunderstood, post above first. but i think, it's not acceptable to save just everything, what is coming in with the form data... what's if a bad guy saves 1GB of nonesense in my database? furthermore it will be loaded into the memory everytime user_save is called - seems to me to be a nice possibilty for a DOS. [1] http://drupal.org/node/17744 ------------------------------------------------------------------------ Thu, 28 Apr 2005 18:13:51 +0000 : moshe weitzman i think apache/php have limits on how much can be posted. if you are worried, perhaps use those settings. ------------------------------------------------------------------------ Thu, 28 Apr 2005 19:53:36 +0000 : fago yes, but then it's still possible to inject data in little pieces by using a new name every time. further you can't limit the size too much, because file uploading should also work. ------------------------------------------------------------------------ Sun, 01 May 2005 20:30:51 +0000 : fago Attachment: http://drupal.org/files/issues/saveform.patch (4.38 KB) i've written a patch, so that $edit is checked before. unfortunately i couldn't see a better method than introducing a new hook type for hook_user 'saveform' for which each module has to return an array of values, which it wants to be saved. as an affect some modules have to be changed to work correctly after applying this patch :( my patch (for the 4.6 branch) includes the changes for profile.module and contact.module and for the user.module of course. what do you think about this? i think something like this is necessary. consider a module, which introduces profile fields, which can only be edited by administrative users. and yes of course, i don't like it, if users are able to fill up there user object with additional variables ;) ------------------------------------------------------------------------ Sat, 07 May 2005 00:08:18 +0000 : fago Attachment: http://drupal.org/files/issues/saveform_0.patch (4.52 KB) unfortunately i noticed some mistakes in the previous patch. (module_invoke doesn't reference $edit, but then i patched profile.module to use the saveform hook for saving.) i've attached a new patch.

1 0