Hi there!
In my quest to improve caching for Drupal, I've devised a cache that
would let us have more than one cache table. The idea came from the
observation that some cached items will quickly be invalidated again
(page cache) while others will last a while (menu, filter). This is a
problem for the mysql query cache, which will then also be invalidated.
Also, operations on smaller tables will be faster.
The patch can be found here:
http://drupal.org/node/72290
I have now run this patch on drupal.org for several hours and here are
some results.
First, let us look at the cache hits vs cache call ratio as a percentage
of the total tries. In our case that is the number of SELECTs minus the
number of UPDATEs devided by the SELECTs on a particular cache table:
(# selects - # updates) / # selects
Explanation: We run a SELECT in every case to determine if there is a
cached item, but only the ones which do not trigger a subsequent cache
update were successfull.
filter: (242318-21780)/242318 = 91%
page: (13315-7285)/13315 = 45%
menu: (9989-161)/9989 = 98%
the rest: (986-488)/986 = 50%
So, we have now proof that our page cache isn't working well for a busy
site such as drupal.org, but the filter cache seems to work fine while
the menu cache is truly excellent. The "rest" are a few feeble attemps
on caching in cvs and project module, which apparently don't work too
well either.
The high part of cache misses for the page cache is probably in part due
to the fact that several crawlers are doing their business on drupal.org
at all times. These will generate a cached page, but if the page is in
fact very uninteresting to humans then the only likely receiver of a
cached page is another crawler. And there aren't that many of them that
this would happen. I've suggested to make Drupal less attractive for
crawlers here: http://drupal.org/node/65017 If we were successful with
this, we'd have less pages generated and thus the percentage of
successfull cache hits would increase.
I have to note that the filter on drupal.org usually only caches the
entries for four hours, not 24 h as a stock drupal. This change was made
to keep the number of cached items at bay. I did change this to the full
24 hours. This was not intentional as I can not compare my data with the
older data I got in all cases.
Number of cached items
This numbers obviously fluctuates a lot:
filter: 132825 (usually about 7000 for the 4h cache)
page: 7345 (a very high value, usually between 1000 and 3000)
menu: 1040
rest: 79328 (I had forgotten to remove the previous items from the
table, there were only 500 relevant items)
Speed
The individual cached items differ in number and size. Also does the
time these queries take.
SELECT INSERT UPDATE DELETE
filter: 0.002 0.005 0.002 0.087
page: 0.007 0.009 0.003 0.188
menu: 0.011 0.014 0.016 n/a
rest: 0.002 0.007 0.004 n/a
The page and menu items are the largest, so no surprise that the time is
also longest. The number of entries in the table does not seem to matter
that much.
Total weighted average:
SELECT INSERT
Now 0.003 0.007
Test 1 0.010 0.011
Test 2 0.007 0.008
There is a marked improvement here. Splitting up the cache apparently
has the benefits I tried to achieve: Reduction of query times by having
smaller tables and splitting off less-often-changing data into separate
tables to allow the query cache to operate more effectively.
Considering that the cache_get query is the query which accumulates more
time than any other query on drupal.org, I think that we should strongly
consider this patch.
Cheers,
Gerhard
Hi guys,
I've noticed that the past 2 commits to settings.php were both made
inadvertently, due to someone including a settings.php $db_url change
in their patch by mistake. As you can see, both of these commits had
to be reverted:
http://drupal.org/cvs?file=/sites/default/settings.php
Can I request that all of the patch authors among us (including
myself) _please_ check your patch files, and that you ensure that
there are no custom settings.php $db_url (or other) changes that have
slipped into your patch. I think that the primary responsibility for
this should lie with patch authors. However, it would be good if the
core committers could be on the lookout for accidental settings.php
changes in patches as well.
If anyone has any tips for excluding settings.php from a patch, that
would be greatly appreciated. Personally, I just run "cvs diff -u >
foo.patch" from the root Drupal install dir, and then manually edit
the file to remove the settings.php cruft. I'm sure there's a better
way to do it. ;-)
The biggest problem that I have with accidental settings.php changes,
is that running "cvs up -dP" after such a change results in CVS
complaining about a conflict/merge in settings.php, even though the
change was reverted. So let's try and keep _intentional_ settings.php
changes to a minimum (which we seem to be doing quite well at), and
keep _accidental_ settings.php changes to zero.
Thanks,
Jaza.
Have you guys had a look at the SandsCSS theme? http://drupal.org/node/57629
I'm no CSS expert, so I don't know if it's all that under the hood, but it
does make for a prettier Bluemarine.
MichelleC
On 8/3/2006 1:50:40 PM, Neil Drumm (drumm(a)delocalizedham.com) wrote:
> Bluemarine refresh:
> 1. Replace the flat Druplicon default icon with the cell-shaded one.
> 2. Throw in a tableless layout.
> 3. Saturate the light blue colors a bit.
> 4. Add whitespace.
> 5. Improve the fonts a bit.
> 6. Repeat in 1-2 years or as needed.
i'm looking for some opinions, ideas, feedback...
I have a client with a bricolage (perl-based cms that spits out static
html files for publishing, popular with newsletter, newspaper sites)
site, that wants to offer a user comments feature. There was already
an attempted integration with vbulletin that didn't work super well,
and they liked the other parts of drupal so I was approached to see
what kind of drupal integration might work. That was many months ago -
at this point I have a working prototype waiting for the client to
decide what to do next. I'll describe here my strategy and am looking
for general feedback, criticism, other approaches or refinements.
If I was starting from scratch, I might use an approach I've seen or
heard about that just pumps the bricolage output into node bodies, but
this bricolage install has already got a lot of logic about the
navigation/theming in it that I thought would lend itself to a less
drupal-centric approach.
So (omitting the tedious details):
1. I've laid a drupal install over the existing bricolage site's document root.
2. i've created a 'bricolage' module which defines a
'bricolage_render' callback function for the path 'bricolage'.
3. I've created a custom 404 function in the module called
bricolage_not_found to handle page not found replies.
4. I've adapted the .htaccess file to have drupal handle .html files.
The html that this bricolage install generates includes some php,
which was being used to provide the vbulletin interface and that
defines the bricolage publication id, as well as whether the article
should allow comments.
Here's how my module works now:
1. when a bricolage url gets called for the first time (they're always
different from regular drupal paths), the bricolage_not_found function
is invoked and reads in the corresponding bricolage .html file and
generates a new 'bricolage' type node that has an empty body and some
extra fields for the url and the bricolage id, and then it also
generates a path alias to convert the bricolage url to the drupal path
'bricolage/<bricolage_id>'. (it figures out the bricolage id by doing
a drupal_eval on the bricolage output). Then it does a drupal_goto to
reload the same url.
2. the second (and all subsequent invocations) of the bricolage url
then trigger the bricolage_render callback function. This function
looks up the location of the bricolage-generated content (yes, it
could try to just get it from the request_uri, but we don't want to
rely on that for a few reasons), and then does a drupal_eval to get
the output of the page. Then it can selectively invoke whatever other
modules it wants (in my case, the comments module), to generate the
dynamic content, and sticks it into the output.
In other words - it treats the bricolage html pages as templates that
drupal outputs as is with some selective mangling (in my case, just
comments, but potentially other bits as well, like blocks).
Issues:
1. i've chosen to output the content via a callback instead of as a
node because I didn't want the overhead and potential mangling of
other nodeapi-type modules. It means that these urls are special and
don't behave like the other node content on the site - in particular,
the drupal search doesn't work. This could be fixed, but my client
doesn't need it (yet anyway, they're using google).
I did put in a function so that if the corresponding node gets invoked
directly with a node/<nid> url, it'll redirect to the bricolage/<bid>
url (or rather, it's path equivalent).
2. it's definitely not plug-and-play (yet?) - in my case I needed also
to migrate the existing comments into drupal (i did put that in a
separate module), and there's probably other code that would need to
be written to make this module useable as is with other bricolage
sites.
Anyone?
--
Alan Dixon, Web Developer
http://alan.g.dixon.googlepages.com/
a couple of months ago i posted an issue about some core (user.module)
code which didn't get much attention (none), presumably because I gave
it low priority. Assuming from recent posts that code quality is of
higher priority than I thought, and since Jax just posted a patch for
my concern, i invite you all to look at:
http://drupal.org/node/66945
--
Alan Dixon, Web Developer
http://alan.g.dixon.googlepages.com/
Kahlid-
I wrote that page, so I think the answer is a guarded 'yes.'
We did find some odd behavior when we turned the patch on, which is noted in
the Handbook page:
-- WARNING --
There is a known issue that occurs if you block .png files with this method.
It affects the system_test() function in system.module, which relies on a
phantom .png call to the directory '/system/test/'.
----
There may be other bugs caused by our original patch, so I think more
testing (especially on 4.7, since the patch was put in place for 4.6) is
definitely needed.
To make debugging easier, I have opened http://drupal.org/node/76928 and
attached the .htaccess file that we currently use.
I also had to modify function system_view_general() in system.module to
hard-code clean url settings, since the .htaccess rules broke Drupal's
clean_url checker.
- Ken Rickard
agentrickard
Date: Wed, 2 Aug 2006 23:13:38 -0400
From: "Khalid B" <kb(a)2bits.com>
Subject: [development] .htaccess patch for 404 of Non-drupal files
To: development(a)drupal.org
Message-ID:
<4a9fdc630608022013k5b15ae14kdf2fc0195911e510(a)mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
It is worth including the change in this page http://drupal.org/node/56773
to the standard .htaccess we distribute?
This has two advantages:
a) Avoid a Drupal bootstrap for every 404 of these files.
b) Send a smaller number of bytes when this happen.
I believe the majority of sites would not want Drupal to handle 404 for
these
files.
I can make the patch if it has a chance of getting through.
Work on http://drupal.org/node/74326 (to use FormAPI-style structured
arrays to build node bodies) has been going nicely. Things seem to work
very well, and the flexibility payoff is big. Right now, though, we need
to gather some good numbers on how much the change impacts page
rendering times.
Several of us have taken a stab at it, but the results are inconclusive
-- working on local laptops or busy shared servers where any real impact
was lost in the noise of other server load issues.
If there are any folks out there with experience benchmarking, any help
would be appreciated.
--Jeff
Hi everyone,
Can somebody tell me what the proceedure is with unmaintained modules?
Specifically Im talking about the Invite module -- It *appears* that the
author is not contactable and does not look like the module is being
maintained. (his last activity on drupal.org was in march)
I've been working on the module, and have fixed a number of bugs and
made some improvements but don't know how to go about getting them
included/reviewed for this module.
I would be happy to take over the maintenance, or commit as a new module
-- whatever is the norm.
thanks
--
Nick Wilson
http://performancing.com/user/1
Hi,
I've work on the drupal interface of the social network code. The module
is in early stage yet, but can be tested and tried out here:
http://sna.drupaler.net . The installation and the usage of the module
is not trivial yet.
The source code and the details are available at the URL too.
Thanks for any feedback,
Aron Novak
