On Mon, 2009-01-05 at 09:00 -0500, Earnie Boyd wrote:

Quoting Matej Svetlik <matej.svetlik@cetelem.sk>:

...

hello, We have drupal page with 50+ modules. With every new module added, drupal is getting slower and slower (with load times near 2 seconds).

After installation of APC load times dropped drastically but are still slow.

Are there any common ways to handle installations with lots of modules (and even more include files)?

pages depend on 2 external systems, but loading data form them takes only 0.3 second.

PS: I tough about "hacking" core to load only modules that are really needed for the page that is loading (modules doesn't depend on each other). I'm just curious if it's possible?

Be sure to test with the version 7 betas. Version 7 is trying to accomplish speed improvement. However, this may require you to modify your 50 modules. The way the hook methods are currently implemented are the cause for the need to load all modules for every page. Caching will help with the page loading, you do have caching turned on, correct? Have you looked at the database optimization?

thanks for reply ... 1) i will try 7 as soon as possible, but is really hard to convert all modules (I converted all of them from 5.x to 6.2 few months ago :D) 2) I haven't try database optimization yet. But i didn't think database is the problem. In devel module there is an option to show database access times and they were irrelevant compared to page load times. Database runs on the same server and the server is almost never fully loaded. Oh, and it's intranet server just for about 200 employees. 3) I think the main problem is _hook system, because on fresh installation it takes only 0.3 second to load page. that's why I tough about hacking module loading code. I only need to load core modules, one custom module and one module displaying current page. 4) Another option could be splitting it into multi-site/multi-page installation ... but i like core hacking idea much more :D

hello, thanks for all replies here is what i got so far:

The main problem is that Drupal 5 and earlier by design loaded ALL code on every page load. In Drupal 6 we introduced split include files for page callbacks, so most of those "foo.admin.inc" and "foo.pages.inc" files are not, in fact, loaded on every page request but only when needed. There are similar techniques that individual modules can implement if necessary for their own code, and some of the larger ones like Views and CCK do, but there's no central mechanism for it.

All my modules are split to many .inc files loaded only when needed. But drupal still includes all 67 .module files.

Hooks aren't the issue per se, although it is true that there's a cost to checking 50 modules for a hook.

Both of those are seeing major improvements in D7, but they're not anywhere near production ready yet.

An opcode cache will help immensely for the first problem, as that's the main thing it solves. Proper tuning of the cache (you can set it to not even stat the disk) will help, too. I tried APC ... load times are faster but not enough :D (according to APC stat, there is enough cache memory and other settings seem OK too) I tried turn off file modification chceking ... not much improvement and APC include_once hack ... with that option set, drupal refused to start ...

Beyond that, depends on your modules. You may find some you do not, in fact, need. Actually, problem is that i need all of them ;). There are few core modules. All other modules are custom and are used daily. (Most of them are just form and "do-something-with-external-system" when submitted.)

Others you could look into improving by implementing some sort of lazy- loading mechanism like Views does. (I believe Views' mechanism is being split off to the ctools module, now in alpha.) You'd have to work with the maintainers for those. Some modules wouldn't really support it, sadly. For external systems, definitely look into proper caching, even if you think it's fast. You can also look into tuning the block cache, or using memcache instead of the default db caching, perhaps, but that's probably overkill if you are just running an intranet server.

Problem with caching is that all modules use "live" data, or are generating unique reports. I can't cache anything.

Anyway, that's just general purpose advise. And, um, no, don't hack core. Really. You will regret it. Maybe not today, maybe not tomorrow, but soon, and for the rest of your life.

I know that hacking core is the last resort and is generally bad idea :D. ------------------------------------------------------------------------------------------------- And there is another thing that can help: I need that only one module (used most and in critical process) should be as fast as possible. So i tried little trick: There is function list_modules in core returning list of modules witch are then loaded. As i only need one module to be as fast as possible and i don't really care about others. In the list_modules function i check witch module is being loaded (something like if $q is like /is/*). If it's my module - instead of full module list, I return manually created reduced module list. Result is that only 17 modules out of 67 are loaded ... but the problem is that it didn't helped much :( - maybe the bottleneck is somewhere else and not in drupal module load system. I'm going to try few more tricks :) and I write back when I find out something interesting.

On Wed, 2009-01-07 at 10:04 -0500, Earnie Boyd wrote: it's set to "public" download method

Quoting Matej Svetlik <matej.svetlik@cetelem.sk>:

...

hello, thanks for all replies

here is what i got so far:

What do you have the Download Method set to on admin/settings/file-system? If it is set to Private you have increased the load time of every page regardless of if the page supplies a method to upload/download a file.