Posted by toolsmythe http://drupal.org/user/1746988 on January 4, 2012 at 8:18pm
I'm dealing with an odd performance issue on a production Drupal 7.7 site.
I have a machine that is configured thus:
4gig Quad proc 900GB disk
OS: Debian 6
For apache we are using version ii apache2 2.2.16-6+squeeze1 Apache HTTP Server metapackage ii apache2-mpm-prefork 2.2.16-6+squeeze1 Apache HTTP Server - traditional non-threaded model
For PHP we are running version ii php5 5.3.3-7+squeeze3 server-side, HTML-embedded scripting language (metapackage) ii libapache2-mod-php5 5.3.3-7+squeeze3 server-side, HTML-embedded scripting language (Apache 2 module)
ii mysql-client-5.1 5.1.49-3 MySQL database client binaries
Database is on a seperate server.
The Problem We are getting a pretty consistant 5+ second delay before any page on the site renders. It renders quickly once it starts to render, but there is a delay of 5 or more seconds before that happens.
Oddly enough, we have a copy of the site running under cPanel that has no delay whatsoever. That box is configured thus:
Apache 2.2 PHP 5.2.17 Mod SuPHP CentOS release 5.7 (Final)
Not sure if this is worthy of note, but the production server has Suhosin (PHP hardening) and the copy does not.
We made a second copy on the prod server and it is experiencing the same delay.
All the database instances are practically empty.
What we've tried so far We've tried optimizing the tables in MySQL.
We've tried applying various optimizations to Apache and MySQL recommended on Drupal pages (for example: http://drupal.org/node/215516 and http://drupal.org/node/2601).
No joy.
We did an ab benchmark on both servers. as follows:
ab -n 500 -c 50
The test server showed this:
Benchmarking .com (be patient) Completed 100 requests Completed 200 requests Completed 300 requests Completed 400 requests Finished 500 requests
Server Software: Apache Server Hostname: .com Server Port: 80
Document Path: /clients/// Document Length: 18543 bytes
Concurrency Level: 50 Time taken for tests: 50.229747 seconds Complete requests: 500 Failed requests: 0 Write errors: 0 Total transferred: 9663634 bytes HTML transferred: 9327129 bytes Requests per second: 9.95 [#/sec] (mean) Time per request: 5022.975 [ms] (mean) Time per request: 100.459 [ms] (mean, across all concurrent requests) Transfer rate: 187.88 [Kbytes/sec] received
Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 1.0 0 9 Processing: 703 4901 5312.1 3312 38328 Waiting: 657 4653 4788.4 3134 33105 Total: 703 4902 5312.4 3312 38330
Percentage of the requests served within a certain time (ms) 50% 3312 66% 4721 75% 5718 80% 6374 90% 10416 95% 14444 98% 24669 99% 33351 100% 38330 (longest request)
Meanwhile, here's the same test for our prod server:
Server Software: Apache/2.2.16 Server Hostname: .com Server Port: 80 Document Path: /clients// Document Length: 18760 bytes Concurrency Level: 50 Time taken for tests: 314.671 seconds Complete requests: 500 Failed requests: 0 Write errors: 0 Total transferred: 9757000 bytes HTML transferred: 9380000 bytes Requests per second: 1.59 [#/sec] (mean) Time per request: 31467.146 [ms] (mean) Time per request: 629.343 [ms] (mean, across all concurrent requests) Transfer rate: 30.28 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 0.3 0 1 Processing: 11813 29570 12525.7 27341 126615 Waiting: 11745 29509 12525.6 27281 126535 Total: 11813 29570 12525.7 27341 126615 Percentage of the requests served within a certain time (ms) 50% 27341 66% 31105 75% 33440 80% 35335 90% 40918 95% 47849 98% 66717 99% 84633 100% 126615 (longest request)
We tried turnning on Long Running Queries in MySQL; it logged nothing.
top showed nothing of interest.
We're stumped over here (though to be quite honest, stumping us is no great feat <grin>, at least when it comes to Drupal). Any thoughts would be much appreciated.