Please share, if 2k qps for loaded drupal for 3k concurrent users is OK. My server (2x2G Xeon 1G Ram) goes mad of such load and once a day just hanged within second (even did not write err_log). What can it be? Might it be core problem?
2000 queries per second sounds high. I have seen up to 800 QPS on a quad Xeon 2.8GHz with 2GB RAM, and 179,000 page views (peak) per day. Check if the number of Apache processes is too high (e.g. 256 which may not fit in your RAM without swapping), and lower it if necessary. Also, post in the performance and scalability forum. On 10/10/06, Tamir Khason <tamirk@gmail.com> wrote:
Please share, if 2k qps for loaded drupal for 3k concurrent users is OK. My server (2x2G Xeon 1G Ram) goes mad of such load and once a day just hanged within second (even did not write err_log). What can it be? Might it be core problem?
Check if the number of Apache processes is too high (e.g. 256 which may not fit in your RAM without swapping), and lower it if necessary.
Regarding this, what I do is watch the process table, see the average memory footprint for each apache process, let's say it's 6 mb. You have 1000 mb, leave some RAM for mysql and other services, so remaining is 600 mb (just an example). 600/6 = 100. And that's the Max apache processes I will be allowing. Because more than that it means my apache will swap. If your theme has lots of images, consider tweaking your Keep-Alive (http://httpd.apache.org/docs/1.3/keepalive.html) and cache (http://www.mnot.net/cache_docs/), it will help save unnecessary apache processes. Also, consider serving your non php files under thttpd (http://www.acme.com/software/thttpd/) or another light web server. Consider moving your rewriting rules to to conf files instead of .htaccess, due to way rewriting works, it internally resets part of the request processing cycle when invoked too late (e..g via .htaccess) while having it in conf files enables it to take action early enough. Consider removing your .htaccess completely, move the configuration to .conf files, and disable .htaccess in apache conf. This will save you a few system calls. If your site is a community one that gets updated a lot, consider turning off query cache for some tables. Because in that case, cache becomes invalid in very short intervals and maintaining cache becomes overhead. No one sugguested a Drupal performance tip. I will kick the ball off, it's a bit nuts, might break things but this is the devel list right ? I find many sites not using Drupal's aliases feature. If you are one of those, consider turning off aliases lookup. I do this quickly by adding return FALSE in first line in (http://api.drupal.org/api/4.7/function/drupal_lookup_path). This will save you some queries. Of course you will want to benchmark all that and see what works best for you.
I have another client site that shoots up to 3,000 QPS, but I don't have the statistics of page access for it. This one has 4 Xeons 3.2GHZ each and 4GB. Write a simple script to run vmstat every 15 seconds or so, log the output to a file. Keep it running until the "hang" happens, and then see what it tells you about the system. Are you out of CPU? Do you have too many blocked processes or runable processes? Do you have a lot of wait on i/o? Are you swapping to the point of thrashing. Only after you do this analysis you should decide whether to upgrade the RAM or just tune things. Regarding decreasing the number of Apache processes, yes, that means you can only serve so many users, but you prevent the chaos caused by swapping. Think of it as a popular restaurant. If they let in more people than they have tables/seats, would the customer be happy? Would the staff cope with it?
No one sugguested a Drupal performance tip. I will kick the ball off, it's a bit nuts, might break things but this is the devel list right ? I find many sites not using Drupal's aliases feature. If you are one of those, consider turning off aliases lookup. I do this quickly by adding return FALSE in first line in (http://api.drupal.org/api/4.7/function/drupal_lookup_path). This will save you some queries.
you save all those queries just by having an empty url_aliases table. no need to change code.
On Oct 10, 2006, at 4:32 PM, Tamir Khason wrote:
Please share, if 2k qps for loaded drupal for 3k concurrent users is OK. My server (2x2G Xeon 1G Ram) goes mad of such load and once a day just hanged within second (even did not write err_log). What can it be? Might it be core problem?
This isn't a software solution, but bumping up your RAM could buy you a lot. I don't know what kind of caching you're doing, but more ram gives you room for (more) query_cache, which is a little thing that can buy you a lot. IME, once MySQL runs out of ram, the read/write temps can bring you to a crawl, and then the queries just pile up. RAM is cheap. Even after you optimize your code, that headroom can help performance. Laura
So decreasing the apache processes can help? For me it makes sense to increase it in order to process cuncurrently as much users as i can On 10/11/06, Laura Scott <laura@pingv.com> wrote:
On Oct 10, 2006, at 4:32 PM, Tamir Khason wrote:
Please share, if 2k qps for loaded drupal for 3k concurrent users is OK. My server (2x2G Xeon 1G Ram) goes mad of such load and once a day just hanged within second (even did not write err_log). What can it be? Might it be core problem?
This isn't a software solution, but bumping up your RAM could buy you a lot. I don't know what kind of caching you're doing, but more ram gives you room for (more) query_cache, which is a little thing that can buy you a lot. IME, once MySQL runs out of ram, the read/write temps can bring you to a crawl, and then the queries just pile up. RAM is cheap. Even after you optimize your code, that headroom can help performance.
Laura
So decreasing the apache processes can help? For me it makes sense to increase it in order to process cuncurrently as much users as i can
Not always. It's true that more processes can handle more requests depending on where the bottleneck is. But think like this, if you add too many apache processes and run into memory problems then your processes begin to swap. And that's the real problem as your disks start trashing. Better to leave the incoming connection in your OS's ip network queue rather than force open another apache you haven't room for. That may seem odd (to leave it queued) but at least the running apaches can handle them faster if memory swapping isn't going on. --AjK
On 11 Oct 2006, at 06:44, Laura Scott wrote:
This isn't a software solution, but bumping up your RAM could buy you a lot. I don't know what kind of caching you're doing, but more ram gives you room for (more) query_cache, which is a little thing that can buy you a lot. IME, once MySQL runs out of ram, the read/ write temps can bring you to a crawl, and then the queries just pile up. RAM is cheap. Even after you optimize your code, that headroom can help performance.
Be careful here because Tamir might take your advice, spends his money on RAM, only to find out that it is not helping him one bit. RAM _might_ help iff (i) the problem is memory-bound and (ii) he properly tuned MySQL. If, however, the problem is CPU-bound or I/O- bound (disks), extra RAM is not likely to help him. Or maybe he hasn't turned on the query cache to begin with. -- Dries Buytaert :: http://www.buytaert.net/
participants (7)
-
AjK -
Amr Mostafa -
Dries Buytaert -
Khalid B -
Laura Scott -
Moshe Weitzman -
Tamir Khason