Hello, I'm having trouble with CPU usage on my Drupal 5 site. I just enabled caching, and hopefully that will solve the problem, but I want to look at what else I can do to reduce CPU usage further. On the performance page, Drupal gives me the option to "Aggregate and compress CSS files". This sounds pretty good, as it would reduce bandwidth and HTTP requests. But I'm concerned that the compression might increase CPU usage. Can anyone tell me if selecting the "aggregate and compress CSS files" option will increase my CPU usage? Can anyone suggest other ideas to decrease my CPU usage? Thanks, Daniel.
Back in the 4.x days there was a caching module for the sidebar blocks. I don't know if the 5.x caching includes the sidebar blocks, but you might want to look into that. A On Wed, May 13, 2009 at 6:40 AM, Daniel Carrera <daniel.carrera@theingots.org> wrote:
Hello,
I'm having trouble with CPU usage on my Drupal 5 site. I just enabled caching, and hopefully that will solve the problem, but I want to look at what else I can do to reduce CPU usage further.
On the performance page, Drupal gives me the option to "Aggregate and compress CSS files". This sounds pretty good, as it would reduce bandwidth and HTTP requests. But I'm concerned that the compression might increase CPU usage.
Can anyone tell me if selecting the "aggregate and compress CSS files" option will increase my CPU usage?
Can anyone suggest other ideas to decrease my CPU usage?
Thanks, Daniel. -- [ Drupal support list | http://lists.drupal.org/ ]
-- Adam Gurno adam@gurno.com
See http://drupal.org/project/blockcacheI tried it before but did not see much difference, but its easy to try. In D6 block cache is in core. There is also http://drupal.org/project/advcache for caching of authenticated users (on site where you have a lot of logged in users like social networking sites) but I haven't used that one yet, I'm curious to hear some comments of people that have used this module. Hans www.koba.be 2009/5/13 Adam Gurno <adam.gurno@gmail.com>
Back in the 4.x days there was a caching module for the sidebar blocks. I don't know if the 5.x caching includes the sidebar blocks, but you might want to look into that.
A
Hi Daniel, Some ideas. If you can change settings on your server check out the archive with very interesting articles on the subject on http://2bits.com/articles/drupal-performance-tuning-and-optimization-for-lar... The building of the cache takes a bit of time and consumes CPU also of course. It also all depends whats your ratio of logged in and anonymous users (you can see this easily in admin menu). If your problem with CPU comes from a lot of logged in users Make sure you disabled and uninstalled all modules which you don't use. Reduce number of cron runs and backups. Try to see where the load comes from. Search and views (especially views with relationships) can be heavy consumers. With your own server you could use Apache Solr as a replacement for the Drupal search. A radical but very effective solution for reducing CPU is to install http://drupal.org/project/boost it makes a static pure html copy of your pages. I'm using it on a few sites with great success, page loads dropped to 0.3s for anonymous users and also server load decreased with factor between 5 and 10. css aggregation is very good for loading speed performance; I'm not sure about the exact influence on cpu but I expect its not of much influence so I would put it on. Wim Leers has a very interesting article on page loading performance: http://wimleers.com/article/improving-drupals-page-loading-performance Good luck, Hans 2009/5/13 Daniel Carrera <daniel.carrera@theingots.org>
Hello,
I'm having trouble with CPU usage on my Drupal 5 site. I just enabled caching, and hopefully that will solve the problem, but I want to look at what else I can do to reduce CPU usage further.
On the performance page, Drupal gives me the option to "Aggregate and compress CSS files". This sounds pretty good, as it would reduce bandwidth and HTTP requests. But I'm concerned that the compression might increase CPU usage.
Can anyone tell me if selecting the "aggregate and compress CSS files" option will increase my CPU usage?
Can anyone suggest other ideas to decrease my CPU usage?
Thanks, Daniel. -- [ Drupal support list | http://lists.drupal.org/ ]
-- Hans Rossel KOBA Webdevelopment Kerkstraat 228 9050 Gent 09-334.52.60 0472-79.32.16 www.koba.be info@koba.be
Hello Hans, Thanks for the great info Replies below:
If you can change settings on your server...
I can't. Shared host.
The building of the cache takes a bit of time and consumes CPU also of course. It also all depends whats your ratio of logged in and anonymous users (you can see this easily in admin menu). If your problem with CPU comes from a lot of logged in users
Where can I find this ratio? The best I've found is the list of top visitors in the past 3 days. It looks like anonymous visitors dominate.
Make sure you disabled and uninstalled all modules which you don't use. Reduce number of cron runs and backups.
I disabled one module, but not much. Cron jobs are only once a day, in the morning, and backups are done by the host.
Try to see where the load comes from. Search and views (especially views with relationships) can be heavy consumers.
Hmm... I do have search and views. I'll take a closer look at those. I think I can probably do without the views.
A radical but very effective solution for reducing CPU is to install http://drupal.org/project/boost it makes a static pure html copy of your pages. I'm using it on a few sites with great success, page loads dropped to 0.3s for anonymous users and also server load decreased with factor between 5 and 10.
Thanks. I'll take a closer look. Is there any drawback to making static pages? Can you edit the pages later? Why doesn't everybody use this?
css aggregation is very good for loading speed performance; I'm not sure about the exact influence on cpu but I expect its not of much influence so I would put it on.
Ok, I've added it.
Wim Leers has a very interesting article on page loading performance: http://wimleers.com/article/improving-drupals-page-loading-performance
Thanks! Daniel.
http://drupal.org/project/admin_menu it gives you a dropdown menu on top with the ratio, but its actually not really neccesary because its again an extra module and you probably need less modules. The core "Who's online" block should give you the same data, enable it and make it visibly only for your role (make an admin role for yourself if you would not have done this yet). Hans
The building of the cache takes a bit of time and consumes CPU also of
course. It also all depends whats your ratio of logged in and anonymous users (you can see this easily in admin menu). If your problem with CPU comes from a lot of logged in users
Where can I find this ratio? The best I've found is the list of top visitors in the past 3 days. It looks like anonymous visitors dominate.
You could try installing apc cache which will reduce the amount of cpu time spent rendering php files, which should have an impact on cpu. if you have enough RAM. steev On Wed, May 13, 2009 at 1:31 PM, KOBA | Hans Rossel <hans.rossel@koba.be>wrote:
http://drupal.org/project/admin_menu it gives you a dropdown menu on top with the ratio, but its actually not really neccesary because its again an extra module and you probably need less modules. The core "Who's online" block should give you the same data, enable it and make it visibly only for your role (make an admin role for yourself if you would not have done this yet).
Hans
The building of the cache takes a bit of time and consumes CPU also of
course. It also all depends whats your ratio of logged in and anonymous users (you can see this easily in admin menu). If your problem with CPU comes from a lot of logged in users
Where can I find this ratio? The best I've found is the list of top visitors in the past 3 days. It looks like anonymous visitors dominate.
-- [ Drupal support list | http://lists.drupal.org/ ]
-- -- -- Steve Power Principal Consultant Mobile: +44 (0) 7747 027 243 Initsix Technology and Media --
I once asked the web host to install the APC cache and my Drupal site stopped working entirely :-( I can't afford to have the site go down at this time. This is the end of the school year, which is the most important time for us. I'm really interested in that Boost module that Hans pointed out. I'm just trying to learn more about it right now. Daniel. Steve Power wrote:
You could try installing apc cache which will reduce the amount of cpu time spent rendering php files, which should have an impact on cpu. if you have enough RAM.
steev
On Wed, May 13, 2009 at 1:31 PM, KOBA | Hans Rossel <hans.rossel@koba.be <mailto:hans.rossel@koba.be>> wrote:
http://drupal.org/project/admin_menu
it gives you a dropdown menu on top with the ratio, but its actually not really neccesary because its again an extra module and you probably need less modules. The core "Who's online" block should give you the same data, enable it and make it visibly only for your role (make an admin role for yourself if you would not have done this yet).
Hans
The building of the cache takes a bit of time and consumes CPU also of course. It also all depends whats your ratio of logged in and anonymous users (you can see this easily in admin menu). If your problem with CPU comes from a lot of logged in users
Where can I find this ratio? The best I've found is the list of top visitors in the past 3 days. It looks like anonymous visitors dominate.
-- [ Drupal support list | http://lists.drupal.org/ ]
-- -- -- Steve Power Principal Consultant Mobile: +44 (0) 7747 027 243 Initsix Technology and Media --
------------------------------------------------------------------------
-- [ Drupal support list | http://lists.drupal.org/ ]
This is not good: :-( http://codesorcery.net/2007/07/23/boost-your-drupal-site "The second issue is that currently, Boost will not work for sites that are not at the top-level. That is, if your site is domain.com/mysite, it will not work -- only domain.com would work." This is bad for me because my Drupal site is not top-level. Daniel.
Do you have any specs re your hosting environment? i'd check fundamentals (cpu, ram, disk, other customers on the same box) before installing more modules. On Wed, May 13, 2009 at 2:26 PM, Daniel Carrera < daniel.carrera@theingots.org> wrote:
This is not good: :-(
http://codesorcery.net/2007/07/23/boost-your-drupal-site
"The second issue is that currently, Boost will not work for sites that are not at the top-level. That is, if your site is domain.com/mysite, it will not work -- only domain.com would work."
This is bad for me because my Drupal site is not top-level.
Daniel.
-- [ Drupal support list | http://lists.drupal.org/ ]
-- -- -- Steve Power Principal Consultant Mobile: +44 (0) 7747 027 243 Initsix Technology and Media --
CPU: Quad-Core AMD Opteron(tm) Processor 2350 RAM: 8GB The host is unitedhosting.co.uk which stands out because they do not over-sell the system resources like most shared hosts do (and hence, UH costs more too). I don't know how many users per box. They say that it's variable. Daniel. Steve Power wrote:
Do you have any specs re your hosting environment? i'd check fundamentals (cpu, ram, disk, other customers on the same box) before installing more modules.
On Wed, May 13, 2009 at 2:26 PM, Daniel Carrera <daniel.carrera@theingots.org <mailto:daniel.carrera@theingots.org>> wrote:
This is not good: :-(
http://codesorcery.net/2007/07/23/boost-your-drupal-site
"The second issue is that currently, Boost will not work for sites that are not at the top-level. That is, if your site is domain.com/mysite <http://domain.com/mysite>, it will not work -- only domain.com <http://domain.com> would work."
This is bad for me because my Drupal site is not top-level.
Daniel.
-- [ Drupal support list | http://lists.drupal.org/ ]
-- -- -- Steve Power Principal Consultant Mobile: +44 (0) 7747 027 243 Initsix Technology and Media --
------------------------------------------------------------------------
-- [ Drupal support list | http://lists.drupal.org/ ]
Blimey. I have the same spec (albeit EC2 instance), which is hosting 6 drupal sites, and can serve pages in under 1s with 50 concurrent users with LOTS of modules. erm, if i were a betting man, i would question what they are saying, against what you are seeing... i dont use these guys anymore, but their vps is excellent. www.bytemark.co.uk On Wed, May 13, 2009 at 2:58 PM, Daniel Carrera < daniel.carrera@theingots.org> wrote:
CPU: Quad-Core AMD Opteron(tm) Processor 2350 RAM: 8GB
The host is unitedhosting.co.uk which stands out because they do not over-sell the system resources like most shared hosts do (and hence, UH costs more too). I don't know how many users per box. They say that it's variable.
Daniel.
Steve Power wrote:
Do you have any specs re your hosting environment? i'd check fundamentals (cpu, ram, disk, other customers on the same box) before installing more modules.
On Wed, May 13, 2009 at 2:26 PM, Daniel Carrera < daniel.carrera@theingots.org <mailto:daniel.carrera@theingots.org>> wrote:
This is not good: :-(
http://codesorcery.net/2007/07/23/boost-your-drupal-site
"The second issue is that currently, Boost will not work for sites that are not at the top-level. That is, if your site is domain.com/mysite <http://domain.com/mysite>, it will not work -- only domain.com <http://domain.com> would work."
This is bad for me because my Drupal site is not top-level.
Daniel.
-- [ Drupal support list | http://lists.drupal.org/ ]
-- -- -- Steve Power Principal Consultant Mobile: +44 (0) 7747 027 243 Initsix Technology and Media --
------------------------------------------------------------------------
-- [ Drupal support list | http://lists.drupal.org/ ]
-- -- -- Steve Power Principal Consultant Mobile: +44 (0) 7747 027 243 Initsix Technology and Media --
apc is a particularly easy install, and practically zero config (well, six lines) with a potentially enormous performance increase. Its worth trying again if your hosting company is willing to work through the problem with you. Another way of looking at it could be that if your hosting company cant configure apc without taking the site down, then perhaps a new host is in order. steev On Wed, May 13, 2009 at 2:15 PM, Daniel Carrera < daniel.carrera@theingots.org> wrote:
I once asked the web host to install the APC cache and my Drupal site stopped working entirely :-( I can't afford to have the site go down at this time. This is the end of the school year, which is the most important time for us.
I'm really interested in that Boost module that Hans pointed out. I'm just trying to learn more about it right now.
Daniel.
Steve Power wrote:
You could try installing apc cache which will reduce the amount of cpu time spent rendering php files, which should have an impact on cpu. if you have enough RAM.
steev
On Wed, May 13, 2009 at 1:31 PM, KOBA | Hans Rossel <hans.rossel@koba.be<mailto: hans.rossel@koba.be>> wrote:
http://drupal.org/project/admin_menu
it gives you a dropdown menu on top with the ratio, but its actually not really neccesary because its again an extra module and you probably need less modules. The core "Who's online" block should give you the same data, enable it and make it visibly only for your role (make an admin role for yourself if you would not have done this yet).
Hans
The building of the cache takes a bit of time and consumes CPU also of course. It also all depends whats your ratio of logged in and anonymous users (you can see this easily in admin menu). If your problem with CPU comes from a lot of logged in users
Where can I find this ratio? The best I've found is the list of top visitors in the past 3 days. It looks like anonymous visitors dominate.
-- [ Drupal support list | http://lists.drupal.org/ ]
-- -- -- Steve Power Principal Consultant Mobile: +44 (0) 7747 027 243 Initsix Technology and Media --
------------------------------------------------------------------------
-- [ Drupal support list | http://lists.drupal.org/ ]
-- [ Drupal support list | http://lists.drupal.org/ ]
-- -- -- Steve Power Principal Consultant Mobile: +44 (0) 7747 027 243 Initsix Technology and Media --
It's not that they "take the site down", it's that Drupal really just dies when APC is installed. I forget what the error was. It was back in January when we tried APC and Drupal stopped working. Steve Power wrote:
apc is a particularly easy install, and practically zero config (well, six lines) with a potentially enormous performance increase. Its worth trying again if your hosting company is willing to work through the problem with you. Another way of looking at it could be that if your hosting company cant configure apc without taking the site down, then perhaps a new host is in order.
steev
On Wed, May 13, 2009 at 2:15 PM, Daniel Carrera <daniel.carrera@theingots.org <mailto:daniel.carrera@theingots.org>> wrote:
I once asked the web host to install the APC cache and my Drupal site stopped working entirely :-( I can't afford to have the site go down at this time. This is the end of the school year, which is the most important time for us.
I'm really interested in that Boost module that Hans pointed out. I'm just trying to learn more about it right now.
Daniel.
Steve Power wrote:
You could try installing apc cache which will reduce the amount of cpu time spent rendering php files, which should have an impact on cpu. if you have enough RAM.
steev
On Wed, May 13, 2009 at 1:31 PM, KOBA | Hans Rossel <hans.rossel@koba.be <mailto:hans.rossel@koba.be> <mailto:hans.rossel@koba.be <mailto:hans.rossel@koba.be>>> wrote:
http://drupal.org/project/admin_menu
it gives you a dropdown menu on top with the ratio, but its actually not really neccesary because its again an extra module and you probably need less modules. The core "Who's online" block should give you the same data, enable it and make it visibly only for your role (make an admin role for yourself if you would not have done this yet).
Hans
The building of the cache takes a bit of time and consumes CPU also of course. It also all depends whats your ratio of logged in and anonymous users (you can see this easily in admin menu). If your problem with CPU comes from a lot of logged in users
Where can I find this ratio? The best I've found is the list of top visitors in the past 3 days. It looks like anonymous visitors dominate.
-- [ Drupal support list | http://lists.drupal.org/ ]
-- -- -- Steve Power Principal Consultant Mobile: +44 (0) 7747 027 243 Initsix Technology and Media --
------------------------------------------------------------------------
-- [ Drupal support list | http://lists.drupal.org/ ]
-- [ Drupal support list | http://lists.drupal.org/ ]
-- -- -- Steve Power Principal Consultant Mobile: +44 (0) 7747 027 243 Initsix Technology and Media --
------------------------------------------------------------------------
-- [ Drupal support list | http://lists.drupal.org/ ]
And logging of users takes also CPU so I would disable Drupal statistics module and install Google Analytics (or any other external statistics just paste the code before </body>, dont use the Google Analytics module if you have performance problems).
Where can I find this ratio? The best I've found is the list of top visitors in the past 3 days. It looks like anonymous visitors dominate.
KOBA | Hans Rossel wrote:
And logging of users takes also CPU so I would disable Drupal statistics module and install Google Analytics (or any other external statistics just paste the code before </body>, dont use the Google Analytics module if you have performance problems).
Thanks. And what about that Boost module you mentioned? Are there any drawbacks I should be aware of? I also activated the Throttle module, but I didn't know what options to pick so I just left the default. Was this a good idea? Thanks for all the help. Daniel.
On Wed, May 13, 2009 at 6:17 AM, Daniel Carrera <daniel.carrera@theingots.org> wrote:
I can't. Shared host.
A cheap VPS is $20/month and will likely solve the problem. If nothing else, it will make it easier to determine where the problem is and open you up to more potential solutions (installing an opcode cache, tweaking apache/mysql configs). Debugging a slow site on shared hosting is an exercise in futility. Regardless of the value you place on your own time, it doesn't take much time troubleshooting before a low cost shared hosting plan becomes a false savings. Regards, Greg -- Greg Knaddison | 303-800-5623 | http://growingventuresolutions.com Cracking Drupal - Learn to protect your Drupal site from hackers Now available from Wiley http://crackingdrupal.com
Greg Knaddison wrote:
A cheap VPS is $20/month and will likely solve the problem.
I am not a very good sysadmin and I hate sys admin work. I cry at the thought of having to configure Mailman (I have no idea how) or spam filters, or Apache, PHP and MySQL, or handle backup and recovery. I speak from experience. I have been the sys admin of a non-critical Drupal site and I know I would not like to do the same for a mission critical site. I chose this host as a way to get a balance between getting my fair share of resources while having someone else do the day-to-day admin work that I have neither interest nor aptitude for.
Debugging a slow site on shared hosting is an exercise in futility. Regardless of the value you place on your own time, it doesn't take much time troubleshooting before a low cost shared hosting plan becomes a false savings.
I did not choose the present shared host based on savings. I chose them because they have the best reputation for excellent support and uptime that is second to none. So far they have lived up to their reputation. My support requests receive an intelligent and useful response in less than 2 or 3 minutes. My top priority is reliability. I don't have enough confidence in my admin skills to believe that I can achieve better reliability myself than I can with United Hosting. Daniel.
ask them to move you to a less busy machine? On Wed, May 13, 2009 at 3:18 PM, Daniel Carrera < daniel.carrera@theingots.org> wrote:
Greg Knaddison wrote:
A cheap VPS is $20/month and will likely solve the problem.
I am not a very good sysadmin and I hate sys admin work. I cry at the thought of having to configure Mailman (I have no idea how) or spam filters, or Apache, PHP and MySQL, or handle backup and recovery.
I speak from experience. I have been the sys admin of a non-critical Drupal site and I know I would not like to do the same for a mission critical site.
I chose this host as a way to get a balance between getting my fair share of resources while having someone else do the day-to-day admin work that I have neither interest nor aptitude for.
Debugging a slow site on shared hosting is an exercise in futility.
Regardless of the value you place on your own time, it doesn't take much time troubleshooting before a low cost shared hosting plan becomes a false savings.
I did not choose the present shared host based on savings. I chose them because they have the best reputation for excellent support and uptime that is second to none. So far they have lived up to their reputation. My support requests receive an intelligent and useful response in less than 2 or 3 minutes.
My top priority is reliability. I don't have enough confidence in my admin skills to believe that I can achieve better reliability myself than I can with United Hosting.
Daniel.
-- [ Drupal support list | http://lists.drupal.org/ ]
-- -- -- Steve Power Principal Consultant Mobile: +44 (0) 7747 027 243 Initsix Technology and Media --
It would help if I knew what the CPU usage actually was. Perhaps my site is just very badly configured (e.g. I didn't even have caching enabled). Also, the problem is not *technically* that I'm hitting a CPU quota. What happens is that we have a lot of requests coming from the same IP address (which is used by a number of schools, which are our customers). So the firewall at the server decides that this IP is mounting a DOS attack, so it blocks the IP, hence blocking our customers. I spoke with support, and they said if the CPU usage went down, it would reduce the chances of that school IP being blocked in the future. And that's how I ended up here asking about CPU usage. Cheers, Daniel. Steve Power wrote:
ask them to move you to a less busy machine?
On Wed, May 13, 2009 at 3:18 PM, Daniel Carrera <daniel.carrera@theingots.org <mailto:daniel.carrera@theingots.org>> wrote:
Greg Knaddison wrote:
A cheap VPS is $20/month and will likely solve the problem.
I am not a very good sysadmin and I hate sys admin work. I cry at the thought of having to configure Mailman (I have no idea how) or spam filters, or Apache, PHP and MySQL, or handle backup and recovery.
I speak from experience. I have been the sys admin of a non-critical Drupal site and I know I would not like to do the same for a mission critical site.
I chose this host as a way to get a balance between getting my fair share of resources while having someone else do the day-to-day admin work that I have neither interest nor aptitude for.
Debugging a slow site on shared hosting is an exercise in futility. Regardless of the value you place on your own time, it doesn't take much time troubleshooting before a low cost shared hosting plan becomes a false savings.
I did not choose the present shared host based on savings. I chose them because they have the best reputation for excellent support and uptime that is second to none. So far they have lived up to their reputation. My support requests receive an intelligent and useful response in less than 2 or 3 minutes.
My top priority is reliability. I don't have enough confidence in my admin skills to believe that I can achieve better reliability myself than I can with United Hosting.
Daniel.
-- [ Drupal support list | http://lists.drupal.org/ ]
-- -- -- Steve Power Principal Consultant Mobile: +44 (0) 7747 027 243 Initsix Technology and Media --
------------------------------------------------------------------------
-- [ Drupal support list | http://lists.drupal.org/ ]
Why not just whitelist the ip and be done with it? Sent from my iPhone. Please excuse any grammatical or spelling errors. On May 13, 2009, at 8:23 AM, Daniel Carrera <daniel.carrera@theingots.org
wrote:
It would help if I knew what the CPU usage actually was. Perhaps my site is just very badly configured (e.g. I didn't even have caching enabled).
Also, the problem is not *technically* that I'm hitting a CPU quota. What happens is that we have a lot of requests coming from the same IP address (which is used by a number of schools, which are our customers). So the firewall at the server decides that this IP is mounting a DOS attack, so it blocks the IP, hence blocking our customers.
I spoke with support, and they said if the CPU usage went down, it would reduce the chances of that school IP being blocked in the future. And that's how I ended up here asking about CPU usage.
Cheers, Daniel.
Steve Power wrote:
ask them to move you to a less busy machine? On Wed, May 13, 2009 at 3:18 PM, Daniel Carrera <daniel.carrera@theingots.org <mailto:daniel.carrera@theingots.org>> wrote: Greg Knaddison wrote: A cheap VPS is $20/month and will likely solve the problem. I am not a very good sysadmin and I hate sys admin work. I cry at the thought of having to configure Mailman (I have no idea how) or spam filters, or Apache, PHP and MySQL, or handle backup and recovery. I speak from experience. I have been the sys admin of a non- critical Drupal site and I know I would not like to do the same for a mission critical site. I chose this host as a way to get a balance between getting my fair share of resources while having someone else do the day-to-day admin work that I have neither interest nor aptitude for. Debugging a slow site on shared hosting is an exercise in futility. Regardless of the value you place on your own time, it doesn't take much time troubleshooting before a low cost shared hosting plan becomes a false savings. I did not choose the present shared host based on savings. I chose them because they have the best reputation for excellent support and uptime that is second to none. So far they have lived up to their reputation. My support requests receive an intelligent and useful response in less than 2 or 3 minutes. My top priority is reliability. I don't have enough confidence in my admin skills to believe that I can achieve better reliability myself than I can with United Hosting. Daniel. -- [ Drupal support list | http://lists.drupal.org/ ] -- -- -- Steve Power Principal Consultant Mobile: +44 (0) 7747 027 243 Initsix Technology and Media -- --- --------------------------------------------------------------------- -- [ Drupal support list | http://lists.drupal.org/ ]
-- [ Drupal support list | http://lists.drupal.org/ ]
Yup. Same here. Sounds like shennanigans. On Wed, May 13, 2009 at 4:42 PM, Joshua Rogers <me@joshuarogers.net> wrote:
I spoke with support, and they said if the CPU usage went down, it would reduce the chances of that school IP being blocked in the future. And that's how I ended up here asking about CPU usage.
That doesn't sound like a firewall. That sounds like trying to sale a bigger package. -- [ Drupal support list | http://lists.drupal.org/ ]
-- -- -- Steve Power Principal Consultant Mobile: +44 (0) 7747 027 243 Initsix Technology and Media --
Steve Power wrote:
Yup. Same here. Sounds like shennanigans.
What do you recommend? I spent a lot of time researching hosts before I selected this one. And United Hosting has glowing reviews spanning several years. I don't want a VPS because I have neither skill nor interest in sys admin tasks. Btw, they have not suggested that I upgrade, or suggested that an upgrade might help. In fact, what they did is try to suggest ways I might lower the CPU consumption of my site. Daniel.
If this is a serious site I would recommend a managed vps / managed server. The amount of things that you will be able to tweak to work with YOUR site will allow your site to run much better. You might want to start migrating your site to a host who knows how to manage a network. Auto blacklisting legitimate customers is a sure sign of mismanagement. If you are making significant money from your site a host like Rackspace: http://www.rackspace.com/solutions/managed_hosting/configurations/index.php Is 100% solid and will do anything and everything you need. Yes they are expensive, but they have the knowledge and experience to charge that fairly. -----Original Message----- From: support-bounces@drupal.org [mailto:support-bounces@drupal.org] On Behalf Of Daniel Carrera Sent: Wednesday, May 13, 2009 10:25 AM To: support@drupal.org Subject: Re: [support] CPU usage Steve Power wrote:
Yup. Same here. Sounds like shennanigans.
What do you recommend? I spent a lot of time researching hosts before I selected this one. And United Hosting has glowing reviews spanning several years. I don't want a VPS because I have neither skill nor interest in sys admin tasks. Btw, they have not suggested that I upgrade, or suggested that an upgrade might help. In fact, what they did is try to suggest ways I might lower the CPU consumption of my site. Daniel. -- [ Drupal support list | http://lists.drupal.org/ ]
Boris Berenberg wrote:
If this is a serious site I would recommend a managed vps / managed server.
A *managed* VPS sounds like a good idea. The flexibility of a VPS, with a competent sys admin to make the system actually work. But I'm not sure we can afford that. I think we can reasonably spend as much as $100/month if the system is worth it. I would, however, prefer to keep the cost closer to $50/month if that can be done. Looking at Rackspace, I don't know if that $399 is per month or per year (I don't know if its a real server or just a VPS). But I think that $399/month is probably outside our budget right now.
If you are making significant money from your site a host like Rackspace: http://www.rackspace.com/solutions/managed_hosting/configurations/index.php Is 100% solid and will do anything and everything you need. Yes they are expensive, but they have the knowledge and experience to charge that fairly.
Do you use them? How much experience do you have with them? We are not making "significant money" but the website is central to the business and it is worth spending extra money to get a good service. I am a bit hesitate to migrate right now. We are close to the end of the school year and that's our busiest period. On the one hand, that means that we really don't want customers getting blocked right now. On the other, I wouldn't want any down time due to a server move. Do you know if Rackspace has anything in our budget range? Can you point me to reviews of Rackspace? Generally I have been very satisfied with United Hosting. This is the first time we have a problem like this and generally they are very professional and responsive. That said, I don't feel any loyalty to them and I'm willing to change if it'll make things better. But then again, it is always a bit hard to move because you don't really know what you'll get on the new host. Will it really be better? Might you have a different set of issues? Maybe the new host will take a day to reply to an email, or something... Anyways, I'm willing to investigate Rackspace. Daniel.
Daniel Carrera wrote:
Generally I have been very satisfied with United Hosting. This is the first time we have a problem like this and generally they are very professional and responsive. That said, I don't feel any loyalty to them and I'm willing to change if it'll make things better. But then again, it is always a bit hard to move because you don't really know what you'll get on the new host. Will it really be better? Might you have a different set of issues? Maybe the new host will take a day to reply to an email, or something...
United Hosting does four backups a day, so I have a lot of granularity when selecting a backup ("last Tuesday, 10AM backup"). I've only been with this company since January and I've already used this feature twice. I don't imagine that Rackspace can do the same... Daniel.
Btw, we currently use Mailfoundry as our spam filter, and Mailfoundry also has problems red-listing the IP of the school in question. It's the same reason - a large number of emails originating from this IP are interpreted as spam. How would you interpret this? Does the fact that Mailfoundry also blocks the exact same IP mean anything? It seems odd that both services would see a problem with this IP sending a lot of queries and decide to red-list it. Daniel. Daniel Carrera wrote:
Daniel Carrera wrote:
Generally I have been very satisfied with United Hosting. This is the first time we have a problem like this and generally they are very professional and responsive. That said, I don't feel any loyalty to them and I'm willing to change if it'll make things better. But then again, it is always a bit hard to move because you don't really know what you'll get on the new host. Will it really be better? Might you have a different set of issues? Maybe the new host will take a day to reply to an email, or something...
United Hosting does four backups a day, so I have a lot of granularity when selecting a backup ("last Tuesday, 10AM backup"). I've only been with this company since January and I've already used this feature twice. I don't imagine that Rackspace can do the same...
Daniel. -- [ Drupal support list | http://lists.drupal.org/ ]
Quoting Daniel Carrera <daniel.carrera@theingots.org>:
Btw, we currently use Mailfoundry as our spam filter, and Mailfoundry also has problems red-listing the IP of the school in question. It's the same reason - a large number of emails originating from this IP are interpreted as spam.
Is there some means for you to verify that it isn't SPAM? Is it possible to white list the IP address? Are there controls at the school to prevent its servers from being used by external users/servers for Relay, etc.? If the school's servers aren't sound then they should get that fixed. If someone in the school is abusing the schools services they should get that fixed. -- Earnie
This almost sounds like a pre-fetching proxy being used by the school, such as Blue Coat. Those can be a real nightmare and are common at creating these sorts of problems. Just for an idea of what I am talking about, here's a post where a Wordpress site had a similar problem: http://www.homelandstupidity.us/2006/02/15/defense-contractor-attacks-homela... Blue Coat is common in government and educational institutions, as well as big corporations. Jamie Holly Earnie Boyd wrote:
Quoting Daniel Carrera <daniel.carrera@theingots.org>:
Btw, we currently use Mailfoundry as our spam filter, and Mailfoundry also has problems red-listing the IP of the school in question. It's the same reason - a large number of emails originating from this IP are interpreted as spam.
Is there some means for you to verify that it isn't SPAM? Is it possible to white list the IP address? Are there controls at the school to prevent its servers from being used by external users/servers for Relay, etc.? If the school's servers aren't sound then they should get that fixed. If someone in the school is abusing the schools services they should get that fixed.
-- Earnie
-- [ Drupal support list | http://lists.drupal.org/ ]
Hhhmm... that's interesting. I'm pretty sure that the school has a proxy but I don't know which one. I just sent an email to the school admin asking about that. Thanks. Daniel. Jamie Holly wrote:
This almost sounds like a pre-fetching proxy being used by the school, such as Blue Coat. Those can be a real nightmare and are common at creating these sorts of problems. Just for an idea of what I am talking about, here's a post where a Wordpress site had a similar problem:
http://www.homelandstupidity.us/2006/02/15/defense-contractor-attacks-homela...
Blue Coat is common in government and educational institutions, as well as big corporations.
Jamie Holly
Earnie Boyd wrote:
Quoting Daniel Carrera <daniel.carrera@theingots.org>:
Btw, we currently use Mailfoundry as our spam filter, and > Mailfoundry also has problems red-listing the IP of the school in > question. It's the same reason - a large number of emails > originating from this IP are interpreted as spam.
Is there some means for you to verify that it isn't SPAM? Is it possible to white list the IP address? Are there controls at the school to prevent its servers from being used by external users/servers for Relay, etc.? If the school's servers aren't sound then they should get that fixed. If someone in the school is abusing the schools services they should get that fixed.
-- Earnie
-- [ Drupal support list | http://lists.drupal.org/ ]
-- [ Drupal support list | http://lists.drupal.org/ ]
Earnie Boyd wrote:
Is there some means for you to verify that it isn't SPAM? Is it possible to white list the IP address?
I haven't thought of any way to verify that it isn't SPAM. :-( The web host has said that it is not possible to white-list the IP address. I did send an email to the school admin with other questions, like what proxy they use and how many schools share this IP.
Are there controls at the school to prevent its servers from being used by external users/servers for Relay, etc.?
That will be my next question to the school admin. Thanks for suggesting it.
If the school's servers aren't sound then they should get that fixed. If someone in the school is abusing the schools services they should get that fixed.
Indeed. Daniel.
Ok, I have a little more information: All the schools in Kent county, UK (pop 1.4 mil) go through the same IP address. So that's may thousand PCs. Most schools "have an ISA server on site running Websense". Websense appears to be some sort of gateway software, and I imagine that ISA is the Microsoft "Internet Security and Acceleration" Server. I have no idea if either of these products has the problems of Blue Coat. Daniel.
I know Websense does pretty much the same thing as Blue Coat in "pre scanning" sites. The best solution is to use css compression/aggregation (in D5+) and javascript compression/aggregation (D6) to reduce requests. Those are some of the files these services latch onto, and when they read the entire site they won't cache the files, but rather reread them on every page request.This will help keep the cpu usage down since Apache won't have to keep spawning connections for the files. Also any kind of caching you can get in there. There are some scenarios and modules it won't work with. For example - if your Drupal install is in a subfolder of the web root. One way to fix that is to put Drupal on its own subdomain. Also trying op-code level caching like APC is a big plus (generally a 20%+ increase). Another option is using something like cacherouter with memcache and then enabling the page caching through that. With apc+cache router+memcache, I can pump out over 200 requests per second on an old dual-core AMD server and my load never goes above 60 using D6. Of course if you are on shared hosting getting these things setup can be very difficult, if not impossible (APC and Memcache require a system admin to install and configure the modules). I would then look at going to a managed VPS (if you aren't comfortable handling your own system admin). Companies like wired tree have these services starting at around $40 (us) a month. Jamie Holly Daniel Carrera wrote:
Ok, I have a little more information:
All the schools in Kent county, UK (pop 1.4 mil) go through the same IP address. So that's may thousand PCs. Most schools "have an ISA server on site running Websense". Websense appears to be some sort of gateway software, and I imagine that ISA is the Microsoft "Internet Security and Acceleration" Server. I have no idea if either of these products has the problems of Blue Coat.
Daniel. -- [ Drupal support list | http://lists.drupal.org/ ]
Hi all, I have new information. At the time the school was blocked, this IP was taking up 500 HTTP connections and the server is configured to support 1000. So this one IP was taking up half of the HTTP connections. 500 connections seems absurd, and so I'm convinced that there's something wrong at the school. The best hypothesis so far is the pre-scanning thing (thanks Jamie). That said, I would like to better understand what "500 HTTP connections" means. If you get a single web page that has one HTML file, two images and two CSS files, is that one HTTP connection or five? In other words, does "500 HTTP connections" equate to 500 users or 100? Also, I have no idea if 1000 HTTP connections is a reasonable configuration or not. Thanks. Daniel. Jamie Holly wrote:
I know Websense does pretty much the same thing as Blue Coat in "pre scanning" sites. The best solution is to use css compression/aggregation (in D5+) and javascript compression/aggregation (D6) to reduce requests. Those are some of the files these services latch onto, and when they read the entire site they won't cache the files, but rather reread them on every page request.This will help keep the cpu usage down since Apache won't have to keep spawning connections for the files.
Also any kind of caching you can get in there. There are some scenarios and modules it won't work with. For example - if your Drupal install is in a subfolder of the web root. One way to fix that is to put Drupal on its own subdomain. Also trying op-code level caching like APC is a big plus (generally a 20%+ increase). Another option is using something like cacherouter with memcache and then enabling the page caching through that. With apc+cache router+memcache, I can pump out over 200 requests per second on an old dual-core AMD server and my load never goes above 60 using D6.
If you're servicing all the schools in kent, then maybe the figures are real. But tbh, you're barking up the wrong tree. Dont take this the wrong way, but you need to run this like a popular e-commerce site, not a hobby site. If you have thousands of users then shared hosting is no longer appropriate and just wont scale as their business model relies on lots of inactive websites. Even if you could move hosts, its not really going help however you configure Drupal as the problem seems to be at an edge traffic shaping device. Thats only my opinion tho. On Thu, May 14, 2009 at 6:51 PM, Daniel Carrera < daniel.carrera@theingots.org> wrote:
Hi all,
I have new information. At the time the school was blocked, this IP was taking up 500 HTTP connections and the server is configured to support 1000. So this one IP was taking up half of the HTTP connections.
500 connections seems absurd, and so I'm convinced that there's something wrong at the school. The best hypothesis so far is the pre-scanning thing (thanks Jamie).
That said, I would like to better understand what "500 HTTP connections" means. If you get a single web page that has one HTML file, two images and two CSS files, is that one HTTP connection or five? In other words, does "500 HTTP connections" equate to 500 users or 100?
Also, I have no idea if 1000 HTTP connections is a reasonable configuration or not.
Thanks. Daniel.
Jamie Holly wrote:
I know Websense does pretty much the same thing as Blue Coat in "pre scanning" sites. The best solution is to use css compression/aggregation (in D5+) and javascript compression/aggregation (D6) to reduce requests. Those are some of the files these services latch onto, and when they read the entire site they won't cache the files, but rather reread them on every page request.This will help keep the cpu usage down since Apache won't have to keep spawning connections for the files.
Also any kind of caching you can get in there. There are some scenarios and modules it won't work with. For example - if your Drupal install is in a subfolder of the web root. One way to fix that is to put Drupal on its own subdomain. Also trying op-code level caching like APC is a big plus (generally a 20%+ increase). Another option is using something like cacherouter with memcache and then enabling the page caching through that. With apc+cache router+memcache, I can pump out over 200 requests per second on an old dual-core AMD server and my load never goes above 60 using D6.
-- [ Drupal support list | http://lists.drupal.org/ ]
-- -- -- Steve Power Principal Consultant Mobile: +44 (0) 7747 027 243 Initsix Technology and Media --
Steve Power wrote:
If you're servicing all the schools in kent, then maybe the figures are real.
I'm not really. I don't know how many schools we have in Kent. For all I know, it could just be this one school. For one school, I would expect at *MOST* 100 simultaneous users, but more likely 30 (one classroom). How many HTTP connections would 30 kids using a website generate? I have no real sense of proportion here...
But tbh, you're barking up the wrong tree. Dont take this the wrong way, but you need to run this like a popular e-commerce site, not a hobby site.
We would not have moved to UH if we saw this as a hobby site. In any case, I do expect to scale up the service as our needs increase. I just would like to have a better understanding of what our needs actually are (e.g. is 500 connections reasonable?).
If you have thousands of users then shared hosting is no longer appropriate and just wont scale as their business model relies on lots of inactive websites.
I think we have about 2,000 users on the Drupal site or so. The large majority are inactive. Would you guess that kind of site merits a dedicated server? As I've said earlier, UH does not over-sell their resources (e.g. disk space and bandwidth). So the comment about their business model, while not wrong, is less true for this particular host than you might think.
Even if you could move hosts, its not really going help however you configure Drupal as the problem seems to be at an edge traffic shaping device. Thats only my opinion tho.
Thank you for sharing your thoughts. Daniel.
heys, i didnt mean to offend. For a dedicated server, you will be able to deliver a much better user experience as the performance will deteriorate only in accordance with your load which you can monitor and plan for. On a shared host you are at the mercy of a shared hosting company. You prob dont need a dedicated host, can get away with a good vps, but if you've got 30 users hammering the server for 30 minutes then you have spikes to deal with, which was the bane of web1.0. there are more options to deal with this now like automating ec2 instances to come up and down dependant on demand, but they are nothing to do with drupal. they are infrastructure engineering issues, which is what i was trying to say. It seems to me that the performance issue is an infrastructure, not an application, performance issue. Sorry if i came across badly. been up all day! steev On Thu, May 14, 2009 at 7:40 PM, Daniel Carrera < daniel.carrera@theingots.org> wrote:
Steve Power wrote:
If you're servicing all the schools in kent, then maybe the figures are real.
I'm not really. I don't know how many schools we have in Kent. For all I know, it could just be this one school. For one school, I would expect at *MOST* 100 simultaneous users, but more likely 30 (one classroom).
How many HTTP connections would 30 kids using a website generate? I have no real sense of proportion here...
But tbh, you're barking up the wrong tree. Dont take this the wrong way,
but you need to run this like a popular e-commerce site, not a hobby site.
We would not have moved to UH if we saw this as a hobby site. In any case, I do expect to scale up the service as our needs increase. I just would like to have a better understanding of what our needs actually are (e.g. is 500 connections reasonable?).
If you have thousands of users then shared hosting is no longer
appropriate and just wont scale as their business model relies on lots of inactive websites.
I think we have about 2,000 users on the Drupal site or so. The large majority are inactive. Would you guess that kind of site merits a dedicated server?
As I've said earlier, UH does not over-sell their resources (e.g. disk space and bandwidth). So the comment about their business model, while not wrong, is less true for this particular host than you might think.
Even if you could move hosts, its not really going help however you
configure Drupal as the problem seems to be at an edge traffic shaping device. Thats only my opinion tho.
Thank you for sharing your thoughts.
Daniel.
-- [ Drupal support list | http://lists.drupal.org/ ]
-- -- -- Steve Power Principal Consultant Mobile: +44 (0) 7747 027 243 Initsix Technology and Media --
Steve Power wrote:
heys, i didnt mean to offend.
I wasn't offended. Sorry if I gave that impression. Email is really bad at conveying emotion, you know. Thank you for taking the time to help me. I appreciate your input.
You prob dont need a dedicated host, can get away with a good vps, but if you've got 30 users hammering the server for 30 minutes then you have spikes to deal with, which was the bane of web1.0.
Can you recommend a managed VPS that might be closer to my preferred price range? If we get something unmanaged we are likely to run into a whole other different set of problems.
It seems to me that the performance issue is an infrastructure, not an application, performance issue.
I took a closer look at the site and it appears that loading the front page requires in the order of 30 HTTP requests. That seems like way too much. Do you still feel its all infrastructure, or should we blame the website a bit too? Daniel.
Hello all. First of all, I'd like to thank everyone or their help so far. My website is likely to be very low traffic most of the time, but with a few sharp spikes. Most of the year it's quiet, but close to the end of the school year it gets much busier. Most of the day it is quiet, but when one classroom starts using the site, I have 30 new users. Is this site a good candidate for cloud computing, such as Amazon's EC2 service? My very limited understanding of EC2 is that it is "pay per use", so when the site is quiet we pay little and when there's a spike the service expands and we pay more. Is that correct? Is that how cloud computing works? Do you think that the cost of EC2 for us might be somewhere in between a shared host and a dedicated host? I was thinking that EC2 or similar might be a way for us to bridge the gap between a shared host and a dedicated host. But I'm not sure. Thanks again. Daniel.
Daniel Carrera wrote:
That said, I would like to better understand what "500 HTTP connections" means. If you get a single web page that has one HTML file, two images and two CSS files, is that one HTTP connection or five? In other words, does "500 HTTP connections" equate to 500 users or 100?
Each request to the server is considered a connection. So if you view the front page and you have the index file (index.php), 3 CSS files, 4 JS files and 10 images that would amount to 18 connections. That's why things like CSS and JS aggregation/compression help out so much. Also other tuning methods when it comes to themes, such as using CSS sprites to handle most of the images. Op-code caching can help a lot in this situation. Even if 10% of those connections are only for the actual Drupal page, that's still 50 hits at once. That also means 50 times that the server has to read in all the source files that makes up Drupal, then compile them over to the op-code that computers understand and execute it.
Also, I have no idea if 1000 HTTP connections is a reasonable configuration or not.
That's one of the biggest variables their is when it comes to server tuning. For shared hosting a setting that high is about par, though it isn't optimal. Yeah they are offering server side scripting like PHP, but they are tuned more to being a straight static server and can handle some server side requests in there. Basically its the standard operation of hosting companies when it comes to shared packages. Hosting companies do keep those settings high so they can find out who is taking up the most resources on a server. Then they have a selling point for getting people to upgrade (chalk that up as one of the things I learned working for a hosting company years ago). If the server is starting to choke at 500 connections, then ideally it would be tuned to a smaller portion of that. I would say around 300-400 connections. You also need to keep in mind things like MySQL running on the same server. Sure it can operate at 500 99% of the time, but if you end up getting fragmented tables or the query cache isn't primed then MySQL is going to need some processing power (as well as dozens of other things that run on a typical server) also and that will reduce the number of connections, so its always best to set the maximum connections on Apache to somewhere between 60 and 80% of what you see it handling, or do further benchmarking that alters site content and will invalidate things like query cache. Overall though I would say you are at the point shared hosting isn't going to cut it anymore. I doubt they will change to much to accommodate your site since they also have to consider the other sites on the machine. I would start looking at going to some other hosting solution, be it dedicated, VPS or cloud. Jamie Holly
Hi Jamie, Thanks for all the information. Jamie Holly wrote:
Overall though I would say you are at the point shared hosting isn't going to cut it anymore. I doubt they will change to much to accommodate your site since they also have to consider the other sites on the machine. I would start looking at going to some other hosting solution, be it dedicated, VPS or cloud.
Three questions: 1. What's cloud? 2. Does a VPS avoid the problems of a shared host? 3. Can these option handle the 500 HTTP requests? 4. Can you recommend a service for a *managed* VPS or similar that has very good support for a modest price? At the current exchange rate, we are spending about $35/month. We can up this a lot, but I think $400/month is out of our range right now. Daniel.
Daniel Carrera wrote:
Three questions:
1. What's cloud?
A cloud is basically a webserver that will grow as your demand grow. A good example of that is Amazon's EC2 service: http://en.wikipedia.org/wiki/Cloud_computing
2. Does a VPS avoid the problems of a shared host?
Yeah - with a VPS you have more control over the server. You can tune it to handle the connections your software/system combination can handle. Every platform is basically different. Wordpress would lead to a different number of maximum connections than Drupal. It's basically all a part of server tuning. For some good articles on server tuning with Drupal, check out 2bits: http://2bits.com/articles/drupal-performance-tuning-and-optimization-for-lar...
3. Can these option handle the 500 HTTP requests?
There's no way to answer that. You have way too many variables in there - CPU, memory, bus speed, drive speed, drive configuration, network latency, operating system, what is running on the operating system, etc., etc., etc. That's why when sites grow beyond what shared hosting can offer they need to move to their own servers where they can actually tune it themselves (or have a managed service with a knowledgeable staff that can do it for you).
4. Can you recommend a service for a *managed* VPS or similar that has very good support for a modest price?
At the current exchange rate, we are spending about $35/month. We can up this a lot, but I think $400/month is out of our range right now.
Not really. I pretty much recommend people go with a bigger service like RackSpace, which has a very solid reputation, but also comes with a steep price tag (of course the old saying "you get what you pay for" comes into play here). One suggestion would be to go with an unmanaged host and look at bringing in another company to handle your tuning of the site and whatever management you might need (companies such as Acquia or 2bits). That might be a more cost effective route overall. Jamie Holly
Jamie Holly wrote:
Not really. I pretty much recommend people go with a bigger service like RackSpace, which has a very solid reputation, but also comes with a steep price tag (of course the old saying "you get what you pay for" comes into play here).
How about Mosso? (www.mosso.com) A cloud service owned by RackSpace. But if I understand their pricing correctly, the service starts at $350/mo. I hope I'm wrong. If you look at http://www.mosso.com/cloud.jsp it says: "Step Two: Watch it Scale ... * Scale your bandwidth Starts at 500GB/mo. As much as you need for 25¢ per GB * Scale your storage Starts at 50GB/mo. As much as you need for 50¢ per GB * Scale your compute Starts at 10,000 compute cycles/mo. As many as you need for 1¢ per compute cycle" So we have a minimum of: 500*0.25 + 50*0.25 + 10000*0.01 = $350/mo. :-( Honestly, our site would do fine with 5GB of disk space and 10GB of bandwidth. Daniel.
EC2 can be viewed in the same way as a dedicated host in terms of management. i.e. you have to be able to run the servers and OS like you would a dedicated host. bytemark.co.uk are the only people i can recommend for a vps, but again once you start paying for engineers to do stuff, you're looking at similar prices to rackspace. On Fri, May 15, 2009 at 11:21 AM, Daniel Carrera < daniel.carrera@theingots.org> wrote:
Jamie Holly wrote:
Not really. I pretty much recommend people go with a bigger service like RackSpace, which has a very solid reputation, but also comes with a steep price tag (of course the old saying "you get what you pay for" comes into play here).
How about Mosso? (www.mosso.com) A cloud service owned by RackSpace. But if I understand their pricing correctly, the service starts at $350/mo. I hope I'm wrong. If you look at http://www.mosso.com/cloud.jsp it says:
"Step Two: Watch it Scale ... * Scale your bandwidth Starts at 500GB/mo. As much as you need for 25¢ per GB * Scale your storage Starts at 50GB/mo. As much as you need for 50¢ per GB * Scale your compute Starts at 10,000 compute cycles/mo. As many as you need for 1¢ per compute cycle"
So we have a minimum of: 500*0.25 + 50*0.25 + 10000*0.01 = $350/mo.
:-(
Honestly, our site would do fine with 5GB of disk space and 10GB of bandwidth.
Daniel.
-- [ Drupal support list | http://lists.drupal.org/ ]
-- -- -- Steve Power Principal Consultant Mobile: +44 (0) 7747 027 243 Initsix Technology and Media --
you could always try acquia On Fri, May 15, 2009 at 11:29 AM, Steve Power <steev@initsix.co.uk> wrote:
EC2 can be viewed in the same way as a dedicated host in terms of management. i.e. you have to be able to run the servers and OS like you would a dedicated host. bytemark.co.uk are the only people i can recommend for a vps, but again once you start paying for engineers to do stuff, you're looking at similar prices to rackspace.
On Fri, May 15, 2009 at 11:21 AM, Daniel Carrera < daniel.carrera@theingots.org> wrote:
Jamie Holly wrote:
Not really. I pretty much recommend people go with a bigger service like RackSpace, which has a very solid reputation, but also comes with a steep price tag (of course the old saying "you get what you pay for" comes into play here).
How about Mosso? (www.mosso.com) A cloud service owned by RackSpace. But if I understand their pricing correctly, the service starts at $350/mo. I hope I'm wrong. If you look at http://www.mosso.com/cloud.jsp it says:
"Step Two: Watch it Scale ... * Scale your bandwidth Starts at 500GB/mo. As much as you need for 25¢ per GB * Scale your storage Starts at 50GB/mo. As much as you need for 50¢ per GB * Scale your compute Starts at 10,000 compute cycles/mo. As many as you need for 1¢ per compute cycle"
So we have a minimum of: 500*0.25 + 50*0.25 + 10000*0.01 = $350/mo.
:-(
Honestly, our site would do fine with 5GB of disk space and 10GB of bandwidth.
Daniel.
-- [ Drupal support list | http://lists.drupal.org/ ]
-- -- -- Steve Power Principal Consultant Mobile: +44 (0) 7747 027 243 Initsix Technology and Media --
-- -- -- Steve Power Principal Consultant Mobile: +44 (0) 7747 027 243 Initsix Technology and Media --
See http://acquia.com/ Specifically, they provide a commercially supported Drupal distribution of Drupal core and contributed modules, as well as monitoring, spam reduction, search and other special services. Victor Kane http://awebfactory.com.ar http://projectflowandtracker.com On Fri, May 15, 2009 at 7:38 AM, Daniel Carrera <daniel.carrera@theingots.org> wrote:
Steve Power wrote:
you could always try acquia
What is acquia? Sorry if you already told me and I missed it. -- [ Drupal support list | http://lists.drupal.org/ ]
Victor Kane wrote:
Specifically, they provide a commercially supported Drupal distribution of Drupal core and contributed modules, as well as monitoring, spam reduction, search and other special services.
Ok. But they don't provide hosting, right? Thanks. Daniel.
I think they will be. There are several plans in various companies for special Drupal hosting. On Fri, May 15, 2009 at 7:59 AM, Daniel Carrera <daniel.carrera@theingots.org> wrote:
Victor Kane wrote:
Specifically, they provide a commercially supported Drupal distribution of Drupal core and contributed modules, as well as monitoring, spam reduction, search and other special services.
Ok. But they don't provide hosting, right?
Thanks. Daniel. -- [ Drupal support list | http://lists.drupal.org/ ]
Regarding Mosso, I'll understand their basic plan will fit you ? It's 50GB HD, 500GB Bandwith and 10000 compute cycles for $100, so it looks good for your usage ? Regarding Acquia, they will provide hosting later this year with Fields and Garden, see http://acquia.com/community/projects/acquia-2009-roadmap Bests ----- Original Message ----- From: "Daniel Carrera" <daniel.carrera@theingots.org> To: <support@drupal.org> Sent: Friday, May 15, 2009 5:21 AM Subject: Re: [support] CPU usage Jamie Holly wrote:
Not really. I pretty much recommend people go with a bigger service like RackSpace, which has a very solid reputation, but also comes with a steep price tag (of course the old saying "you get what you pay for" comes into play here).
How about Mosso? (www.mosso.com) A cloud service owned by RackSpace. But if I understand their pricing correctly, the service starts at $350/mo. I hope I'm wrong. If you look at http://www.mosso.com/cloud.jsp it says: "Step Two: Watch it Scale ... * Scale your bandwidth Starts at 500GB/mo. As much as you need for 25¢ per GB * Scale your storage Starts at 50GB/mo. As much as you need for 50¢ per GB * Scale your compute Starts at 10,000 compute cycles/mo. As many as you need for 1¢ per compute cycle" So we have a minimum of: 500*0.25 + 50*0.25 + 10000*0.01 = $350/mo. :-( Honestly, our site would do fine with 5GB of disk space and 10GB of bandwidth. Daniel. -- [ Drupal support list | http://lists.drupal.org/ ]
Nicolas Tostin wrote:
Regarding Mosso, I'll understand their basic plan will fit you ? It's 50GB HD, 500GB Bandwith and 10000 compute cycles for $100, so it looks good for your usage ?
I must have made a mistake. I calculated the cost and I got a very different value: 500*0.25 + 50*0.25 + 10000*0.01 = $350/mo. Where did I go wrong?
Regarding Acquia, they will provide hosting later this year with Fields and Garden, see http://acquia.com/community/projects/acquia-2009-roadmap
Thanks. I've spoken with my employer, and he actually likes the managed dedicated server idea after all. It looks like the £200-300/mo I quoted him doesn't sound so bad to him. So we might go that route. Daniel.
Regarding Mosso, I'll understand their basic plan will fit you ? It's 50GB HD, 500GB Bandwith and 10000 compute cycles for $100, so it looks good for your usage ?
I must have made a mistake. I calculated the cost and I got a very different value:
500*0.25 + 50*0.25 + 10000*0.01 = $350/mo.
Where did I go wrong?
Don't know where you got these numbers, see their pricing page http://www.mosso.com/pricing.jsp Start at $100, extra GB of storage is $0.5, extra GB of bandwith is $0.25 What I'm not really sure is how the compute cycle will stand, for exemple, I'm building a web site which abuse of views, and some dashboard admin page generates hundreds of queries... I know I should optimize this but the optimization will cost me a lot of time so if it doesn't hurt so much the bill it's ok... But I think you'll have to evaluate this to be sure not to explode the compute cycles in Mosso.
I'd like to thank everyone for all the help you've provided. We have decided to move to a dedicated server after all. It'll be a managed dedicated server, so I don't have to do the admin work. The server will be at the same host we have now, at least until the end of the school year (because this is the least disruptive option). After school is over we can evaluate whether we are fully satisfied with the service or not and I have bookmarked RackSpace as an alternative. The cloud computing sounds very interesting, but unless we can find a *managed* cloud computing service, I don't think it is right for us. I'd like to thank everyone for your assistance and guidance. You have been most helpful. Especially considering that the last 2/3 of the thread were not even about Drupal. Best regards. Daniel.
There are plenty of shared hosting companies that offer great support and can handle very busy sites. And this is coming from someone who hosts about 40 Drupal sites for clients on my own dedicated server. I have to say, I'm thinking about getting out of the hosting biz altogether, because I've seem really inexpensive hosting companies do a great job. I manage a number of GoDaddy hosted sites that work great with a lot of traffic and logged in users. The thing you do have to watch out for on shared hosting, are sites that consume a massive amount of php memory, such as a site running Ubercart, CiviCRM, Views, Devel etc -- with some shared host companies, even though it says you have plenty of php memory, it turns out they allocate it dynamically and you might wind up with the white screen of death on some of your pages. On one such site, I recently went with a managed VPS at servint.net -- which was recommended by a number of Drupal consultants. So far I've been happy with them and their prices are good. You can also use this code to get 50% off your first month - whtreturn09 For shared hosting, there's also http://hotdrupal.com/ -- which I haven't used, but they have been recommended by a number of people. Sam On Fri, May 15, 2009 at 11:02 AM, Nicolas Tostin <nicolast@logis.com.mx>wrote:
Regarding Mosso, I'll understand their basic plan will fit you ? It's 50GB HD, 500GB Bandwith and 10000 compute cycles for $100, so it looks good for your usage ?
I must have made a mistake. I calculated the cost and I got a very different value:
500*0.25 + 50*0.25 + 10000*0.01 = $350/mo.
Where did I go wrong?
Don't know where you got these numbers, see their pricing page http://www.mosso.com/pricing.jsp Start at $100, extra GB of storage is $0.5, extra GB of bandwith is $0.25
What I'm not really sure is how the compute cycle will stand, for exemple, I'm building a web site which abuse of views, and some dashboard admin page generates hundreds of queries... I know I should optimize this but the optimization will cost me a lot of time so if it doesn't hurt so much the bill it's ok... But I think you'll have to evaluate this to be sure not to explode the compute cycles in Mosso.
-- [ Drupal support list | http://lists.drupal.org/ ]
It's true that sometimes shared hosting gives great value. The problem is that the service is never consistent. Their fortunes and the service they offer varies tremendously over time. I started out with Site5, several years ago. For the first year they were great, recommendable. Then their server performance began to suck as evident overselling took place. Now, however, they have migrated to new servers (new owners) and right now are tremendously recommendable. Service is great too. However, I prefer the freedom of VPS and have been working with linode.com for over a year, and love it. Never a problem, strong performance, great service, irc discussion group, great wiki... Highly recommendable. My referral code is http://www.linode.com/?r=a105db8a0cca040b67cc8221fe25ffc84d45e25b Use it! Victor Kane http://awebfactory.com.ar On Fri, May 15, 2009 at 12:59 PM, Sam Cohen <support@newms.net> wrote:
There are plenty of shared hosting companies that offer great support and can handle very busy sites.
And this is coming from someone who hosts about 40 Drupal sites for clients on my own dedicated server.
I have to say, I'm thinking about getting out of the hosting biz altogether, because I've seem really inexpensive hosting companies do a great job. I manage a number of GoDaddy hosted sites that work great with a lot of traffic and logged in users.
The thing you do have to watch out for on shared hosting, are sites that consume a massive amount of php memory, such as a site running Ubercart, CiviCRM, Views, Devel etc -- with some shared host companies, even though it says you have plenty of php memory, it turns out they allocate it dynamically and you might wind up with the white screen of death on some of your pages.
On one such site, I recently went with a managed VPS at servint.net -- which was recommended by a number of Drupal consultants. So far I've been happy with them and their prices are good. You can also use this code to get 50% off your first month - whtreturn09
For shared hosting, there's also http://hotdrupal.com/ -- which I haven't used, but they have been recommended by a number of people.
Sam
On Fri, May 15, 2009 at 11:02 AM, Nicolas Tostin <nicolast@logis.com.mx> wrote:
Regarding Mosso, I'll understand their basic plan will fit you ? It's 50GB HD, 500GB Bandwith and 10000 compute cycles for $100, so it looks good for your usage ?
I must have made a mistake. I calculated the cost and I got a very different value:
500*0.25 + 50*0.25 + 10000*0.01 = $350/mo.
Where did I go wrong?
Don't know where you got these numbers, see their pricing page http://www.mosso.com/pricing.jsp Start at $100, extra GB of storage is $0.5, extra GB of bandwith is $0.25
What I'm not really sure is how the compute cycle will stand, for exemple, I'm building a web site which abuse of views, and some dashboard admin page generates hundreds of queries... I know I should optimize this but the optimization will cost me a lot of time so if it doesn't hurt so much the bill it's ok... But I think you'll have to evaluate this to be sure not to explode the compute cycles in Mosso.
-- [ Drupal support list | http://lists.drupal.org/ ]
-- [ Drupal support list | http://lists.drupal.org/ ]
If you're interested in either talking about working with someone or just combining knowledge pools of active web hosts get in touch with me - I'd be happy to work with a bunch of you to get some really detailed plans in place for helping all of us grow in this crazy market. -- Dan Horning American Digital Services - Where you are only limited by imagination. dan.horning@planetnoc.com :: http://www.americandigitalservices.com 1-518-444-0213 x502 . toll free 1-800-863-3854 . fax 1-888-474-6133 15 Third Street, PO Box 746, Troy, NY 12180 (by appointment only) From: support-bounces@drupal.org [mailto:support-bounces@drupal.org] On Behalf Of Sam Cohen Sent: Friday, May 15, 2009 12:00 PM To: support@drupal.org Subject: Re: [support] CPU usage There are plenty of shared hosting companies that offer great support and can handle very busy sites. And this is coming from someone who hosts about 40 Drupal sites for clients on my own dedicated server. I have to say, I'm thinking about getting out of the hosting biz altogether, because I've seem really inexpensive hosting companies do a great job. I manage a number of GoDaddy hosted sites that work great with a lot of traffic and logged in users. The thing you do have to watch out for on shared hosting, are sites that consume a massive amount of php memory, such as a site running Ubercart, CiviCRM, Views, Devel etc -- with some shared host companies, even though it says you have plenty of php memory, it turns out they allocate it dynamically and you might wind up with the white screen of death on some of your pages. On one such site, I recently went with a managed VPS at servint.net -- which was recommended by a number of Drupal consultants. So far I've been happy with them and their prices are good. You can also use this code to get 50% off your first month - whtreturn09 For shared hosting, there's also http://hotdrupal.com/ -- which I haven't used, but they have been recommended by a number of people. Sam On Fri, May 15, 2009 at 11:02 AM, Nicolas Tostin <nicolast@logis.com.mx> wrote:
Regarding Mosso, I'll understand their basic plan will fit you ? It's 50GB HD, 500GB Bandwith and 10000 compute cycles for $100, so it looks good for your usage ?
I must have made a mistake. I calculated the cost and I got a very different value:
500*0.25 + 50*0.25 + 10000*0.01 = $350/mo.
Where did I go wrong?
Don't know where you got these numbers, see their pricing page http://www.mosso.com/pricing.jsp Start at $100, extra GB of storage is $0.5, extra GB of bandwith is $0.25 What I'm not really sure is how the compute cycle will stand, for exemple, I'm building a web site which abuse of views, and some dashboard admin page generates hundreds of queries... I know I should optimize this but the optimization will cost me a lot of time so if it doesn't hurt so much the bill it's ok... But I think you'll have to evaluate this to be sure not to explode the compute cycles in Mosso. -- [ Drupal support list | http://lists.drupal.org/ ]
Question: I just counted and our front page probably requires 17 HTTP requests to load (several small images, two CSS files). That means that 500 HTTP requests 29 users. So if a classroom of 30 kids visits the front page at the same time, we would exceed 500 HTTP requests, right? Am I right in my reasoning? Part of me thinks that it is unlikely that 30 kids would all visit the site at precisely the same time. I figure that it's more likely that the page visits be spread out at least over a 10 second period, so maybe 50 requests is a more reasonable guess for one classroom. Is that right? Thanks.
If you have shell access to the server you can just run "top" (or I think also "load") and that will give the machines CPU utilization. I had a load problem with Dreamhost when then surprised me by moving my hosting account to a heavily loaded machine. But what made it worse was them then danced for weeks with excuses while they attempted to find the culprit on the machine abusing resources. And they never answered the question as to why they moved me in the first place. So, it fell upon me to run "top" regularly and emailing support complaining of loads in 100's and even 1000's! (I know the average sys admin says just move to a dedicated machine and be done with it, but during a ramp up when one can't afford it one must deal with what they have. Plus, moving a site(s) is no picnic. Note: I am not recommending Dreamhost any longer because of this. In fact, what I did was move part of my hosting to Site5 where they guaranteed that server loads would stay in single digits (which is where it's been since). They even post their machine loads on their website for anyone to see at any time, which felt reassuring. The only thing bad I've heard about Site5 is a post where they cut some guy off at the knees because of server resource abuse and wouldn't restart his account (at least that was his story, but I'll be he was warned and chose to do nothing about it. As apposed to the DH approach of telling the client something like "come on, stop that or we'll... --- and then do nothing. And I'm only saying this because I can't imagine it's too hard to figure out what resource is running away and simply killing it - of course, I'm not a sys admin, and don't care to be, so maybe it's harder than I think. Best Cozzi -----Original Message----- From: support-bounces@drupal.org [mailto:support-bounces@drupal.org]On Behalf Of Daniel Carrera Sent: Wednesday, May 13, 2009 10:23 AM To: support@drupal.org Subject: Re: [support] CPU usage It would help if I knew what the CPU usage actually was. Perhaps my site is just very badly configured (e.g. I didn't even have caching enabled). Also, the problem is not *technically* that I'm hitting a CPU quota. What happens is that we have a lot of requests coming from the same IP address (which is used by a number of schools, which are our customers). So the firewall at the server decides that this IP is mounting a DOS attack, so it blocks the IP, hence blocking our customers. I spoke with support, and they said if the CPU usage went down, it would reduce the chances of that school IP being blocked in the future. And that's how I ended up here asking about CPU usage. Cheers, Daniel. Steve Power wrote:
ask them to move you to a less busy machine?
On Wed, May 13, 2009 at 3:18 PM, Daniel Carrera <daniel.carrera@theingots.org <mailto:daniel.carrera@theingots.org>> wrote:
Greg Knaddison wrote:
A cheap VPS is $20/month and will likely solve the problem.
I am not a very good sysadmin and I hate sys admin work. I cry at the thought of having to configure Mailman (I have no idea how) or spam filters, or Apache, PHP and MySQL, or handle backup and recovery.
I speak from experience. I have been the sys admin of a non-critical Drupal site and I know I would not like to do the same for a mission critical site.
I chose this host as a way to get a balance between getting my fair share of resources while having someone else do the day-to-day admin work that I have neither interest nor aptitude for.
Debugging a slow site on shared hosting is an exercise in futility. Regardless of the value you place on your own time, it doesn't take much time troubleshooting before a low cost shared hosting plan becomes a false savings.
I did not choose the present shared host based on savings. I chose them because they have the best reputation for excellent support and uptime that is second to none. So far they have lived up to their reputation. My support requests receive an intelligent and useful response in less than 2 or 3 minutes.
My top priority is reliability. I don't have enough confidence in my admin skills to believe that I can achieve better reliability myself than I can with United Hosting.
Daniel.
-- [ Drupal support list | http://lists.drupal.org/ ]
-- -- -- Steve Power Principal Consultant Mobile: +44 (0) 7747 027 243 Initsix Technology and Media --
------------------------------------------------------------------------
-- [ Drupal support list | http://lists.drupal.org/ ]
-- [ Drupal support list | http://lists.drupal.org/ ]
Hi Cozzi, Thanks for your reply. I'm glad someone here understands me :-) United Hosting is much better than DH (and much more expensive too) as they do not over-sell resources and they have good support. Still, they are much cheaper than a managed dedicated server. UH has not moved me to a new server. And *in general* the site works fine. It's just this one very important IP that has problems. I am hesitant to blame UH because I know that another service (Mailfoundry) also tends to block the same IP for a similar-sounding reason. I have shell access to the server so I can run "top". And every user has his own instance of PHP, so I could configure that. But it looks like the problem is not with PHP but with MySQL. It is my hope that page caching will noticeably lower the number of MySQL requests. Cheers. Cosmo wrote:
If you have shell access to the server you can just run "top" (or I think also "load") and that will give the machines CPU utilization.
I had a load problem with Dreamhost when then surprised me by moving my hosting account to a heavily loaded machine. But what made it worse was them then danced for weeks with excuses while they attempted to find the culprit on the machine abusing resources. And they never answered the question as to why they moved me in the first place. So, it fell upon me to run "top" regularly and emailing support complaining of loads in 100's and even 1000's! (I know the average sys admin says just move to a dedicated machine and be done with it, but during a ramp up when one can't afford it one must deal with what they have. Plus, moving a site(s) is no picnic. Note: I am not recommending Dreamhost any longer because of this. In fact, what I did was move part of my hosting to Site5 where they guaranteed that server loads would stay in single digits (which is where it's been since). They even post their machine loads on their website for anyone to see at any time, which felt reassuring. The only thing bad I've heard about Site5 is a post where they cut some guy off at the knees because of server resource abuse and wouldn't restart his account (at least that was his story, but I'll be he was warned and chose to do nothing about it. As apposed to the DH approach of telling the client something like "come on, stop that or we'll... --- and then do nothing. And I'm only saying this because I can't imagine it's too hard to figure out what resource is running away and simply killing it - of course, I'm not a sys admin, and don't care to be, so maybe it's harder than I think.
Best Cozzi
-----Original Message----- From: support-bounces@drupal.org [mailto:support-bounces@drupal.org]On Behalf Of Daniel Carrera Sent: Wednesday, May 13, 2009 10:23 AM To: support@drupal.org Subject: Re: [support] CPU usage
It would help if I knew what the CPU usage actually was. Perhaps my site is just very badly configured (e.g. I didn't even have caching enabled).
Also, the problem is not *technically* that I'm hitting a CPU quota. What happens is that we have a lot of requests coming from the same IP address (which is used by a number of schools, which are our customers). So the firewall at the server decides that this IP is mounting a DOS attack, so it blocks the IP, hence blocking our customers.
I spoke with support, and they said if the CPU usage went down, it would reduce the chances of that school IP being blocked in the future. And that's how I ended up here asking about CPU usage.
Cheers, Daniel.
Steve Power wrote:
ask them to move you to a less busy machine?
On Wed, May 13, 2009 at 3:18 PM, Daniel Carrera <daniel.carrera@theingots.org <mailto:daniel.carrera@theingots.org>> wrote: Greg Knaddison wrote:
A cheap VPS is $20/month and will likely solve the problem.
I am not a very good sysadmin and I hate sys admin work. I cry at the thought of having to configure Mailman (I have no idea how) or spam filters, or Apache, PHP and MySQL, or handle backup and recovery.
I speak from experience. I have been the sys admin of a non-critical Drupal site and I know I would not like to do the same for a mission critical site.
I chose this host as a way to get a balance between getting my fair share of resources while having someone else do the day-to-day admin work that I have neither interest nor aptitude for.
Debugging a slow site on shared hosting is an exercise in futility. Regardless of the value you place on your own time, it doesn't take much time troubleshooting before a low cost shared hosting plan becomes a false savings.
I did not choose the present shared host based on savings. I chose them because they have the best reputation for excellent support and uptime that is second to none. So far they have lived up to their reputation. My support requests receive an intelligent and useful response in less than 2 or 3 minutes.
My top priority is reliability. I don't have enough confidence in my admin skills to believe that I can achieve better reliability myself than I can with United Hosting.
Daniel.
-- [ Drupal support list | http://lists.drupal.org/ ]
-- -- -- Steve Power Principal Consultant Mobile: +44 (0) 7747 027 243 Initsix Technology and Media --
------------------------------------------------------------------------
-- [ Drupal support list | http://lists.drupal.org/ ]
-- [ Drupal support list | http://lists.drupal.org/ ]
-- [ Drupal support list | http://lists.drupal.org/ ]
participants (14)
-
Adam Gurno -
Boris Berenberg -
Cosmo -
Daniel Carrera -
Daniel Horning -
Earnie Boyd -
Greg Knaddison -
Jamie Holly -
Joshua Rogers -
KOBA | Hans Rossel -
Nicolas Tostin -
Sam Cohen -
Steve Power -
Victor Kane