Kevin, Thanks for the feedback. When you say 'moving a discrete change from dev -> integration -> test -> production' do you mean the scenario where you have a updated version of your site with new/updated modules (or modules removed) + new content on your staging site and wish to automatie deploy those changes to your production site? In that case then you would need the ability to package an entire Drupal install into a structured file that a script could use to recreate the entire site on another machine, and then automatically run a complete set of smoke-tests to verify the entire unpackaged site is completely functional. I could definitely see this as an extended application of my idea. You would have to take your live site offline when you rollout the new version though I'm not sure about the 'hot-deploy' part - you mean basically updating the exisiting Drupal db + filesystem in-place? I had a somewhat related conversation with Doug Muth who maintains the backup module about a restore feature that could roll back a live site to a previous backup, and he was concerned about how Drupal would react when it has 'the rug pulled out under it' Modifying tables directly on a live site could have a whole bunch of bad consequences. Allister. On 19/03/07, KevinImNotSpacey <kevin.amerson@gmail.com> wrote:
Hey Allister,
I can't speak to Google SoC requirements, but your idea sounds great. I could see some code being written to automate setting up the environment, perhaps that could be the focus. I know there is a set of repetitive tasks that we go through whenever evaluating a new module or upgrading an existing one:
1) Get the module from drupal.org 2) Stage our database and other contrib modules 3) Copy over the module code and enable it through the admin UI 4) Test expected behaviors
A separate but somewhat related question/task/complicated activity is moving changes between environments, such as moving a discrete change from dev -> integration -> test -> production. There is a module called importexportapi that does some of this, but no real complete solution. Perhaps your project could include a methodology + utility to allow us to hot deploy drupal changes to a prod environment.
Kevin
On 3/19/07, Allister Beharry <allister.beharry@gmail.com> wrote:
Thanks for the heads-up Robert. I've been reading the SoC project proposal for automating testing of patches, and also your posts on a performance and scalability testing environment for Drupal. Virtual machine images have been posited as part of an solution for both. So I'm going to channel Laskey wrt my original idea and propose a better one. (I'm just punching this out before I race to class so it's gonna be brief/fragmented with typos. I'll elaborate further when I get back.)
A key requirement of automated testing of Drupal is the ability to create a Drupal site with a specific configuration. In the case of unit testing patches and code and also regression testing, the Drupal site must be running specfic versions of core and module code. Now automating tests is one thing but at the end-of-the day a human will have to be able to eyeball a Drupal site with specific versions of core and module code. There needs to be an automated way for a tester to create a complete Drupal sit by simply specifiying what code versions he/she wants running
For performance and scalability testing,a human tester will want to vary things like choice of web server - Apache, lightpd, IIS - choice of db - MySQL, PostgreSQL - PHP configuration - mod_pho, FastCGI, PHP enhancements like APC, Zend optimizer, reverse proxies, as well as varying load parameters for the Drupal site as you mentioned - number of core/contrib modules, number of databases, memcache, database topography, so on. You'll also need to populate the db with sample content and user account. For testing on Amazon EC you'd need an AMI image
What is needed is some kind of interface or configuration script that a tester can use to quickly generate a Drupal site configured with these parameters. The script interpreter would take a configuration script and a designated target, and spit out: 1. A ready to run tarball with drupal settings.php + php.ini + database script thatcan be immediately deployed to an existing web + database server setup 2. A virtual machine image in either VMWare or Xen or AMI format.
Well it sounds kind of ambitious, but even if done on a scaled back way, could have applications well outside Drupal. What do you think?
Allister.
On 19/03/07, Robert Douglass <rob@robshouse.net> wrote:
Allister,
before I discuss your idea, it should be noted that you can't propose a GSoC project which doesn't produce code. As you describe it, your idea is mostly a task for a sysadmin and doesn't produce code, therefore wouldn't be eligible for the program.
Your idea, however, is very good. I had envisioned a similar use of virtual images here: http://groups.drupal.org/node/2742 The problem with doing my idea as GSoC is that it is more than one person could reasonably manage, I think, and is dependent on infrastructure that we're not in a position to provide.
I think what is needed is some brainstorming that combines your software appliance with performance, scalability and regression testing. Please chew on these ideas and come up with an alternate proposal. We'd love to have you apply to GSoC with a Drupal project.
cheers,
Robert
Allister Beharry wrote:
Hello all, I'll be participating in SoC 07 and I'd appreciate it if I can get some feedback on my idea for a project with Drupal. Here is the synopsis:
My project idea for SoC 2007 is a Drupal software appliance – a self-contained virtual image consisting of a minimal Linux environment, a LAMP stack, and Drupal 5.1 – which can be distributed as a bootable LiveCD or virtual machine image which can be run on virtualization software like VMware Player or MS Virtual PC.
At ~50Mb or less, a Drupal appliance is a perfect way for potential users with no system administration experience to evaluate Drupal. Advocacy for Drupal can be extended by distributing the Live CD with magazines and in trade shows. Drupal companies and consultants can rapidly prototype and demonstrate a Drupal a solution to their clients starting with the base appliance image. Drupal admins can easily test module patches before deployment to live sites. The virtual machine image can also be part of an automated testing framework by utilizing the scripting capabilities of virtualization software like VMware Server.
This project can build on existing LiveDistro images from Knoppix or Damn Small Linux, and virtual appliance images like those from Virtual Appliance ( http://virtualappliances.net/products/lamp/). The main tasks will be: 1.Collecting requirements from the Drupal community; 2.Building the target environment; 3.Installing and integrating Drupal and necessary modules; 4.Comprehensive testing of the entire appliance; 5.Coding hooks in the virtualization software for use of the appliance by automated testing tools; 6.Promotion and obtaining distribution channels for the release of the appliance.
Thanks a lot. Allister
-- * * * * * Lullabot's First Ever Advanced Workshops Are Here! Drupal API & Module Building - Advanced Drupal Themeing April 9th-13th - Providence, RI Early Bird Discounts Available Now http://www.lullabot.com/training * * * * *