Issue status update for http://drupal.org/node/5496 Project: Drupal Version: cvs Component: base system Category: tasks Priority: normal Assigned to: adrian Reported by: adrian Updated by: adrian -Status: active +Status: patch Attachment: http://drupal.org/files/issues/install_api_phase_1.tar.bz2 (21.49 KB) Attached is the first iteration of my new install system. To install : Extract tarball into a clean checkout, and apply the install_api_phase_1.patch file. This patch implements the following : Core install api. /(includes/install.inc)/ Modifications to drupal_load that disables a module that has an out of data schema. /(includes/bootstrap.inc)/ Does not allow modules to be enabled unless they have been installed, or updated and the allow for them to be installed and updated directly from admin/modules. /(modules/system.module)/ This patch does NOT : Install or update Drupal core, however it is possible to do so with the api (and I have done so before), significant changes to Drupal need to happen before it is practical to handle core with it. Description : This is the first phase of a four or five phase project, so it's not meant to solve the whole problem. I am writing a post for the Drupal forums at present detailing the progress with the install system, and directions I want to take it. To enable a module to be installable, you add an implementation of hook_info() to it , i.e. : <?php function buddylist_info($key) { $info['schema'] = 1; return $info[$key]; } ?> This tells the system that a schema version of 1 is required to operate this module. You then create a file called modulename.install (preferably in the same directory in the module, although it doesn't really matter. You then need to specify which schema version it implements by : <?php function buddylist_schema($key) { $info['version'] = 1; return $info[$key]; } ?> Although, in the current implementation this function is optional when you only have a single schema version (the default is 1). Then you implement your hook_install(). <?php function buddylist_install() { $query = " CREATE TABLE {buddylist} ( uid int(10) UNSIGNED NOT NULL, buddy int(10) UNSIGNED NOT NULL, timestamp int(11) NOT NULL, PRIMARY KEY (uid, buddy) )"; db_query($query); db_query("ALTER TABLE {buddylist} ADD received tinyint(1) NOT NULL;"); } ?> This function would be called for any database type that has not specified it's own function. To add Postgres support, the function would be called buddylist_install_pgsql();. If the schema of your module ever changes, to allow for it to be updated, you proceed as follows : increment the 'schema' value in your hook_info() function. increment the 'version' value in your hook_schema function, within your modulename.install file. modify your schema as required, by changing it in the hook_install() function. provide an update script for users of the previous version. The following is an example of an update script. <?php function buddylist_update_1() { db_query("ALTER TABLE .... ..... "); } ?> These updates can also be database specific (ie: buddylist_update_1_pgsql() etc). When a module needs to be updated, it will execute each of the functions in order (exactly like one would in updates.inc currently.) I have included copies of the Trackback and Buddylist modules for testing. adrian Previous comments: ------------------------------------------------------------------------ January 29, 2004 - 11:10 : adrian I am in the process of developing a new Install API for the drupal core, which will allow a completely web based install / set up process, as well as allow contributed modules to integrate more directly into this system. A more detailed description of the system is available in this thread. This task has been created to place each of the successive patches in the patch queue. ------------------------------------------------------------------------ January 29, 2004 - 11:20 : adrian Attachment: http://drupal.org/files/issues/install_api_first_level.patch (6.42 KB) Kjartan correctly identified one of the problems in creating a unified install system. There's a lot of code that gets run by simply including the core files (bootstrap.inc, database.inc, session.inc and common.inc). A lot of this code relies upon the database connection going off without a hitch, like for instance the current system variables, themes and session code. To work around this i created a new drupal_init() function, which needs to be called when you want the system to initialize. drupal_init has a 'db_init' argument, which allows you to bypass the loading of all the database related code with a simple flag. install.php needs to run drupal core, while ignoring any database connections .. until atleast the point where the configuration settings for the database are entered, and confirmed to work. There is a new function created called 'database_connected()' , which returns a boolean true only if the database both connected and the table was selected successfully. This patch does not have a suitable error message for index.php upon not being able to make a database connection ,but that message should very likely be discussed and finalized. In the next level of the patch I will introduce the major changes, or rather additions. I haven't modified any existing files beyond the ones in this patch (at this time). ------------------------------------------------------------------------ January 29, 2004 - 14:24 : Kjartan This patch has some serious flaws though. <?php include_once "includes/bootstrap.inc"; -drupal_page_header(); include_once "includes/common.inc"; ?> By removing the drupal_page_header() you break the whole point of bootstrap as the common.inc stuff is included when its not necessary. It needs some more thought. Maybe you should take the time to make a workflow of the current Drupal init process before modifying it. Also the naming of constants, I would prefer calling them DATABASE_ATTEMPT as it makes it easier to tell them appart. ------------------------------------------------------------------------ February 9, 2004 - 22:18 : adrian I have reworked the patch into a very minimal set of requirements. To prevent the .inc files from running any code when you don't want them to , you 'define' a constant called 'DRUPAL_NO_INIT' before including bootstrap.inc. This relatively small patch is required for the install stuff to work. ------------------------------------------------------------------------ February 9, 2004 - 22:22 : adrian Attachment: http://drupal.org/files/issues/drupal_no_init_flag.patch (3.88 KB) i'll even attach it this time. ------------------------------------------------------------------------ February 9, 2004 - 22:48 : adrian Attachment: http://drupal.org/files/issues/database_init_return.patch (1.96 KB) The following patch is a modification to the includes/database*.inc set of files, to make db_connect return a boolean value depending on wether the connection was successfully made. I have changed database.inc to die() upon connection error with the message "Database Connection Unsuccessful". Cleaning up that message and placing a help page might be a good idea from an user experience point of view. This patch is required for the install system, in that it needs to test wether it has made a successfull database connection before it allows you to actually install the schema. ------------------------------------------------------------------------ February 10, 2004 - 03:15 : adrian Attachment: http://drupal.org/files/issues/install_system_alpha.tar.bz2 (11.59 KB) This is an alpha version of the new install API code. It is completely capable of installing the drupal core system without requiring any interaction with the database side. There are currently 5 configuration screens , but the API is sound for us to add / remove / change these screens as we need. In it's current state, it isn't capable of updating the base system, however the backend stubs exist, and have been tested. In the push to get the install process running smoothly i simply disabled the update interface. The previous 2 patches on this node need to be applied, but have been combined into install_system_alpha.patch which is included in this distribution. The rest of the files are new , and should not interfere with the drupal core at all. They are includes/install.inc ... the file that hosts the install api. install.php ... the front end script that initializes the install api (a total of 5 lines) install/system.install ... the install module for the core. It contains the bulk of code from update.php, and all the configuration information needed to successfully install the core. In the future , the install api will be able to easily install any contrib module through the modulename.install file that will hopefully become well supported. Note: I need to update the postgres compatibility in this .install file, and as such only mysql is fully tested now. I have posted some screenshots at my site ------------------------------------------------------------------------ February 10, 2004 - 20:22 : adrian Unconed just pointed out there is a conflict with windows installations in that there can not be an 'INSTALL' file and an 'install' directory (which contains the system.install currently) He suggested that we rename the README and INSTALL files to README.txt and INSTALL.txt. I personally move that we store the .install files within the modules directory , as in the future installing a contributed module should be as simple as dropping the module directory (containing .install , .module and whichever external files) into modules/ , and then installing the module within drupal. ------------------------------------------------------------------------ February 12, 2004 - 07:10 : moshe weitzman I played around with the patch, and successfully installed a new Drupal instance on the fitst try. Nice work, Adrian! I reviewed the code too, and found it quite Drupalish and satsfactory. Here are some notes from my install: - On the configuration file page, I selected from the radio button options a file which was not conf.php and thus did not exist. I strugged with the 'file is not writeable' error for a while until I realized that you expected me to create the file and then the installer would edit it ... in general, this page is a pretty complex introduction into drupal. I suggest simply assuming 'conf.php' here. I haven't thought about this much though. - Similar to above, I figured Drupal would create a database for me. Not so, I had to create it and then Drupal takes over. Some simple help text adddresses this point. - Use form_password() or similar when requesting a database password - The default $base_url was missing a slall for me. It said 'http://medrinstall' instead of 'http://me/drinstall' - After completing the DB install, there is no page telling you what to do next. A link to the home page is all thats needed IMO. -The site_name chosen during the install was not saved in the 'variable' table. - When you return to install.php some time later, you have no indication what a working site already exists. Not sure thats the right thing to do. Not a big deal though. P.S. Adrians suggestion to store the .install files in module specific subdirs under modules makes sense to me. Thus system.install would move to modules/system/system.install ------------------------------------------------------------------------ February 13, 2004 - 06:04 : skip I had to set up a pMachine blog yesterday for a client. I have to say... it was the easiest install process I've ever gone through (CMS-wise). Very user friendly. Lots of feedback to the user. If you haven't seen it before, you should check it out for ideas. ------------------------------------------------------------------------ February 18, 2004 - 07:26 : adrian Attachment: http://drupal.org/files/issues/install_phase2.tar.bz2 (12.96 KB) So I have spent far too much time over the last few weeks battling with this install system. My current results are in this here tarball. This time I am even less sure of it fully working in all cases, than last time. With this release I added extensive error checking to the install process, finished MOST of what is needed for the update process , but most importantly... it is a lot more secure than previously, and is specifically built to allow people to host their multiple drupal installs from just a single root. I have handled this by adding a global variable to the config file (allow_override_config) , which is checked by the install script before a new configuration file is made that would override the currently active one. There are three settings to this : 'no' , 'any' and 'auth'. I feel auth is generally the best option, as it requires anyone trying to override a working configuration file to enter the username and password for the site he is trying to override. Added checking also exists for the minimum requirements of running a drupal system (the tests probably need to be fleshed out), aswell as refusing to install a database dump into a db_url/db_prefix which seems to have a valid site installed. When a working configuration is detected on startup , the default action is to require the administrator user of the site to log in. This is for security reasons. The install script has lots of code that stops it from ever overwriting any config file that has already been created, but as I said .. it allows for the creation of a config file that will be parsed before the current valid install, but only with the express permission of that config. Config files have been moved to './conf/' , and the default config has been renamed to 'conf/default.php'. I have split install.inc into install.inc and install.module, as there was a lot of interface code specifically for the update/install sequence (and it is best available for reuse .. so i didnt wanna put it in install.php). This has an unrelated quirk for xtemplate .. I left it in the patch because for some reason I was unable to get a working default theme because xtemplate refused to populate the 'path' template variable. I also have an extensive 'gallery' of the new changes if you just want to see what it looks like. Directions for use : untar in a drupal cvs tree , and 'patch -p0 < install_alpha_2.patch' go to http://mysite/install.php ... previous configs in includes/ will be ignored for extra fun , create a hostname entry to sitea.siteb.sitec.sited.mysite , and watch the install script pick up the valid configs. ------------------------------------------------------------------------ February 18, 2004 - 17:14 : TDobes Adrian, thanks for all your work on the install system... I'll check it out once I get home this evening. Just FYI, here's a link to the XTemplate bug [1] you mentioned. (It's a simple two-line patch.) [1] http://drupal.org/node/view/5858 ------------------------------------------------------------------------ June 20, 2004 - 22:29 : adrian taking out of patch queue. Still on my todo list, but I have to get some other things done first. ------------------------------------------------------------------------ July 27, 2004 - 17:25 : JonBob The install system should (will?) handle version numbers. Some related issues to examine: None of these are really duplicates of one another, so while I'd like to close them I can't. Consider this a reminder to do so if and when the install system addresses the problems. New 'version' system field and Drupal version [2] Module dependencies [3] Show drupal version to admins [4] Meta tag with drupal version [5] Version number [6] [2] http://drupal.org/node/view/5811 [3] http://drupal.org/node/view/8402 [4] http://drupal.org/node/view/2986 [5] http://drupal.org/node/view/2828 [6] http://drupal.org/node/view/8762 ------------------------------------------------------------------------ March 13, 2005 - 18:03 : publian It made it based on install_phase2. It makes it with drupal CVS as of 2005.03.14. The improvement point is as follows. 1) The installation query command execution read database/database.mysql or database/database.pgsql. 2) Two head bytes of $_SERVER 'HTTP_ACCEPT_LANGUAGE' were seen, and the installation language was distinguished automatically. This sees the install.lang file in the languege folder. (funcution lt() like funcution t()) Making) 3) In the installation process, the mother tongue can have been selected. (The import is done from the file in the translations folder. ) 4) Additionally, a detailed place was corrected. The bug moves though thinks many still. -- Drupal Japan ------------------------------------------------------------------------ March 13, 2005 - 18:10 : publian Attachment: http://drupal.org/files/issues/install_phase3.tar.gz (134.72 KB) It is as for appending. ------------------------------------------------------------------------ March 26, 2005 - 10:11 : judah I was just about to start creating an install system for modules when someone mentioned you were already working on one. Here is my design specs, http://drupal.org/node/19427 I could not contact you via any other means so please advise what the current status is on this. Thanks