[development] Distilling distributions from configured sites.

[Adrian Rossouw]

So on the topic of distributions (what I am going to call install profiles
from now on), some code
I recently wrote for http://drupal.org/node/25745, that allows you to very
easily rip out the variables
configured on the site and store them as defaults.

function export_defaults() {
  global $conf;

  foreach ($conf as $name => $value) {
    $output .= "\$default['$name'] = " . $value . ";\n"; // doesn't work for
objects...
  }
  return $output;
}

So that led me to think about what other things we have to import / export
for new sites. I also discussed the
same with Jaza (who is working on the import/export api.). He has targetted
the following things in core
that need to be copied :

http://basement.greenash.net.au/soc2006/CurrentTodoList

Jaza's system is far more complex than our meagre needs for the
distributions right now, plus it won't be ready
in time for 4.8.I also think my proposed approach has a lot of benefits for
both our requirements for the distributions
(being able to easily build and maintain them), the Drupal API in general
and will actually probably make his life easier,
possibly even fast tracking some of his code into core.

Now my idea (which is very similar to how the views module does it) , is to
use a set of functions such as :

function drupal_export($resultset, $function, $id_field, $objects = TRUE) {
  $x = 0;
  while ($id = db_fetch_array($resultset)) {
    $object = $function($id[$id_field]);
    $output .= "\$object[$x] = new stdClass();\n";

    foreach ($object as $field => $value) {
      if ($field != $id_field) {
        if (is_object($value)) {
          $output .= "\$object[$x]->$field = (object) " . var_export((array)
$value, TRUE) . ";\n";
        }
        else {
          $output .= "\$object[$x]->$field = " . var_export($value, TRUE) .
";\n";
        }
      }
    }
    $x++;
  }
  return $output;
}



function drupal_import($objects, $function) {
  foreach ($objects as $object) {
    $function($object);
  }
}

for export :
$result = db_query("select nid from node order by nid asc");
print drupal_export($result, 'node_load', 'nid')); // or rather save into a
file

and then on import :
include_once 'distributions/default/node.import';
drupal_import($objects, 'node_save);


How I see this working is that each module (or thing), will get placed in
the distribution/mydistribution/<driver>.import files
which are very simply created, and very simply modified. Since we kind of
know what order things need to be imported in
we will just recurse through the object and import them.

These following things are already handled :

system table               = enabled modules, and themes. Handled by the
.distribution file (until we can get dependencies
                                     into core that allows them to be
configured in the .meta file (ie: requires:theme-bluemarine, blog))
variables (defaults)      = handled by the distribution.inc file. see
http://drupal.org/node/25745

The following things desperately need to be handled, cleanly and easily :

profile-fields, users, node types,  locale ,vocabularies, terms, blocks,
aliases, files (more on this later)  nodes , comments and finally menus.
(in roughly that order.)
I'm unsure about locale in that list. (maybe it should be a setting?)

These are optional, but i'd like to see them :
aggregator: feeds, items and categories .. could be really awesome for
networks of sites.
views : it already does this, but could be used as both the basis for the
api, and one of the most practical uses of it.
cck : might already do this, but it's important that it does.
action/workflow : don't know if it uses it's own table structure or the
variable table, but it would be useful for distribution.

any other contrib module that would benefit from this.

What this however requires , is that we have a consistent set of _load and
_save functions, which are all equal
to node_load. I feel this is something we should have to begin with.
Additionally, especially with node_load and block_load, we
will need a way to access the original php code for php filter fields, and
not just the output. Also, we need to decide whether
initial imports should support revisions (i am not sure they should).

Perhaps we could even get away with just having a module_import and
module_export hook, which could do the
mappings to whatever it's internal functions currently are. This would be a
non-invasive way to implement this
functionality, and give us a very good idea of what we need to make
consisten _save and _load functions for in
future releases. Some of this work is possibly already done as part of
jaza's import / export project. Some of the node
related code might already be living in publish and subscribe.

This moves us further away from doing database dumps, and makes
distributions a lot easier to create
(you would just run 'scripts/build_distro.php --site
mysite.comnewdistroname'. And then new sites would
use that as their basis.

One of the more awesome side effects of this all, being we can also
distribute each of these as packages on drupal.org,
and teach the install system to install them all. It provides a clean way
for modules to distribute views
and cck node types etc, perhaps even custom blocks and templates (with the
inline template editor i want to write)
There are some issues with this on already running sites, with versioning of
all these elements, but i believe
the dependencies work will end up helping with this.

One of the other things I would like to see us do, is to provide variables
that have been submitted as part of the
distribution specific form, as variables to the .import files. A perfect
example of this is having the first node configured
as "welcome to your site $edit[fullname]". Even the creation of the first
user (for instance), could be handled via this.

Another point I came across, is the files directory. I believe the installer
should try and create it, and I also believe the distributions should be
able to have default files that are copied to the directory.

An example being that a brochureware distribution might ship with a
customised logo/favicon, and possibly an initial node configured and made to
be the front page, which has a nice piece of stock photography already
present (giving people a presentable front page to work from).  It might
even ship
with several different templates and allow you to select which one of them
you want your front page to look like.

That's a rather rambling post for what is actually really simple in concept,
and if we build the framework within which to do it
and have a developer per import / export object .. we should hopefully still
be able to get something like this into 4.8... and
then we can watch the distributions bloom =)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.drupal.org/pipermail/development/attachments/20060809/1a911fff/attachment-0001.htm