Oh yes, modules that do this, are designed well and really work and can be easily installed are definitely more than interesting: they empower!<br><br>+1<br><br>On <a href="http://drupal.org">drupal.org</a> we can all download and use the module and converge our feedback into the issue queue!<br>
<br>Looking forward to seeing it!<br><br>Victor Kane<br><a href="http://awebfactory.com.ar">http://awebfactory.com.ar</a><br><br><div class="gmail_quote">On Sat, Aug 1, 2009 at 10:50 AM, Pierre Rineau <span dir="ltr"><<a href="mailto:pierre.rineau@makina-corpus.com">pierre.rineau@makina-corpus.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">Hello all,<br>
<br>
Working on some custom project for my company, I developed a module to<br>
do massive migration between sites.<br>
<br>
This module uses a full OO layer.<br>
<br>
Its internal mechanism is based on abstracting objects to migrate from a<br>
master site to clients. This abstraction defines how to construct object<br>
dependency tree and how to serialize objects.<br>
<br>
Object implementation (node, user, taxonomy, whatever) is really simple<br>
to use, it's only 3 methods classes (register dependencies, save, and<br>
update) using some kind of custom registry for developer to save/get<br>
back data before and after serialization.<br>
<br>
All error handling is exception oriented, and lower software layers<br>
won't fail on higher layers unrecoverable errors.<br>
<br>
Object fetching is based on a push/pull mechanism. Server push the sync<br>
order, client responds OK or not. If OK, it creates a job using DataSync<br>
module which allow it to run as CLI thread (which won't hurt the web<br>
server, and allow us a larger memory limit at run time). DataSync module<br>
uses MySQL transactions (shame it's only MySQL compliant, but I hope it<br>
will evolve, I'm thinking about PostgreSQL).<br>
<br>
During the DataSync job execution, client will pull an original set of<br>
content, and browsing it will do incremental dependencies fetching (by<br>
pulling again server), based on xmlrpc (fetching component is also<br>
abstracted, and could be any other communication method than xmlrpc).<br>
<br>
To be unobtrusive on the system, smart unset() is done after building a<br>
dependencies subtree, and there is a recursion breaker in case of<br>
circular dependencies.<br>
<br>
This module was created because the deploy module seems to be so<br>
unstable, I did not want to risk client's production sites to run with<br>
it. I started implementation of some sort of "deploy plan", using<br>
profile based on views, you construct a set of views, saved them in a<br>
profile, then all objects that these views reference will be<br>
synchronized.<br>
<br>
Right now, the module fully synchronize taxonomy and content types,<br>
partially synchronize users (including core account information and<br>
passwords), and I have a small bug left to handle with nodes (revision<br>
problem I think).<br>
<br>
There might be a performance or overhead problem with this conception<br>
with a very large amount of data, it could break easily. The only way to<br>
be sure it won't break is I think to migrate stuff with a numerous small<br>
set of data. But the problem doing this is that it will be really hard<br>
to keep the transactional context of DataSync module.<br>
<br>
There is a lot of other custom goodies coming.<br>
<br>
First thing is, what do you think about such module, should I commit it<br>
on <a href="http://drupal.org" target="_blank">drupal.org</a>? Is there people interested?<br>
<br>
And, now that I described the module, what name should I give him,<br>
considering the fact I'll probably commit it on <a href="http://drupal.org" target="_blank">drupal.org</a>, if people<br>
are interested.<br>
<br>
I though about "YAMM" (Yet Another Migration Module), or YADM (Yet<br>
Another Deployment Module).<br>
<br>
The fact is there is *a lot* of modules which want to do the same thing<br>
as this one, I just want a simple an expressive name.<br>
<font color="#888888"><br>
Pierre.<br>
<br>
</font></blockquote></div><br>