[consulting] consulting Digest, Vol 58, Issue 25

Kelly Bell kelly at gothamcitydrupal.com
Wed Nov 17 21:47:38 UTC 2010


Have any of you used migrate, schema and tw (table wizard) together before? I used this method to migrate a wp blog once (THAT was super-slick and fast), and a couple of other non-drupal databases (an excel spreadsheet once, and also a KickApps db dump). Using these modules together makes for kind of a swiss-army-knife (or maybe "universal can-opener" would be more apt) of migration tools, IMHO.

-Kelly Bell
twitter: kelly
linkedin/in/kellyrbell
gothamcitydrupal.com
drupal.org: kbell

On Nov 17, 2010, at 4:21 PM, consulting-request at drupal.org wrote:

> Send consulting mailing list submissions to
> 	consulting at drupal.org
> 
> To subscribe or unsubscribe via the World Wide Web, visit
> 	http://lists.drupal.org/mailman/listinfo/consulting
> or, via email, send a message with subject or body 'help' to
> 	consulting-request at drupal.org
> 
> You can reach the person managing the list at
> 	consulting-owner at drupal.org
> 
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of consulting digest..."
> 
> 
> Today's Topics:
> 
>   1. Re: Approaches for retrieving external site's data (Tom Geller)
>   2. Re: Approaches for retrieving external site's data
>      (Ant?nio P. P. Almeida)
>   3. Re: Approaches for retrieving external site's data
>      (Christopher M. Jones)
>   4. Re: Approaches for retrieving external site's data (nan wich)
> 
> 
> ----------------------------------------------------------------------
> 
> Message: 1
> Date: Wed, 17 Nov 2010 10:58:48 -0500
> From: Tom Geller <tom at tomgeller.com>
> Subject: Re: [consulting] Approaches for retrieving external site's
> 	data
> To: consulting at drupal.org
> Message-ID: <DBE358C3-7D6E-4CB3-8558-33E35CE6FE70 at tomgeller.com>
> Content-Type: text/plain; charset=us-ascii
> 
> Alfonso Montero Lopez <amontero at tinet.org> writes:
> 
>> The missing gap is in how to move data from the custom app table to
>> Drupal.
> 
> ...[possible solution]
> 
>> *Create a [password-protected] RSS feed script for the source site and
>> fetch the leads from it to the Drupal extranet via FeedAPI and alikes.
> 
> Yup, this is basically what I'm doing right now. But one change: Use Feeds (http://drupal.org/project/feeds) instead. I think it's more capable and modern than FeedAPI et al.
> 
> Victor Kane suggests:
> 
>> The migrate module http://drupal.org/project/migrate was designed for this
>> kind of problem and is a joy to work with.
> 
> ...if you have the chops for it. :) Migrate requires knowledge of PHP and a stronger grasp of technology generally than the point-and-click simplicity of Feeds. Migrate does appear more capable, though.
> 
> A "Migrating a Web Site to Drupal" paper I wrote for Acquia is at http://acquia.com/resources/library/white-paper-download-migrating-web-site-drupal .
> 
> Cheers,
> 
> ---
>          Tom Geller  *  Oberlin, Ohio  *  415-317-1805
>             Writer/Editor * http://www.tomgeller.com
>    articles, marketing, training materials, user guides, books
> 
> 
> 
> 
> ------------------------------
> 
> Message: 2
> Date: Wed, 17 Nov 2010 16:00:40 +0000
> From: Ant?nio P. P. Almeida <appa at perusio.net>
> Subject: Re: [consulting] Approaches for retrieving external site's
> 	data
> To: A list for Drupal consultants and Drupal service/hosting providers
> 	<consulting at drupal.org>
> Message-ID: <87fwv0djo7.wl%appa at perusio.net>
> Content-Type: text/plain; charset=US-ASCII
> 
> On 17 Nov 2010 15h58 WET, tom at tomgeller.com wrote:
> 
>> Alfonso Montero Lopez <amontero at tinet.org> writes:
>> 
>>> The missing gap is in how to move data from the custom app table to
>>> Drupal.
>> 
>> ...[possible solution]
>> 
>>> *Create a [password-protected] RSS feed script for the source site
>>> and fetch the leads from it to the Drupal extranet via FeedAPI and
>>> alikes.
>> 
>> Yup, this is basically what I'm doing right now. But one change: Use
>> Feeds (http://drupal.org/project/feeds) instead. I think it's more
>> capable and modern than FeedAPI et al.
>> 
>> Victor Kane suggests:
>> 
>>> The migrate module http://drupal.org/project/migrate was designed
>>> for this kind of problem and is a joy to work with.
>> 
>> ...if you have the chops for it. :) Migrate requires knowledge of
>> PHP and a stronger grasp of technology generally than the
>> point-and-click simplicity of Feeds. Migrate does appear more
>> capable, though.
> 
> I disagree. Feeds demands knowing how to code also, unless your
> migration is dead simple. E.g., if migrating from a another DB and
> using CSV you need to massage the data before creating the CSV file
> that is going to be used. For that you need coding.
> 
> Another approach is using a drush script with calls to the proper
> Drupal API functions for creating/updating content.
> 
> --- appa
> 
> 
> 
> ------------------------------
> 
> Message: 3
> Date: Wed, 17 Nov 2010 11:04:16 -0500
> From: "Christopher M. Jones" <cjones at partialflow.com>
> Subject: Re: [consulting] Approaches for retrieving external site's
> 	data
> To: consulting at drupal.org
> Message-ID: <4CE3FD00.1040303 at partialflow.com>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
> 
> Don't use FeedAPI, if you choose to go the feeds route. Use Feeds, its 
> successor. This opens up a whole new line of possibilities for you, as 
> you can combine Feeds.module with Data.module. Feeds.module will import 
> your data into custom tables. Data.module will surface those tables and 
> expose the data to views.
> 
> Whatever you do, I wouldn't mess with the external database connection. 
> Aside from the security risk, it could be very messy.
> 
> Yet another alternative would be to implement an XMLRPC webservice on 
> your non Drupal site and let Drupal consume it. This would provide the 
> ability for authentication. (Drupal does not, to my knowledge, have any 
> ability to handle password protected feeds. Someone correct me if I'm 
> wrong).
> 
> On 11/17/2010 06:12 AM, Victor Kane wrote:
>> The migrate module http://drupal.org/project/migrate was designed for
>> this kind of problem and is a joy to work with. Depends on a few other
>> modules, but one of the end results is that if you can get it onto a
>> MySql database, you can work to get views going with it! Etc. etc.
>> 
>> Victor Kane
>> http://awebfactory.com.ar
>> 
>> 2010/11/17 Alfonso Montero L?pez <amontero at tinet.org
>> <mailto:amontero at tinet.org>>
>> 
>>    Hi listers.
>> 
>>    I have recently got in charge of a non-Drupal website and I need some
>>    advice in approaching an issue.
>>    The site is about listing learning centers along their training
>>    courses and allowing users to request more info by filling a form. No
>>    big deal here. You fill the form, data gets dumped into a table and
>>    the webmaster (me) receives a mail.
>>    All the subsequent workflow is done more or less manually on the table
>>    that gets the information requests. However, it's a custom coded PHP
>>    site and expanding or trying to automate the workflow any further
>>    seems not worth the investment as in its current state, it's kind of a
>>    mess. I think it's a good Drupal candidate.
>>    The problem is that the website cannot be replaced quickly enough
>>    because of its design and look that I'm not able to translate to
>>    Drupal. By now, no budget will be allocated to theming and
>>    consequently I can't port it to Drupal. Since Drupal still has to
>>    prove its worth in the eyes of decision makers. A site rebuild with
>>    Drupal from scratch is not [now] an option, since management thinks
>>    "hey! we've invested >6 mo and blahblahblah". I would like to shift
>>    this by creating an extranet to show what can be done and get some
>>    serious attention upstairs :)
>> 
>>    What I'm figuring out how to accomplish is to create a
>>    non-public-facing extranet for the learning centers built with Drupal,
>>    so I can use nodes to get info requests, route them according my
>>    workflow and do all the fancy stuff that Drupal can do with rules,
>>    views, etc. This extranet would be used initially by us and later by
>>    our customers (the learning centers) and not by site visitors, they
>>    should only fill the form in the website exactly as they're doing now.
>>    The missing gap is in how to move data from the custom app table to
>>    Drupal. Thus, I'll be able to do automatic mails, route leads from one
>>    user/departments to another, do reports with views, etc.
>> 
>>    So, here I am thinking in how to approach this one the right way. I've
>>    thought in installing a Drupal alongside the current website, so I can
>>    access the site's MySQL DB from Drupal and someway and import the
>>    records periodically into nodes, but for hosting limitations, it's not
>>    possible now.
>> 
>>    The solutions I found are these:
>>    *Connecting remotely to MySQL via port 3306 and load new leads. Has
>>    firewall issues and may not be supported by all hostings all the time.
>>    Also, seems to require quite some programming effort and still have to
>>    test Data module capabilities. As long as I can see, data will not be
>>    nodes, so I will lose all future workflow, rules, etc. capabilities,
>>    right?
>>    *Create a [password-protected] RSS feed script for the source site and
>>    fetch the leads from it to the Drupal extranet via FeedAPI and alikes.
>>    By now, it's the most elegant solution I've found. I assume that
>>    FeedAPI won't have problems if I periodically empty the source site
>>    leads table and it will gracefully catch up.
>>    *(All and every single alternatives are welcome here)
>> 
>>    Obviously, when the extranet is finished and working, I plan to move
>>    all to a fully Drupal site and make it hit the front page.
>>    BTW, planning to do it in D6.x.
>> 
>>    So, previous experiences and caveats will be very welcomed. I'm sure
>>    awesome ideas from the listers here will come up. I learnt a lot by
>>    lurking here and in devel during 3 years.
>>    Thanks in advance.
>> 
>>    Alfonso.
>>    _______________________________________________
>>    consulting mailing list
>>    consulting at drupal.org <mailto:consulting at drupal.org>
>>    http://lists.drupal.org/mailman/listinfo/consulting
>> 
>> 
>> 
>> 
>> _______________________________________________
>> consulting mailing list
>> consulting at drupal.org
>> http://lists.drupal.org/mailman/listinfo/consulting
> 
> 
> ------------------------------
> 
> Message: 4
> Date: Wed, 17 Nov 2010 13:20:47 -0800 (PST)
> From: nan wich <nan_wich at bellsouth.net>
> Subject: Re: [consulting] Approaches for retrieving external site's
> 	data
> To: A list for Drupal consultants and Drupal service/hosting providers
> 	<consulting at drupal.org>
> Message-ID: <701637.35703.qm at web180309.mail.gq1.yahoo.com>
> Content-Type: text/plain; charset="iso-8859-1"
> 
> If the real point is proof-of-concept that Drupal can handle this, then just 
> copy the table into the Drupal database where you can access it like any other 
> table. You can easily explain that it is not up-to-date at the moment because 
> you are still in test. ?As?the testing goes on, it can be refreshed at any point 
> with a simple export/import. If you get a go-ahead nod, then import it one last 
> time and turn the old application off.
> ?
> Nancy
> ?
> Injustice anywhere is a threat to justice everywhere. -- Dr. Martin L. King, Jr.
> 
> 
> 
> 
> ________________________________
> From: Christopher M. Jones <cjones at partialflow.com>
> To: consulting at drupal.org
> Sent: Wed, November 17, 2010 11:04:16 AM
> Subject: Re: [consulting] Approaches for retrieving external site's data
> 
> Don't use FeedAPI, if you choose to go the feeds route. Use Feeds, its 
> successor. This opens up a whole new line of possibilities for you, as 
> you can combine Feeds.module with Data.module. Feeds.module will import 
> your data into custom tables. Data.module will surface those tables and 
> expose the data to views.
> 
> Whatever you do, I wouldn't mess with the external database connection. 
> Aside from the security risk, it could be very messy.
> 
> Yet another alternative would be to implement an XMLRPC webservice on 
> your non Drupal site and let Drupal consume it. This would provide the 
> ability for authentication. (Drupal does not, to my knowledge, have any 
> ability to handle password protected feeds. Someone correct me if I'm 
> wrong).
> 
> On 11/17/2010 06:12 AM, Victor Kane wrote:
>> The migrate module http://drupal.org/project/migrate was designed for
>> this kind of problem and is a joy to work with. Depends on a few other
>> modules, but one of the end results is that if you can get it onto a
>> MySql database, you can work to get views going with it! Etc. etc.
>> 
>> Victor Kane
>> http://awebfactory.com.ar
>> 
>> 2010/11/17 Alfonso Montero L?pez <amontero at tinet.org
>> <mailto:amontero at tinet.org>>
>> 
>> ? ? Hi listers.
>> 
>> ? ? I have recently got in charge of a non-Drupal website and I need some
>> ? ? advice in approaching an issue.
>> ? ? The site is about listing learning centers along their training
>> ? ? courses and allowing users to request more info by filling a form. No
>> ? ? big deal here. You fill the form, data gets dumped into a table and
>> ? ? the webmaster (me) receives a mail.
>> ? ? All the subsequent workflow is done more or less manually on the table
>> ? ? that gets the information requests. However, it's a custom coded PHP
>> ? ? site and expanding or trying to automate the workflow any further
>> ? ? seems not worth the investment as in its current state, it's kind of a
>> ? ? mess. I think it's a good Drupal candidate.
>> ? ? The problem is that the website cannot be replaced quickly enough
>> ? ? because of its design and look that I'm not able to translate to
>> ? ? Drupal. By now, no budget will be allocated to theming and
>> ? ? consequently I can't port it to Drupal. Since Drupal still has to
>> ? ? prove its worth in the eyes of decision makers. A site rebuild with
>> ? ? Drupal from scratch is not [now] an option, since management thinks
>> ? ? "hey! we've invested >6 mo and blahblahblah". I would like to shift
>> ? ? this by creating an extranet to show what can be done and get some
>> ? ? serious attention upstairs :)
>> 
>> ? ? What I'm figuring out how to accomplish is to create a
>> ? ? non-public-facing extranet for the learning centers built with Drupal,
>> ? ? so I can use nodes to get info requests, route them according my
>> ? ? workflow and do all the fancy stuff that Drupal can do with rules,
>> ? ? views, etc. This extranet would be used initially by us and later by
>> ? ? our customers (the learning centers) and not by site visitors, they
>> ? ? should only fill the form in the website exactly as they're doing now.
>> ? ? The missing gap is in how to move data from the custom app table to
>> ? ? Drupal. Thus, I'll be able to do automatic mails, route leads from one
>> ? ? user/departments to another, do reports with views, etc.
>> 
>> ? ? So, here I am thinking in how to approach this one the right way. I've
>> ? ? thought in installing a Drupal alongside the current website, so I can
>> ? ? access the site's MySQL DB from Drupal and someway and import the
>> ? ? records periodically into nodes, but for hosting limitations, it's not
>> ? ? possible now.
>> 
>> ? ? The solutions I found are these:
>> ? ? *Connecting remotely to MySQL via port 3306 and load new leads. Has
>> ? ? firewall issues and may not be supported by all hostings all the time.
>> ? ? Also, seems to require quite some programming effort and still have to
>> ? ? test Data module capabilities. As long as I can see, data will not be
>> ? ? nodes, so I will lose all future workflow, rules, etc. capabilities,
>> ? ? right?
>> ? ? *Create a [password-protected] RSS feed script for the source site and
>> ? ? fetch the leads from it to the Drupal extranet via FeedAPI and alikes.
>> ? ? By now, it's the most elegant solution I've found. I assume that
>> ? ? FeedAPI won't have problems if I periodically empty the source site
>> ? ? leads table and it will gracefully catch up.
>> ? ? *(All and every single alternatives are welcome here)
>> 
>> ? ? Obviously, when the extranet is finished and working, I plan to move
>> ? ? all to a fully Drupal site and make it hit the front page.
>> ? ? BTW, planning to do it in D6.x.
>> 
>> ? ? So, previous experiences and caveats will be very welcomed. I'm sure
>> ? ? awesome ideas from the listers here will come up. I learnt a lot by
>> ? ? lurking here and in devel during 3 years.
>> ? ? Thanks in advance.
>> 
>> ? ? Alfonso.
>> ? ? _______________________________________________
>> ? ? consulting mailing list
>> ? ? consulting at drupal.org <mailto:consulting at drupal.org>
>> ? ? http://lists.drupal.org/mailman/listinfo/consulting
>> 
>> 
>> 
>> 
>> _______________________________________________
>> consulting mailing list
>> consulting at drupal.org
>> http://lists.drupal.org/mailman/listinfo/consulting
> _______________________________________________
> consulting mailing list
> consulting at drupal.org
> http://lists.drupal.org/mailman/listinfo/consulting
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: http://lists.drupal.org/pipermail/consulting/attachments/20101117/4f47d26c/attachment.html 
> 
> ------------------------------
> 
> _______________________________________________
> consulting mailing list
> consulting at drupal.org
> http://lists.drupal.org/mailman/listinfo/consulting
> 
> 
> End of consulting Digest, Vol 58, Issue 25
> ******************************************
> 



More information about the consulting mailing list