[consulting] Approaches for retrieving external site's data

[Christopher M. Jones]

Don't use FeedAPI, if you choose to go the feeds route. Use Feeds, its 
successor. This opens up a whole new line of possibilities for you, as 
you can combine Feeds.module with Data.module. Feeds.module will import 
your data into custom tables. Data.module will surface those tables and 
expose the data to views.

Whatever you do, I wouldn't mess with the external database connection. 
Aside from the security risk, it could be very messy.

Yet another alternative would be to implement an XMLRPC webservice on 
your non Drupal site and let Drupal consume it. This would provide the 
ability for authentication. (Drupal does not, to my knowledge, have any 
ability to handle password protected feeds. Someone correct me if I'm 
wrong).

On 11/17/2010 06:12 AM, Victor Kane wrote:
> The migrate module http://drupal.org/project/migrate was designed for
> this kind of problem and is a joy to work with. Depends on a few other
> modules, but one of the end results is that if you can get it onto a
> MySql database, you can work to get views going with it! Etc. etc.
>
> Victor Kane
> http://awebfactory.com.ar
>
> 2010/11/17 Alfonso Montero López <amontero at tinet.org
> <mailto:amontero at tinet.org>>
>
>     Hi listers.
>
>     I have recently got in charge of a non-Drupal website and I need some
>     advice in approaching an issue.
>     The site is about listing learning centers along their training
>     courses and allowing users to request more info by filling a form. No
>     big deal here. You fill the form, data gets dumped into a table and
>     the webmaster (me) receives a mail.
>     All the subsequent workflow is done more or less manually on the table
>     that gets the information requests. However, it's a custom coded PHP
>     site and expanding or trying to automate the workflow any further
>     seems not worth the investment as in its current state, it's kind of a
>     mess. I think it's a good Drupal candidate.
>     The problem is that the website cannot be replaced quickly enough
>     because of its design and look that I'm not able to translate to
>     Drupal. By now, no budget will be allocated to theming and
>     consequently I can't port it to Drupal. Since Drupal still has to
>     prove its worth in the eyes of decision makers. A site rebuild with
>     Drupal from scratch is not [now] an option, since management thinks
>     "hey! we've invested >6 mo and blahblahblah". I would like to shift
>     this by creating an extranet to show what can be done and get some
>     serious attention upstairs :)
>
>     What I'm figuring out how to accomplish is to create a
>     non-public-facing extranet for the learning centers built with Drupal,
>     so I can use nodes to get info requests, route them according my
>     workflow and do all the fancy stuff that Drupal can do with rules,
>     views, etc. This extranet would be used initially by us and later by
>     our customers (the learning centers) and not by site visitors, they
>     should only fill the form in the website exactly as they're doing now.
>     The missing gap is in how to move data from the custom app table to
>     Drupal. Thus, I'll be able to do automatic mails, route leads from one
>     user/departments to another, do reports with views, etc.
>
>     So, here I am thinking in how to approach this one the right way. I've
>     thought in installing a Drupal alongside the current website, so I can
>     access the site's MySQL DB from Drupal and someway and import the
>     records periodically into nodes, but for hosting limitations, it's not
>     possible now.
>
>     The solutions I found are these:
>     *Connecting remotely to MySQL via port 3306 and load new leads. Has
>     firewall issues and may not be supported by all hostings all the time.
>     Also, seems to require quite some programming effort and still have to
>     test Data module capabilities. As long as I can see, data will not be
>     nodes, so I will lose all future workflow, rules, etc. capabilities,
>     right?
>     *Create a [password-protected] RSS feed script for the source site and
>     fetch the leads from it to the Drupal extranet via FeedAPI and alikes.
>     By now, it's the most elegant solution I've found. I assume that
>     FeedAPI won't have problems if I periodically empty the source site
>     leads table and it will gracefully catch up.
>     *(All and every single alternatives are welcome here)
>
>     Obviously, when the extranet is finished and working, I plan to move
>     all to a fully Drupal site and make it hit the front page.
>     BTW, planning to do it in D6.x.
>
>     So, previous experiences and caveats will be very welcomed. I'm sure
>     awesome ideas from the listers here will come up. I learnt a lot by
>     lurking here and in devel during 3 years.
>     Thanks in advance.
>
>     Alfonso.
>     _______________________________________________
>     consulting mailing list
>     consulting at drupal.org <mailto:consulting at drupal.org>
>     http://lists.drupal.org/mailman/listinfo/consulting
>
>
>
>
> _______________________________________________
> consulting mailing list
> consulting at drupal.org
> http://lists.drupal.org/mailman/listinfo/consulting