[consulting] Approaches for retrieving external site's data

Alfonso Montero López amontero at tinet.org
Wed Nov 17 10:47:55 UTC 2010


Hi listers.

I have recently got in charge of a non-Drupal website and I need some
advice in approaching an issue.
The site is about listing learning centers along their training
courses and allowing users to request more info by filling a form. No
big deal here. You fill the form, data gets dumped into a table and
the webmaster (me) receives a mail.
All the subsequent workflow is done more or less manually on the table
that gets the information requests. However, it's a custom coded PHP
site and expanding or trying to automate the workflow any further
seems not worth the investment as in its current state, it's kind of a
mess. I think it's a good Drupal candidate.
The problem is that the website cannot be replaced quickly enough
because of its design and look that I'm not able to translate to
Drupal. By now, no budget will be allocated to theming and
consequently I can't port it to Drupal. Since Drupal still has to
prove its worth in the eyes of decision makers. A site rebuild with
Drupal from scratch is not [now] an option, since management thinks
"hey! we've invested >6 mo and blahblahblah". I would like to shift
this by creating an extranet to show what can be done and get some
serious attention upstairs :)

What I'm figuring out how to accomplish is to create a
non-public-facing extranet for the learning centers built with Drupal,
so I can use nodes to get info requests, route them according my
workflow and do all the fancy stuff that Drupal can do with rules,
views, etc. This extranet would be used initially by us and later by
our customers (the learning centers) and not by site visitors, they
should only fill the form in the website exactly as they're doing now.
The missing gap is in how to move data from the custom app table to
Drupal. Thus, I'll be able to do automatic mails, route leads from one
user/departments to another, do reports with views, etc.

So, here I am thinking in how to approach this one the right way. I've
thought in installing a Drupal alongside the current website, so I can
access the site's MySQL DB from Drupal and someway and import the
records periodically into nodes, but for hosting limitations, it's not
possible now.

The solutions I found are these:
*Connecting remotely to MySQL via port 3306 and load new leads. Has
firewall issues and may not be supported by all hostings all the time.
Also, seems to require quite some programming effort and still have to
test Data module capabilities. As long as I can see, data will not be
nodes, so I will lose all future workflow, rules, etc. capabilities,
right?
*Create a [password-protected] RSS feed script for the source site and
fetch the leads from it to the Drupal extranet via FeedAPI and alikes.
By now, it's the most elegant solution I've found. I assume that
FeedAPI won't have problems if I periodically empty the source site
leads table and it will gracefully catch up.
*(All and every single alternatives are welcome here)

Obviously, when the extranet is finished and working, I plan to move
all to a fully Drupal site and make it hit the front page.
BTW, planning to do it in D6.x.

So, previous experiences and caveats will be very welcomed. I'm sure
awesome ideas from the listers here will come up. I learnt a lot by
lurking here and in devel during 3 years.
Thanks in advance.

Alfonso.


More information about the consulting mailing list