[consulting] Is there a Drupal "Web Crawler"?

[Laura]

You're limited by the quality of the search. Much better to create a  
Google search or use some other service where finding is their  
business. Then use aggregator or FeedAPI or some such solution to pull  
in the feed.

Laura

On Oct 13, 2009, at Tue 10/13/09 7:09pm, brendan, fresh-off.com wrote:

> Hello,
> I have a client that wants to know if there are any Drupal modules  
> that search the web for content related to him and his company, and  
> can then return the results (full articles or links to the content)  
> to his drupal website.  For example, search the web for instances  
> where "john doe" + "XYZ Company" both appear in the same piece of  
> content.
>
> Creating the crawler is way beyond my technical ability, so I'm  
> hoping there are some good open source (preferably a Drupal module)  
> options for this functionality.  Wikipedia has a list of open source  
> web crawlers, but since this is a subject I'm unfamiliar with, I'm  
> unsure about whether or not they can be integrated with Drupal - or  
> if any open source web crawlers are even meant to be integrated with  
> a CMS.
>
> A little bit more info about the use case: He and his company  
> operate in the education field and are constantly being featured in  
> articles (interviews, write-ups, etc) across the web.  In addition -  
> and most importantly -  he and his company produce several papers/ 
> articles that are featured in articles and education related blogs  
> across the internet as well.  He is finding that searching manually  
> for this content to be impractical and thus, would love to have it  
> automatically aggregated and sent to his Drupal site.
>
> Any thoughts, ideas, or pointers in the right direction would be  
> apprecaiated!
>
>
> ----
>
> brendan, fresh-off.com
> Creative Direction & Consultation: Web | Print | Brand
>
> http://fresh-off.com
> hello at fresh-off.com
> 206.328.1067
>
>
> _______________________________________________
> consulting mailing list
> consulting at drupal.org
> http://lists.drupal.org/mailman/listinfo/consulting