[consulting] Is there a Drupal "Web Crawler"?

[Tony Zielinski]

I was just writing to say the same thing, although I am not aware of a  
feed for Google Web searches, there are feeds for the Google News,  
Blogs and other searches.  I wonder if your client's idea was inspired  
by the "trackback" feature used on some blogs.  The Trackback module  
will automatically post the URL of a referring webpage, which would  
eliminate the remedial task of searching for external sites linking to  
"xyzcomany.com" with the intention of posting it to your site.


On Oct 13, 2009, at 6:27 PM, Laura wrote:

> You're limited by the quality of the search. Much better to create a
> Google search or use some other service where finding is their
> business. Then use aggregator or FeedAPI or some such solution to pull
> in the feed.
>
> Laura
>
> On Oct 13, 2009, at Tue 10/13/09 7:09pm, brendan, fresh-off.com wrote:
>
>> Hello,
>> I have a client that wants to know if there are any Drupal modules
>> that search the web for content related to him and his company, and
>> can then return the results (full articles or links to the content)
>> to his drupal website.  For example, search the web for instances
>> where "john doe" + "XYZ Company" both appear in the same piece of
>> content.
>>
>> Creating the crawler is way beyond my technical ability, so I'm
>> hoping there are some good open source (preferably a Drupal module)
>> options for this functionality.  Wikipedia has a list of open source
>> web crawlers, but since this is a subject I'm unfamiliar with, I'm
>> unsure about whether or not they can be integrated with Drupal - or
>> if any open source web crawlers are even meant to be integrated with
>> a CMS.
>>
>> A little bit more info about the use case: He and his company
>> operate in the education field and are constantly being featured in
>> articles (interviews, write-ups, etc) across the web.  In addition -
>> and most importantly -  he and his company produce several papers/
>> articles that are featured in articles and education related blogs
>> across the internet as well.  He is finding that searching manually
>> for this content to be impractical and thus, would love to have it
>> automatically aggregated and sent to his Drupal site.
>>
>> Any thoughts, ideas, or pointers in the right direction would be
>> apprecaiated!
>>
>>
>> ----
>>
>> brendan, fresh-off.com
>> Creative Direction & Consultation: Web | Print | Brand
>>
>> http://fresh-off.com
>> hello at fresh-off.com
>> 206.328.1067
>>
>>
>> _______________________________________________
>> consulting mailing list
>> consulting at drupal.org
>> http://lists.drupal.org/mailman/listinfo/consulting
>
> _______________________________________________
> consulting mailing list
> consulting at drupal.org
> http://lists.drupal.org/mailman/listinfo/consulting