+1 to Laura's comment. <br><br>I've written a crawler in php (non drupal project) and there are lots of considerations to think of, many of which don't crop up until your adding "features" to your crawler to make your results meaningful and more "consumable".<br>
<br><div class="gmail_quote">On Tue, Oct 13, 2009 at 6:27 PM, Laura <span dir="ltr"><<a href="mailto:pinglaura@gmail.com">pinglaura@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
You're limited by the quality of the search. Much better to create a<br>
Google search or use some other service where finding is their<br>
business. Then use aggregator or FeedAPI or some such solution to pull<br>
in the feed.<br>
<font color="#888888"><br>
Laura<br>
</font><div><div></div><div class="h5"><br>
On Oct 13, 2009, at Tue 10/13/09 7:09pm, brendan, <a href="http://fresh-off.com" target="_blank">fresh-off.com</a> wrote:<br>
<br>
> Hello,<br>
> I have a client that wants to know if there are any Drupal modules<br>
> that search the web for content related to him and his company, and<br>
> can then return the results (full articles or links to the content)<br>
> to his drupal website. For example, search the web for instances<br>
> where "john doe" + "XYZ Company" both appear in the same piece of<br>
> content.<br>
><br>
> Creating the crawler is way beyond my technical ability, so I'm<br>
> hoping there are some good open source (preferably a Drupal module)<br>
> options for this functionality. Wikipedia has a list of open source<br>
> web crawlers, but since this is a subject I'm unfamiliar with, I'm<br>
> unsure about whether or not they can be integrated with Drupal - or<br>
> if any open source web crawlers are even meant to be integrated with<br>
> a CMS.<br>
><br>
> A little bit more info about the use case: He and his company<br>
> operate in the education field and are constantly being featured in<br>
> articles (interviews, write-ups, etc) across the web. In addition -<br>
> and most importantly - he and his company produce several papers/<br>
> articles that are featured in articles and education related blogs<br>
> across the internet as well. He is finding that searching manually<br>
> for this content to be impractical and thus, would love to have it<br>
> automatically aggregated and sent to his Drupal site.<br>
><br>
> Any thoughts, ideas, or pointers in the right direction would be<br>
> apprecaiated!<br>
><br>
><br>
> ----<br>
><br>
> brendan, <a href="http://fresh-off.com" target="_blank">fresh-off.com</a><br>
> Creative Direction & Consultation: Web | Print | Brand<br>
><br>
> <a href="http://fresh-off.com" target="_blank">http://fresh-off.com</a><br>
> <a href="mailto:hello@fresh-off.com">hello@fresh-off.com</a><br>
> 206.328.1067<br>
><br>
><br>
</div></div><div><div></div><div class="h5">> _______________________________________________<br>
> consulting mailing list<br>
> <a href="mailto:consulting@drupal.org">consulting@drupal.org</a><br>
> <a href="http://lists.drupal.org/mailman/listinfo/consulting" target="_blank">http://lists.drupal.org/mailman/listinfo/consulting</a><br>
<br>
_______________________________________________<br>
consulting mailing list<br>
<a href="mailto:consulting@drupal.org">consulting@drupal.org</a><br>
<a href="http://lists.drupal.org/mailman/listinfo/consulting" target="_blank">http://lists.drupal.org/mailman/listinfo/consulting</a><br>
</div></div></blockquote></div><br><br clear="all"><br>-- <br>Email is not a secure form of communication!<br><br>Drupal Consultant<br><a href="http://www.hazelconsulting.com/">http://www.hazelconsulting.com/</a><br>253.686.0296<br>
<a href="mailto:dave@hazelconsulting.com">dave@hazelconsulting.com</a><br>skype: hazelconsulting<br>gtalk:kananii<br><a href="http://www.facebook.com/davidhazel">http://www.facebook.com/davidhazel</a><br>ICQ: 366587185<br>