[consulting] Is there a Drupal "Web Crawler"?

[David Hazel]

+1 to Laura's comment.

I've written a crawler in php (non drupal project) and there are lots of
considerations to think of, many of which don't crop up until your adding
"features" to your crawler to make your results meaningful and more
"consumable".

On Tue, Oct 13, 2009 at 6:27 PM, Laura <pinglaura at gmail.com> wrote:

> You're limited by the quality of the search. Much better to create a
> Google search or use some other service where finding is their
> business. Then use aggregator or FeedAPI or some such solution to pull
> in the feed.
>
> Laura
>
> On Oct 13, 2009, at Tue 10/13/09 7:09pm, brendan, fresh-off.com wrote:
>
> > Hello,
> > I have a client that wants to know if there are any Drupal modules
> > that search the web for content related to him and his company, and
> > can then return the results (full articles or links to the content)
> > to his drupal website.  For example, search the web for instances
> > where "john doe" + "XYZ Company" both appear in the same piece of
> > content.
> >
> > Creating the crawler is way beyond my technical ability, so I'm
> > hoping there are some good open source (preferably a Drupal module)
> > options for this functionality.  Wikipedia has a list of open source
> > web crawlers, but since this is a subject I'm unfamiliar with, I'm
> > unsure about whether or not they can be integrated with Drupal - or
> > if any open source web crawlers are even meant to be integrated with
> > a CMS.
> >
> > A little bit more info about the use case: He and his company
> > operate in the education field and are constantly being featured in
> > articles (interviews, write-ups, etc) across the web.  In addition -
> > and most importantly -  he and his company produce several papers/
> > articles that are featured in articles and education related blogs
> > across the internet as well.  He is finding that searching manually
> > for this content to be impractical and thus, would love to have it
> > automatically aggregated and sent to his Drupal site.
> >
> > Any thoughts, ideas, or pointers in the right direction would be
> > apprecaiated!
> >
> >
> > ----
> >
> > brendan, fresh-off.com
> > Creative Direction & Consultation: Web | Print | Brand
> >
> > http://fresh-off.com
> > hello at fresh-off.com
> > 206.328.1067
> >
> >
> > _______________________________________________
> > consulting mailing list
> > consulting at drupal.org
> > http://lists.drupal.org/mailman/listinfo/consulting
>
> _______________________________________________
> consulting mailing list
> consulting at drupal.org
> http://lists.drupal.org/mailman/listinfo/consulting
>



-- 
Email is not a secure form of communication!

Drupal Consultant
http://www.hazelconsulting.com/
253.686.0296
dave at hazelconsulting.com
skype: hazelconsulting
gtalk:kananii
http://www.facebook.com/davidhazel
ICQ: 366587185
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.drupal.org/pipermail/consulting/attachments/20091013/2be9de42/attachment.html