Hello all. Drupal core aggregator module does not create nodes from feed items. Therefore, aggregated content is effectively limited to lists or blocks. There seem to be (at least) three solutions for parsing RSS feeds from other sites and converting them to nodes (and hence can have taxonomy, comments, voting, ...etc.) - naggregator is the earliest one. It is 4.6 only at present, even in TRUNK. - aggregator2 had a lot of momentum earlier this year, and is 4.7 compatible but not branched for 4.7 so far. - aggregator_node is the latest one. No branch for 4.7 yet. Now, for someone who wants to setup a site using a feed -> node feature, using Drupal 4.7.x, I am looking for first hand experience on pros and cons of each of the above solution, as well as the future direction for each. Feedback appreciated ...
Khalid B wrote:
- aggregator2 had a lot of momentum earlier this year, and is 4.7 compatible but not branched for 4.7 so far.
I've used aggregator2 quite a bit in the past with great success in 4.7. Recently, I've had a couple clients try to use agg2 and say that it was incredible buggy. Following the issue queue, the author seems to have gone AWOL and the module is in disarray. It's got a lot of potential and with some direction and love I think it could be very worthwhile again. There were talks of moving all RSS parsing code to an API module which makes a lot of sense. I think for agg2 to be successful again, the RSS code should be externalized and a few of the major bugs would need to be squashed. I haven't used the other two agg modules. HTH, Rob Roy Barreca Founder and COO Electronic Insight Corporation http://www.electronicinsight.com rob@electronicinsight.com
Thanks Rob Forum posts also indicate that agg2 has significant issues of late. This seems to be the last activity on agg2 on August 11. http://drupal.org/node/77451 Which has an updated version. But since then, the author is AWOL. May this is why budda wrote agg_node?
On 10/4/06, Rob Barreca <rob@electronicinsight.com> wrote:
Khalid B wrote:
- aggregator2 had a lot of momentum earlier this year, and is 4.7 compatible but not branched for 4.7 so far.
I've used aggregator2 quite a bit in the past with great success in 4.7. Recently, I've had a couple clients try to use agg2 and say that it was incredible buggy. Following the issue queue, the author seems to have gone AWOL and the module is in disarray. It's got a lot of potential and with some direction and love I think it could be very worthwhile again.
There were talks of moving all RSS parsing code to an API module which makes a lot of sense. I think for agg2 to be successful again, the RSS code should be externalized and a few of the major bugs would need to be squashed.
There are a bunch of us that have been looking at this recently. Please see https://svn.bryght.com/dev/wiki/DrupalFeedParsing for a summary. budda's feedparser has the cleanest code and looks like a good path forward. However, I would like to see the "feed enabling" approach that David Ayre took under our direction with the "feed_node" module -- see https://svn.bryght.com/dev/browser/feed_node The other big thing is the ability to take different actions based on the namespace. For example, examining the feed, seeing it has GeoRSS extensions, and location-enabling nodes that are based on it. So, the idea behind feed node is that there are many things in core aggregator that can be improved. In working with it, we'll also be contributing core patches back. We may end up maintaining a backported / patched aggregator for 4.7 (...5.0) as we continue to try and improve core as well. Two of the obvious things that comes to mind for core patches are: * paging the admin/aggregator and aggregator/sources pages * making block creation per feed optional, similar to how the quotes module does it agg2 was never an elegant bit of code. The leech / leech_news successor has even more convoluted code. My vote is for helping budda with feedparser, at this point. -- Boris Mann Vancouver 778-896-2747 San Francisco 415-367-3595 Skype borismann http://www.bryght.com
"Boris Mann" wrote:
Please see https://svn.bryght.com/dev/wiki/DrupalFeedParsing for a summary.
budda's feedparser has the cleanest code and looks like a good path forward. However, I would like to see the "feed enabling" approach that David Ayre took under our direction with the "feed_node" module -- see https://svn.bryght.com/dev/browser/feed_node
The site of the above links is down, from my end, anyway. ("No site by that name can be found.") is the result of clicking either link.
The other big thing is the ability to take different actions based on the namespace. For example, examining the feed, seeing it has GeoRSS extensions, and location-enabling nodes that are based on it.
This sounds good. I wonder, however, whether these kinds of features should not be handed off to one of the "trigger-based" modules, like Actions or Workflow. Core surely needs a robust "triggering API", whatever that turns out to be, down the road. -- inkfree
On 10/5/06, inkfree press <inkfree@gmail.com> wrote:
"Boris Mann" wrote:
Please see https://svn.bryght.com/dev/wiki/DrupalFeedParsing for a summary.
budda's feedparser has the cleanest code and looks like a good path forward. However, I would like to see the "feed enabling" approach that David Ayre took under our direction with the "feed_node" module -- see https://svn.bryght.com/dev/browser/feed_node
Works for me, and hasn't been down to my knowledge.
The other big thing is the ability to take different actions based on the namespace. For example, examining the feed, seeing it has GeoRSS extensions, and location-enabling nodes that are based on it.
This sounds good. I wonder, however, whether these kinds of features should not be handed off to one of the "trigger-based" modules, like Actions or Workflow.
Nope. This is based directly on XML namespaces....the namespaces are the triggers. It's a similar concept to Actions, but works differently. Files are similar...different actions depending on MIME type. Menu callbacks and hooks are essentially "triggers". -- Boris Mann Vancouver 778-896-2747 San Francisco 415-367-3595 Skype borismann http://www.bryght.com
Two quick questions: is this replacing Aggregator2? Does it work in 4.7? On 05.Oct.2006, at 10:30, Boris Mann wrote:
On 10/5/06, inkfree press <inkfree@gmail.com> wrote:
"Boris Mann" wrote:
Please see https://svn.bryght.com/dev/wiki/DrupalFeedParsing for a summary.
budda's feedparser has the cleanest code and looks like a good path forward. However, I would like to see the "feed enabling" approach that David Ayre took under our direction with the "feed_node" module -- see https://svn.bryght.com/dev/browser/feed_node
Works for me, and hasn't been down to my knowledge.
The other big thing is the ability to take different actions based on the namespace. For example, examining the feed, seeing it has GeoRSS extensions, and location-enabling nodes that are based on it.
This sounds good. I wonder, however, whether these kinds of features should not be handed off to one of the "trigger-based" modules, like Actions or Workflow.
Nope. This is based directly on XML namespaces....the namespaces are the triggers. It's a similar concept to Actions, but works differently. Files are similar...different actions depending on MIME type.
Menu callbacks and hooks are essentially "triggers".
-- Boris Mann Vancouver 778-896-2747 San Francisco 415-367-3595 Skype borismann http://www.bryght.com
On 10/5/06, blogdiva@culturekitchen.com <blogdiva@culturekitchen.com> wrote:
Two quick questions: is this replacing Aggregator2? Does it work in 4.7?
It doesn't "replace" agg2 (since development on that is AWOL). This is a new, cleaner solution that budda has put some time into. It runs on 4.7, is still in beta mode. -- Boris Mann Vancouver 778-896-2747 San Francisco 415-367-3595 Skype borismann http://www.bryght.com
We're working a little with the developer of agg2/leech and will try to get his thoughts on where it is going in early this week. In regards to (SimplePie.org) is this the _ideal/holy grail_ parser to use at least for RSS/Atom? I know a general concept w/ leech is to handle plugin of different parsers for different purposes, as well as has namespaces in mind, but that's not together yet, and finally, lets you use any node type, so it leech-enables a node. One more, it's got OPML import support. More soon On Oct 5, 2006, at 11:23 AM, Boris Mann wrote:
(since development on that is AWOL).
Ian Ward Development Seed Inc. Technology Development and Discovery http://www.developmentseed.org http://www.developmentseed.org/blog developmentseedperu(skype) Tel. 202.250.3633 Fax. 806.214.6218
On 07 Oct 2006, at 23:13, Ian Ward wrote:
In regards to (SimplePie.org) is this the _ideal/holy grail_ parser to use at least for RSS/Atom?
It is, but it duplicates a lot of code that is already available in core (eg. input validation). We decided against using it in core. -- Dries Buytaert :: http://www.buytaert.net/
I've just updated the wiki w/ info on leech module set here https://svn.bryght.com/dev/wiki/DrupalFeedParsing#LeechandLeechNews On Oct 5, 2006, at 11:23 AM, Boris Mann wrote:
On 10/5/06, blogdiva@culturekitchen.com <blogdiva@culturekitchen.com> wrote:
Two quick questions: is this replacing Aggregator2? Does it work in 4.7?
It doesn't "replace" agg2 (since development on that is AWOL). This is a new, cleaner solution that budda has put some time into. It runs on 4.7, is still in beta mode.
Ian Ward Development Seed Inc. Technology Development and Discovery http://www.developmentseed.org http://www.developmentseed.org/blog developmentseedperu(skype) Tel. 202.250.3633 Fax. 806.214.6218
On Thu, October 5, 2006 12:27 am, Boris Mann said:
On 10/4/06, Rob Barreca <rob@electronicinsight.com> wrote:
Khalid B wrote:
- aggregator2 had a lot of momentum earlier this year, and is 4.7 compatible but not branched for 4.7 so far.
I've used aggregator2 quite a bit in the past with great success in 4.7. Recently, I've had a couple clients try to use agg2 and say that it was incredible buggy. Following the issue queue, the author seems to have gone AWOL and the module is in disarray. It's got a lot of potential and with some direction and love I think it could be very worthwhile again.
There were talks of moving all RSS parsing code to an API module which makes a lot of sense. I think for agg2 to be successful again, the RSS code should be externalized and a few of the major bugs would need to be squashed.
There are a bunch of us that have been looking at this recently.
Please see https://svn.bryght.com/dev/wiki/DrupalFeedParsing for a summary.
How remarkably timely! I've been looking for something along these lines myself lately. Your findings re aggregator and aggregator2 pretty much match my own, althogh I'd not heard of Feedparser or Feed_node. I was also looking into publish/subscribe, but found it buggy. (Automatic push didn't work, and apparently hasn't for months.) What we need to do, essentially, is selectively expose subsets of event nodes on site A to sites B, C, and D. On B, C, and D, the admins need to be able to set the sticky flag on stories independently of the main site, so that each site gets its own set of "featured" stories. Would either feedparser or feed_node be capable of that, and are either stable enough for production use for that use case? If not, is one or the other close enough that we (my company) could help (code or money) getting it to that point? --Larry Garfield
On 10/5/06, Larry Garfield <larry@garfieldtech.com> wrote:
Please see https://svn.bryght.com/dev/wiki/DrupalFeedParsing for a summary.
How remarkably timely! I've been looking for something along these lines myself lately. Your findings re aggregator and aggregator2 pretty much match my own, althogh I'd not heard of Feedparser or Feed_node. I was also looking into publish/subscribe, but found it buggy. (Automatic push didn't work, and apparently hasn't for months.)
Pub/Sub "worksforme" and is the best answer to your stated problem, since it maintains the full structure of Drupal.
What we need to do, essentially, is selectively expose subsets of event nodes on site A to sites B, C, and D. On B, C, and D, the admins need to be able to set the sticky flag on stories independently of the main site, so that each site gets its own set of "featured" stories.
You need the full content of the event to come across. Otherwise you will be writing an event namespace in RSS, then re-parsing it and turning it back into an event node. Pub/Sub.
Would either feedparser or feed_node be capable of that, and are either stable enough for production use for that use case? If not, is one or the other close enough that we (my company) could help (code or money) getting it to that point?
Pub/Sub. Bug JohnVanDyk. Money works, code would be much appreciated as well. -- Boris Mann Vancouver 778-896-2747 San Francisco 415-367-3595 Skype borismann http://www.bryght.com
Boris Mann wrote:
agg2 was never an elegant bit of code. The leech / leech_news successor has even more convoluted code. My vote is for helping budda with feedparser, at this point.
Can feedparser handled http-auth protected sites/feeds, as well as SSL? Or can those abilities be added easily? They're critical to me. :-)
Op donderdag 5 oktober 2006 05:51, schreef Khalid B:
- naggregator is the earliest one. It is 4.6 only at present, even in TRUNK.
- aggregator2 had a lot of momentum earlier this year, and is 4.7 compatible but not branched for 4.7 so far.
- aggregator_node is the latest one. No branch for 4.7 yet.
Now, for someone who wants to setup a site using a feed -> node feature, using Drupal 4.7.x, I am looking for first hand experience on pros and cons of each of the above solution, as well as the future direction for each.
Feedback appreciated ...
Please start here: http://cvs.drupal.org/viewcvs/drupal/contributions/modules/feedparser/ Its meant to be a feedparsting lib, that can be used by any modyule to do anything. Including making nodes from items and feeds :) I meant to rewrite naggregator to use this lib, the foudation of the lib comes from naggregator. Bèr -- [ Bèr Kessels | Drupal services www.webschuur.com ] Nieuwsartikelen, of weblog.: http://help.sympal.nl/nieuwsartikelen_of_weblog Written while listening to Adonis - We're Rocking Down th by Ru-D on
participants (10)
-
blogdiva＠culturekitchen.com -
Boris Mann -
Bèr Kessels -
Chris Johnson -
Dries Buytaert -
Ian Ward -
inkfree press -
Khalid B -
Larry Garfield -
Rob Barreca