> Are you saying the header Connection: Close is ignored?<br>> <br>> Any reason why Drupal should not use HTTP/1.1?<br><br>Examining the drupal_http_request I found that it actually doesn't use HTTP 1.1 in the first place, so I guess there's no problem in using it at all. But I would still like to maintain the ability to transparently accept feeds from HTTP or FTP as well as providing the users the option to access authenticated URLs. I don't know how widely used these features are, but I'd hate to remove a feature that could help a user out.
<br><br>Seems we were guilty in assuming what SimplePie did during these 11 seconds. Although I still think it's going about it the wrong way. 11 seconds is suicide. I sanitize against the extracted data, rather than the feed string as a whole. That's what I presume SimplePie is doing. I wish I could check it out for myself but my sleep indicators are overloaded.
<br><br>Morbus' suggestion to pass along the string as whole sounds logical, I'll see what I can do about that. Although I really had assumed that aggregation happens from XMLs only so the module would need a considerable amount of change to accommodate non-XML strings. I'll study the option and see what I can do. Anyone care to give me a patch for my next birthday? :-P
<br><br><br>AA<br><br><div><span class="gmail_quote">On 6/20/07, <b class="gmail_sendername">Morbus Iff</b> <<a href="mailto:morbus@disobey.com" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">morbus@disobey.com
</a>> wrote:</span><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
> opinion the second is not sanitization and no aggregator needs to waste<br>> the code and time on trying to handle non-XML or non-standards compliant<br><br>It depends entirely on your definition of "aggregator". In your module,
<br>you have only one parser, really - PHP's SimpleXML (or whatever it's<br>called) that then sends the loaded data structure to the smaller "do<br>things with it" (ie., RSS20.inc, etc.) subparsers. However, I'd think
<br>that it'd be far more flexible to send the raw strings around /as well/<br>- then one could support, for example, non-XML documents (or, in my<br>particular case, I could write scrapers for sites that don't support
<br>feeds [or feeds that contain useful data]) so that I'd be able to hook<br>into the generic aggregating process. Aggregation != just XML, IMO.<br><br>I'd love, for example, to be able to add a "feed" that points to (pff,
<br>making crap outta my ass here) some comic site's "latest comic" HTML,<br>choose a custom-made parser that expects that HTML, and return the same<br>data structure that the aggregation API expects as legit. This /is/
<br>aggregation - pulling disparate sources together.<br><br>> I would be very surprised if I found that SimplePie is wasting 11<br>> seconds out of 12 in preventing XSS or SQL injection attacks alone. But<br>> hey, what do I know about SimplePie. Does anyone know what SimplePie
<br>> actually does within these 11 seconds?<br><br>SimplePie's set_stupidly_fast is a wrapper around:<br><br> $this->enable_order_by_date(false);<br> $this->remove_div(false);<br> $this->strip_comments(false);
<br> $this->strip_htmltags(false);<br> $this->strip_attributes(false);<br> $this->set_image_handler(false);<br><br>None of those are "fix broken XML". I reran the initial test like so:<br><br> $feed->set_stupidly_fast(TRUE);
<br> $feed->enable_order_by_date(TRUE);<br><br>i.e. first shutting everything off, then enabling one command:<br><br> $feed->enable_order_by_date(TRUE) 2 seconds<br> $feed->remove_div(TRUE) 1 second
<br> $feed->strip_comments(TRUE); 2 seconds<br> $feed->strip_htmltags(TRUE); 2 seconds<br> $feed->strip_attributes(TRUE); 2 seconds<br> $feed->set_image_handler(TRUE); 1 second
<br><br>--<br>Morbus Iff ( if god is my witness, god must be blind )<br>Technical: <a href="http://www.oreillynet.com/pub/au/779" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">http://www.oreillynet.com/pub/au/779
</a><br>Culture: <a href="http://www.disobey.com/" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">http://www.disobey.com/
</a> and <a href="http://www.gamegrene.com/" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">http://www.gamegrene.com/</a><br>aim: akaMorbus / skype: morbusiff / icq: 2927491 / <a href="http://jabber.org" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">
jabber.org</a>: morbus<br></blockquote></div><br>