This is a follow-up to the PHP 5 thread from a week or two ago. It looks like some momentum is building. Ken Rickard, Robert Douglas, and I have been talking with some of the Jooma folks, and have a working draft of the "core statement and justification". That is, what the goal is and why it is. Joomla's development team is discussing the matter and is leaning yes. Based on the earlier thread here I am hoping that there isn't much objection to Drupal participating in the "Go PHP5" effort. :-) So far Joomla is leaning yes, CakePHP is interested, and I had a positive first response from Typo3. Robert Douglas has volunteered himself to setup a web site for it. I'm not sure how Dries wants to handle the question of Drupal's participation (by vote, by consensus, or by fiat). Dries? Anyway, here's the working statement. Consider this an official recommendation that Drupal commit to participating in this effort. ------------------------------------ PHP 4 has served the web developer community for seven years now, and served it well. However, it also shows its age. Most of PHP 4's shortcomings have been addressed by PHP 5, released three years ago, but the transition from PHP 4 to PHP 5 has been slow for a number of reasons. PHP developers cannot leverage PHP 5's full potential without dropping support for PHP 4, but PHP 4 is still installed on a majority of shared web hosts and users would then be forced to switch to a different application. Web hosts cannot upgrade their servers to PHP 5 without making it impossible for their users to run PHP 4-targeted web apps, and have no incentive to go to the effort of testing and deploying PHP 5 while most web apps are still compatible with PHP 4 and the PHP development team still provides maintenance support for PHP 4. The PHP development team, of course, can't drop maintenance support for PHP 4 while most web hosts still run PHP 4. It is a dangerous cycle, and one that needs to be broken. The open source PHP developer community has decided that it is indeed now time to move forward, together. Therefore, the listed open source PHP projects have all agreed that effective 5 February 2008, any new feature release will have a minimum required PHP version no older than PHP 5.2.0. It is our believe that this will allow web hosts a reason to upgrade and the PHP development team the ability to retire PHP 4 and focus efforts on PHP 5 and the forthcoming PHP 6, all without penalizing any existing project for being "first out of the gate". ------------------------------------ Notes: - I chose the date because I figure that will be 7-8 months after we officially announce this thing, which I believe should be sufficient time for web hosts. It also comes out to 5/2/2008 (using European convention), and I just like inside references like that. :-) - This does not preclude any project from moving before the deadline date, or from supporting older versions for however long they wish to. That's up to each project. - PHP 5.2 is already the most widely installed version of PHP 5, based on the latest published stats. I know at least two web hosts I work with that either have jumped or are in the process of jumping from PHP 4 straight to PHP 5.2. By the target date it will have been out for nearly a year and a half. It also adds a number of new and useful core features (filter_input, json, a stable PDO, etc.). It's a good version to target. Thoughts? Comments? Support? Rotten tomatoes? -- Larry Garfield AIM: LOLG42 larry@garfieldtech.com ICQ: 6817012 "If nature has made any one thing less susceptible than all others of exclusive property, it is the action of the thinking power called an idea, which an individual may exclusively possess as long as he keeps it to himself; but the moment it is divulged, it forces itself into the possession of every one, and the receiver cannot dispossess himself of it." -- Thomas Jefferson
I just wanted to rehash what Larry said (because saying it many different ways best avoids confusion). We are preparing a public statement of intent. The statement is to be supported by PHP open source projects and it is to reflect their intentions. The entire statement is that by the 5th of February, 2008, the software projects involved will no longer see the necessity of writing software that avoids language features and constructs that are available in PHP5 but not PHP4. Dries has written very clearly that PHP adoption is a real source for concern: http://buytaert.net/php-is-dead-long-live-php Many people on this list have also voiced frustration with PHP4 shackles. The main goals of this public statement are to a) help set the direction of development across many projects, and b) send a loud and clear message to companies who offer web hosting services. -Robert
On 6/6/07 3:33 AM, Robert Douglass wrote:
I just wanted to rehash what Larry said (because saying it many different ways best avoids confusion). We are preparing a public statement of intent. The statement is to be supported by PHP open source projects and it is to reflect their intentions. The entire statement is that by the 5th of February, 2008, the software projects involved will no longer see the necessity of writing software that avoids language features and constructs that are available in PHP5 but not PHP4.
Dries has written very clearly that PHP adoption is a real source for concern: http://buytaert.net/php-is-dead-long-live-php
Many people on this list have also voiced frustration with PHP4 shackles.
The main goals of this public statement are to a) help set the direction of development across many projects, and b) send a loud and clear message to companies who offer web hosting services.
great initiative, guys! Rasmus kinda threw down the gauntlet at OSCMS about how to get php5 adoption going. Nice to see the various communities coming together and unifying on this (that was one of the initial reasons for OSCMS after all). Go Robert and Larry Go! +1 Rah rah! ;) -- James Walker :: http://walkah.net/ :: xmpp:walkah@walkah.net
Adding to the chorus here -- at OSCMS, Rasmus seemed to imply that he would look kindly on initiatives that sped the adoption of PHP5 -- This is a great idea/effort, both in the way that it brings different open source communities together and in how it will eventually result in better functionality for end users. +1 , rah, etc. Cheers, Bill James Walker wrote:
great initiative, guys! Rasmus kinda threw down the gauntlet at OSCMS about how to get php5 adoption going. Nice to see the various communities coming together and unifying on this (that was one of the initial reasons for OSCMS after all).
Go Robert and Larry Go! +1 Rah rah! ;)
-- Bill Fitzgerald http://www.funnymonkey.com Tools for Teachers 503.897.7160
Are there currently that many PHP4 apps that won't run on PHP5? I thought Drupal already does. Why can't ISPs upgrade without breaking older apps? I am a huge fan of this idea overall, BTW. I think migrating apps to PHP5 will start forcing ISPs to at least provide the option to upgrade since they'll start losing customers if they don't. At my company, we own our own servers so it'd be relatively easy for us to upgrade. Larry Garfield wrote:
This is a follow-up to the PHP 5 thread from a week or two ago. It looks like some momentum is building. Ken Rickard, Robert Douglas, and I have been talking with some of the Jooma folks, and have a working draft of the "core statement and justification". That is, what the goal is and why it is. Joomla's development team is discussing the matter and is leaning yes. Based on the earlier thread here I am hoping that there isn't much objection to Drupal participating in the "Go PHP5" effort. :-) So far Joomla is leaning yes, CakePHP is interested, and I had a positive first response from Typo3. Robert Douglas has volunteered himself to setup a web site for it.
I'm not sure how Dries wants to handle the question of Drupal's participation (by vote, by consensus, or by fiat). Dries?
Anyway, here's the working statement. Consider this an official recommendation that Drupal commit to participating in this effort.
------------------------------------ PHP 4 has served the web developer community for seven years now, and served it well. However, it also shows its age. Most of PHP 4's shortcomings have been addressed by PHP 5, released three years ago, but the transition from PHP 4 to PHP 5 has been slow for a number of reasons.
PHP developers cannot leverage PHP 5's full potential without dropping support for PHP 4, but PHP 4 is still installed on a majority of shared web hosts and users would then be forced to switch to a different application. Web hosts cannot upgrade their servers to PHP 5 without making it impossible for their users to run PHP 4-targeted web apps, and have no incentive to go to the effort of testing and deploying PHP 5 while most web apps are still compatible with PHP 4 and the PHP development team still provides maintenance support for PHP 4. The PHP development team, of course, can't drop maintenance support for PHP 4 while most web hosts still run PHP 4.
It is a dangerous cycle, and one that needs to be broken. The open source PHP developer community has decided that it is indeed now time to move forward, together. Therefore, the listed open source PHP projects have all agreed that effective 5 February 2008, any new feature release will have a minimum required PHP version no older than PHP 5.2.0. It is our believe that this will allow web hosts a reason to upgrade and the PHP development team the ability to retire PHP 4 and focus efforts on PHP 5 and the forthcoming PHP 6, all without penalizing any existing project for being "first out of the gate". ------------------------------------
Notes: - I chose the date because I figure that will be 7-8 months after we officially announce this thing, which I believe should be sufficient time for web hosts. It also comes out to 5/2/2008 (using European convention), and I just like inside references like that. :-) - This does not preclude any project from moving before the deadline date, or from supporting older versions for however long they wish to. That's up to each project. - PHP 5.2 is already the most widely installed version of PHP 5, based on the latest published stats. I know at least two web hosts I work with that either have jumped or are in the process of jumping from PHP 4 straight to PHP 5.2. By the target date it will have been out for nearly a year and a half. It also adds a number of new and useful core features (filter_input, json, a stable PDO, etc.). It's a good version to target.
Thoughts? Comments? Support? Rotten tomatoes?
-- Sean Robertson Web Developer NGP Software, Inc. seanr@ngpsoftware.com (202) 686-9330 http://www.ngpsoftware.com
Sean Robertson wrote:
Are there currently that many PHP4 apps that won't run on PHP5? I thought Drupal already does. That's not the problem. The problem is that PHP5 has new features which we can't use.
Why can't ISPs upgrade without breaking older apps?
They can, and we want them to. They're just lazy, afaik.
I am a huge fan of this idea overall, BTW. I think migrating apps to PHP5 will start forcing ISPs to at least provide the option to upgrade since they'll start losing customers if they don't.
That's the point.
On Jun 6, 2007, at 7:45 AM, Robert Douglass wrote:
Sean Robertson wrote:
Why can't ISPs upgrade without breaking older apps?
They can, and we want them to. They're just lazy, afaik.
Indeed, if I might interject a point here to support this idea. While it probably is lazyness to some degree, what I've found to be true with the vast major of hosting companies is that they are running off of the stock Control Panel applications, and many of them are running on SW-Soft's products (Plesk and Virtuozzo). As a former Gold customer (recovering, we're now using mostly XEN, VMWARE, and some other higher end systems for everything), I know that the support policy for SW-Soft with regards to PHP is the OS Support Upgrade Policy (that's what they called it, I've never seen it actually). They will provide support for only whatever the vendor has released on the platform, including any extended applications like PHP and MySQL. For hosting companies with any kind of integrated system, or those in need of second tier support from SW-Soft, doing an upgrade can invalidate their support agreements (you're always riding the line on this one if you need "advanced upgrades," aka something that's relatively new) and break their setups. In fact, doing an upgrade with SW-Soft's popular HSP Complete system of php or mysql will almost certainly functionally break the entire control panel. In addition, it's extremely difficult to do a full OS upgrade with many contol panel solutions, as they're using a lot of custom rpms and the like, so things tend to stay in place and get patched. For example, with HSP Complete (again, just an example), it's not actually possible to do an upgrade - you have to delete the customer virtuozzo environment and re-create it since it tracks rpms granularly. So, we're moving at the speed of RedHat. Actually, most times, RedHat running through CentOS. And since so many of the upgrades are actually attached to the OS Upgrades, and since it's difficult to do an OS Upgrade, things tend to stagnate across the entire industry. Occasionally some brave sysadmin will stand up and start releasing packages. For big hosting customers with a lot of purchase power (think GoDaddy, 1&1, BlueHost, etc), this is absolutely an option. But for the long tail, obstacles remain. But most big companies will only patch when they have to, and leave the support up to companies like SW-Soft. So, it's not as much the hosting companies that need convincing - it's the vendors that write the Control Panels that have the real authority - they're the ones that need convincing. And I don't think they're planning to fix some of the upgrade os problems anytime soon (just ask me the difference between an insert and overlay control panel - my own terms - sometime). I think it would be worth rallying around the flag on this one, and you'll certainly get link juice from me about it, but I don't thin it's a question of hosting company lazyness exclusively.
I am a huge fan of this idea overall, BTW. I think migrating apps to PHP5 will start forcing ISPs to at least provide the option to upgrade since they'll start losing customers if they don't. That's the point.
Indeed. Jonathan
On Wed, 6 Jun 2007 09:12:25 -0700, Jonathan Lambert <j@firebright.com> wrote:
So, it's not as much the hosting companies that need convincing - it's the vendors that write the Control Panels that have the real authority - they're the ones that need convincing. And I don't think they're planning to fix some of the upgrade os problems anytime soon (just ask me the difference between an insert and overlay control panel - my own terms - sometime).
I think it would be worth rallying around the flag on this one, and you'll certainly get link juice from me about it, but I don't thin it's a question of hosting company lazyness exclusively.
I am a huge fan of this idea overall, BTW. I think migrating apps to PHP5 will start forcing ISPs to at least provide the option to upgrade since they'll start losing customers if they don't. That's the point.
Indeed.
Jonathan
If the knock-on result is that hosts put pressure on lazy Control Panel companies to get with the 21st century lest the Control Panel companies start losing business, I am perfectly happy with that outcome. :-) The stagnation needs to break somewhere. --Larry Garfield
So, we're moving at the speed of RedHat. Actually, most times, RedHat running through CentOS. And since so many of the upgrades are actually attached to the OS Upgrades, and since it's difficult to do an OS Upgrade, things tend to stagnate across the entire industry.
The CentOS team is pleased to announce the availability of CentOS 5.0. Major changes in CentOS 5 compared to CentOS 4 include: These updated software versions: Apache-2.2, php-5.1.6 ... RHEL 5 has moved all the way up to PHP 5.1.6 from RHEL 4's mere PHP 4.3.9 We are getting there.
I think the whole idea here is to break that catch-22 situation. If the largest PHP application say they will move off PHP4, then the users/hosts/distros will each pressure/nudge the other entities to move faster, rather than the chicken-egg we are in today. So, +1 on this initiative. What is the worst case? Come February and we see that things have not moved. We evaluate it them and see what other projects did, and decide either to stay the course, or go back to PHP4. So, the risk is not high in such a move. Unless we (all CMS's that is) try, we will never know if it can succeed. -- 2bits.com http://2bits.com Drupal development, customization and consulting.
On 07.06.2007, at 04:15, Khalid Baheyeldin wrote:
So, +1 on this initiative. What is the worst case? Come February and we see that things have not moved. We evaluate it them and see what other projects did, and decide either to stay the course, or go back to PHP4.
I think this is a bad idea. If we go into this process with this attitude, we won't honestly try to get providers to PHP 5 because there is no need to for them – they will know that software projects won't move to 5 when the majority of providers still run 4. When we really make Drupal (and all the other projects) PHP 5 only starting from this date, providers won't have a choice but to upgrade. Konstantin Käfer – http://kkaefer.com/
On 6/9/07, Konstantin Käfer <kkaefer@gmail.com> wrote:
On 07.06.2007, at 04:15, Khalid Baheyeldin wrote:
So, +1 on this initiative. What is the worst case? Come February and we see that things have not moved. We evaluate it them and see what other projects did, and decide either to stay the course, or go back to PHP4.
I think this is a bad idea. If we go into this process with this attitude, we won't honestly try to get providers to PHP 5 because there is no need to for them – they will know that software projects won't move to 5 when the majority of providers still run 4. When we really make Drupal (and all the other projects) PHP 5 only starting from this date, providers won't have a choice but to upgrade.
We don't need to publicize that there is a possibility that we can go back in our decision. This is a reality check that leaves a fallback option open. If February comes and nothing changed as far as distros and hosting companies are concerned. What are we going to do then? Just keep forging ahead as if PHP4 does not exist, and corner ourselves? We can say that we reevaluate this in February and that is it. We don't need to make any decisions now (apart from pushing hard for hosts and distros to go PHP5). -- 2bits.com http://2bits.com Drupal development, customization and consulting.
We don't need to publicize that there is a possibility that we can go back in our decision.
We just said that. I think it is necessary to *force* all participating projects to switch to PHP 5 only, in all circumstances. If we just give in when nothing has changed in february 2008, we end up in the same state as we are now.
Just keep forging ahead as if PHP4 does not exist, and corner ourselves?
If all participating projects (most major ones showed interest) use PHP 5 from 2008 on, we are not cornering ourselves as Drupal's competitors are also PHP5 only.
We can say that we reevaluate this in February and that is it. We don't need to make any decisions now (apart from pushing hard for hosts and distros to go PHP5).
Again, if hosters don't have a reason (= major PHP software is PHP 5 only) to upgrade, they won't. I think it is necessary to decide *now* to give everyone half a year's notice. Konstantin Käfer – http://kkaefer.com/
This discussion brings to mind Thomas Schelling's landmark game theory book, "The Strategy of Conflict." What we have here is a game of coordinated bargaining. Two related quotations from the first chapter: "We have learned that a threat has to be credible to be efficacious, and that its credibility may depend on the costs and risks associated with fulfillment for the party making the threat." "While prudence suggests leaving open a way of escape when one threatens an adversary with mutually painful reprisal, any visible means of escape may make the threat less credible." So, a coordinated shift to PHP5 is more persuasive if there is no possibility for reevaluation. This brings to mind some possible tactics for Drupal to pursue: * Provide a chance for distributions and hosting companies to give feedback on the proposal, prior to the final commitment. * Provide adequate time for distributions and hosting companies to upgrade (this is where the input process will also help). * Recruit as many participating projects as possible. * Secure concrete, irreversible agreements from all participating projects. * Clearly communicate the rationale and benefits of the decision, citing external, legitimate authorities if possible (e.g. Zend). * Secure internal commitment by adding code, or reimplementing old code, in ways that eliminate the possibility of reverting to PHP 4. * Measure the extent to which its community is affected (e.g. by including a question on php4 vs. php5 in the drupal.org user survey). * Reduce migration costs by providing documentation or even automated testing and conversion (coder.module, unit test automation, etc.). * Promote (communicating to its users) distributions and hosting companies that provide PHP 5, and penalize or deemphasize those that do not. .ck Konstantin Käfer wrote:
We don't need to publicize that there is a possibility that we can go back in our decision.
We just said that. I think it is necessary to *force* all participating projects to switch to PHP 5 only, in all circumstances. If we just give in when nothing has changed in february 2008, we end up in the same state as we are now.
Just keep forging ahead as if PHP4 does not exist, and corner ourselves?
If all participating projects (most major ones showed interest) use PHP 5 from 2008 on, we are not cornering ourselves as Drupal's competitors are also PHP5 only.
We can say that we reevaluate this in February and that is it. We don't need to make any decisions now (apart from pushing hard for hosts and distros to go PHP5).
Again, if hosters don't have a reason (= major PHP software is PHP 5 only) to upgrade, they won't. I think it is necessary to decide *now* to give everyone half a year's notice.
Konstantin Käfer – http://kkaefer.com/
On Saturday 09 June 2007, Chris Kennedy wrote:
So, a coordinated shift to PHP5 is more persuasive if there is no possibility for reevaluation.
This brings to mind some possible tactics for Drupal to pursue: * Provide a chance for distributions and hosting companies to give feedback on the proposal, prior to the final commitment.
I have been thinking in the past few days if contacting selected hosts to get their feedback as well would be a good idea. Allie mentioned[1] that she runs a web hosting company and supported this idea because it would make her life simpler, too. Allie, your input here would be hugely welcome. What's a good way to go about getting your company and other web hosts on board? [1] http://lists.drupal.org/archives/development/2007-05/msg00465.html
* Provide adequate time for distributions and hosting companies to upgrade (this is where the input process will also help).
* Recruit as many participating projects as possible. * Secure concrete, irreversible agreements from all participating projects.
I am actively working on these points, which is what this thread is about. :-) If anyone has suggestions for projects that we should contact, let me know. So far my hitlist consists of Drupal, Joomla, CakePHP, Symfony (they say they're in), Symfony's partner projects, Gallery, and WordPress (Dries said he talked to Matt and he's being stubborn, but we need to keep on it).
* Clearly communicate the rationale and benefits of the decision, citing external, legitimate authorities if possible (e.g. Zend).
We should see if we can get a quote or comment out of Rasmus, too, especially giving how much he was haranguing on us about this at DrupalCon. :-)
* Secure internal commitment by adding code, or reimplementing old code, in ways that eliminate the possibility of reverting to PHP 4.
Agreed. I think by just explicitly allowing devs to use PHP 5 features this will happen naturally for Drupal.
* Measure the extent to which its community is affected (e.g. by including a question on php4 vs. php5 in the drupal.org user survey). * Reduce migration costs by providing documentation or even automated testing and conversion (coder.module, unit test automation, etc.).
The GoPHP5.org site can probably include some guides and tips for making sure code is PHP 5-friendly.
* Promote (communicating to its users) distributions and hosting companies that provide PHP 5, and penalize or deemphasize those that do not.
One of the plans for the web site is a listing of web hosts that offer PHP 5.2 as their default setup for new sites (with appropriate "we aren't endorsing or saying these work with every project, just that they're playing nice" text to CYA). Agreed on all points down the line, Chris! -- Larry Garfield AIM: LOLG42 larry@garfieldtech.com ICQ: 6817012 "If nature has made any one thing less susceptible than all others of exclusive property, it is the action of the thinking power called an idea, which an individual may exclusively possess as long as he keeps it to himself; but the moment it is divulged, it forces itself into the possession of every one, and the receiver cannot dispossess himself of it." -- Thomas Jefferson
Larry Garfield wrote:
On Saturday 09 June 2007, Chris Kennedy wrote:
So, a coordinated shift to PHP5 is more persuasive if there is no possibility for reevaluation.
This brings to mind some possible tactics for Drupal to pursue: * Provide a chance for distributions and hosting companies to give feedback on the proposal, prior to the final commitment.
I have been thinking in the past few days if contacting selected hosts to get their feedback as well would be a good idea. Allie mentioned[1] that she runs a web hosting company and supported this idea because it would make her life simpler, too. Allie, your input here would be hugely welcome. What's a good way to go about getting your company and other web hosts on board?
As soon as there is some sort of public announcement, I've got a well place contact who straddles marketing/support of a largish hosting company. I *could* test the waters with these guys... (no, can we just start?) ... well, at your service either way. :)
FWIW, Zend Framework already required PHP 5.1.4 today (1.0RC1). Given Zend's role in PHP overall, especially towards isolated developers, it might be interesting to suggest they make the move at the same time as community projects.
2007/6/10, Larry Garfield <larry@garfieldtech.com>:
If anyone has suggestions for projects that we should contact, let me know.
How about purely administrative projects that aren't meant to face as much towards the visitors of a site as to the people behind it? I'm thinking phpMyAdmin, phpPgAdmin, php(other databases)Admin, web-based system administration applications (I know they're out there, but can't remember the names), and similar. Also, webmails clients such as SquirrelMail and groupware applications like Horde should also be contacted. Finally, going through http://en.wikipedia.org/wiki/Category:PHP_programming_language probably wouldn't hurt either. ;) -- Frederik 'Freso' S. Olesen <http://freso.dk/>
On Saturday 09 June 2007, Khalid Baheyeldin wrote:
We don't need to publicize that there is a possibility that we can go back in our decision.
This is a reality check that leaves a fallback option open. If February comes and nothing changed as far as distros and hosting companies are concerned. What are we going to do then? Just keep forging ahead as if PHP4 does not exist, and corner ourselves?
We can say that we reevaluate this in February and that is it. We don't need to make any decisions now (apart from pushing hard for hosts and distros to go PHP5).
It's really disingenuous to plan for optionally changing our minds later. If all we do come February is say "we only support PHP 5 now" but we've still made sure the code works on PHP 4, then that's frankly dishonest. Rather, whatever version we expect to ship next after that date we should permit ourselves to use PHP 5-only features. There's plenty of places in core where that would greatly simplify things, but using PHP 5-specific features, naturally, means breaking PHP 4. (That's the evil cycle that we're all caught in right now.) And there's no reasonable way to then "undo" that if we decide "oh well, it wasn't quite as successful as we hoped, I guess we'll back out". It would be completely infeasible, not to mention destroying any credibility Drupal has in the OSS world. "Wait and see" means "don't participate at all". That would be really unfortunate, as this effort cannot be successful without big names like Drupal. -- Larry Garfield AIM: LOLG42 larry@garfieldtech.com ICQ: 6817012 "If nature has made any one thing less susceptible than all others of exclusive property, it is the action of the thinking power called an idea, which an individual may exclusively possess as long as he keeps it to himself; but the moment it is divulged, it forces itself into the possession of every one, and the receiver cannot dispossess himself of it." -- Thomas Jefferson
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Khalid Baheyeldin schrieb:
On 6/9/07, Konstantin Käfer <kkaefer@gmail.com> wrote:
On 07.06.2007, at 04:15, Khalid Baheyeldin wrote:
So, +1 on this initiative. What is the worst case? Come February and we see that things have not moved. We evaluate it them and see what other projects did, and decide either to stay the course, or go back to PHP4.
I think this is a bad idea. If we go into this process with this attitude, we won't honestly try to get providers to PHP 5 because there is no need to for them – they will know that software projects won't move to 5 when the majority of providers still run 4. When we really make Drupal (and all the other projects) PHP 5 only starting from this date, providers won't have a choice but to upgrade.
We don't need to publicize that there is a possibility that we can go back in our decision.
Well, you already kind of did. I think that the first patch that goes in after HEAD will be open for development again should be one that cleans up some corners of the code where specific allowance for PHO 4 has been made. (I only know of drupal_clone, though.) Then, we could rewrite our RSS generator to use SimpleXML. If we proceed like this, there will be no way we could revert to support PHP 4 in February 2008. I think that the code freeze for Drupal 7 will be some time early next year. D7 would then be the first release that would make PHP 5.2 mandatory. D8 would then be released late in 2008 or early in 2009. Only when we release D8, D6 will become unsupported and everybody who wants a supported Drupal version will actually _have_ to upgrade to PHP 5.2. I think that having 1.5 years advance warning is plenty and hosters who dont heed it well deserve to go out of business. Cheers, Gerhard -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) iD8DBQFGbRV1fg6TFvELooQRAua0AJ9txrgbnjCWAr6U2D+ypvZizc4XcgCbBuK7 5Px/CLP+kiGkD2iwSIMrcMc= =0/Xu -----END PGP SIGNATURE-----
On 11 Jun 2007, at 11:27, Gerhard Killesreiter wrote:
If we proceed like this, there will be no way we could revert to support PHP 4 in February 2008.
If I can believe my own projections (http://buytaert.net/php-is-dead- long-live-php), PHP5 will have a 30% adoption rate by February 2008, rather than the current 20%. Maybe February 2008 is a little too early? -- Dries Buytaert :: http://www.buytaert.net/
On 6/18/07, Dries Buytaert <dries.buytaert@gmail.com> wrote:
On 11 Jun 2007, at 11:27, Gerhard Killesreiter wrote:
If we proceed like this, there will be no way we could revert to support PHP 4 in February 2008.
If I can believe my own projections (http://buytaert.net/php-is-dead- long-live-php), PHP5 will have a 30% adoption rate by February 2008, rather than the current 20%. Maybe February 2008 is a little too early?
Not if we can help push it ;) --
Dries Buytaert :: http://www.buytaert.net/
-- Regards, Johan Forngren :: http://johan.forngren.com/
Not if we can help push it ;)
Right. We want to help PHP 5 adoption not follow it. We are now big enough to be the leader. We are so deep in OO territory already that we might as well begin using the class construct... in due time, we can retire JonBob's excellent how Drupal is OOP essay. I think Barcelona would be a great time to draft what we can gain with PHP5 OOP constructs... even I have ideas...
I was having a little chat about PHP5 support with a guy at midphase.com. Off the record, at this stage, they generally support the movement, but the main problem for them with *pushing* PHP5 is Fantastico. Had a little look around. Here is a list of scripts supported by Fantastico, along with their PHP5 status. Hope this is useful. http://tdknights.com/fantastico.htm (It was obtained from this thread (Fantastico forum): http://www.netenberg.com/forum/viewtopic.php?t=3933) .s
I'll add those projects to my contact list, thanks. :-) Most of them it looks like already run in PHP 5. For the few that don't, excuse me for being blunt but as far as I'm concerned those projects are abandoned. Not even running on PHP 5 now, after 3 years, is simply inexcusable. (Although I know that at least one or two of those are working on new PHP 5-friendly or PHP 5-only versions.) Does anyone have any contacts at Fantastico or cPanel to see how we could get them on board (which would probably mean not including PHP 5-incompatible apps in their next releases)? --Larry Garfield On Mon, 18 Jun 2007 23:33:18 +1000, sime <info@urbits.com> wrote:
I was having a little chat about PHP5 support with a guy at midphase.com. Off the record, at this stage, they generally support the movement, but the main problem for them with *pushing* PHP5 is Fantastico.
Had a little look around. Here is a list of scripts supported by Fantastico, along with their PHP5 status. Hope this is useful. http://tdknights.com/fantastico.htm
(It was obtained from this thread (Fantastico forum): http://www.netenberg.com/forum/viewtopic.php?t=3933)
.s
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 sime schrieb:
I was having a little chat about PHP5 support with a guy at midphase.com. Off the record, at this stage, they generally support the movement, but the main problem for them with *pushing* PHP5 is Fantastico.
Had a little look around. Here is a list of scripts supported by Fantastico, along with their PHP5 status. Hope this is useful. http://tdknights.com/fantastico.htm
This list does not seem to be up to date. I've checked phpwcms and their latest release is 1.3.3 and according to this url: http://phpmagazin.de/cms/index.php?site=159&start=158&show=twolist php 5 is supported. phpwcms was the only cms that was listed as not supporting php5. Cheers, Gerhard -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) iD8DBQFGdqBJfg6TFvELooQRAr6rAKCmmy8PCNd/BVeh/bD+kS3PxvscyQCgnDD/ KQKd2iq5zkgCVNspIQpVSsQ= =CaFC -----END PGP SIGNATURE-----
Gerhard Killesreiter wrote:
Had a little look around. Here is a list of scripts supported by Fantastico, along with their PHP5 status. Hope this is useful. http://tdknights.com/fantastico.htm
This list does not seem to be up to date. I've checked phpwcms and their latest release is 1.3.3 and according to this url:
http://phpmagazin.de/cms/index.php?site=159&start=158&show=twolist
Thanks Gerhard I've updated the thread, assuming that the author of the list will update. http://netenberg.com/forum/viewtopic.php?p=27362#27362
On Mon, 18 Jun 2007 11:05:47 +0200, "Johan Forngren" <johan@forngren.com> wrote:
On 6/18/07, Dries Buytaert <dries.buytaert@gmail.com> wrote:
On 11 Jun 2007, at 11:27, Gerhard Killesreiter wrote:
If we proceed like this, there will be no way we could revert to support PHP 4 in February 2008.
If I can believe my own projections (http://buytaert.net/php-is-dead- long-live-php), PHP5 will have a 30% adoption rate by February 2008, rather than the current 20%. Maybe February 2008 is a little too early?
Not if we can help push it ;)
That is precisely the intent of the GoPHP5 effort. :-) Change the curve so that it's not 20%-30%, but 50%+. That pushes past the tipping point, and PHP 4 can more quickly go the direction of PHP 3. To do that, though, requires market pressure. We're trying to be that market pressure. --Larry Garfield
On 18 Jun 2007, at 16:49, Larry Garfield wrote:
If I can believe my own projections (http://buytaert.net/php-is- dead- long-live-php), PHP5 will have a 30% adoption rate by February 2008, rather than the current 20%. Maybe February 2008 is a little too early?
Not if we can help push it ;)
That is precisely the intent of the GoPHP5 effort. :-) Change the curve so that it's not 20%-30%, but 50%+. That pushes past the tipping point, and PHP 4 can more quickly go the direction of PHP 3. To do that, though, requires market pressure. We're trying to be that market pressure.
Sure, I understand that much. The question is: what if we fail to bend the curve? My guess is that all Drupal installations take up at most 0.5% of PHP's total install base. That doesn't make for a lot of "bending power", does it? Add a dozen of other systems, and it might add up to 10% of PHP's total install base? Are we willing to take this bet and to leave most of our users behind when we failed to significantly bend the curve 6 months down the road? Clearly, being able to use PHP5 would help us help our users. Not providing an upgrade path for 70-80% of our install base seems to be at odd with that. It certainly has something kamikaze-like. Will people remember gophp5.org two weeks from now after it fell of the Digg main page? Just asking. I'm all for pushing PHP5, but I'm not sure I want to take such an extreme stance. As I mentioned in my blog post, let's start by making some features PHP5 only. Like, let's rewrite the aggregator module with PHP5's simplified XML parser API. That's disruptive too, but at least we'd be shooting ourselves in the foot, rather than through the head. -- Dries Buytaert :: http://www.buytaert.net/
I'm all for pushing PHP5, but I'm not sure I want to take such an extreme stance. As I mentioned in my blog post, let's start by making some features PHP5 only. Like, let's rewrite the aggregator module with PHP5's simplified XML parser API. That's disruptive too, but at least we'd be shooting ourselves in the foot, rather than
I'm against this, personally - I'd much rather see us standardize around an external library like SimplePie. RSS and Atom parsers are a very very complicated thing, and our parsing issue queue is quite indicative of that. I'd much rather use a library that is /dedicated/ to that task then the parser we currently have. What if, instead of rewriting aggregator, we simply shove it out of core (ala archive.module) and recommend people use something like Feed Parser or SimpleFeed, both of which use and require SimplePie, both require a healthy dose of loving, and both offer a much stronger API? -- Morbus Iff ( my name is legion, for we are many... ) Technical: http://www.oreillynet.com/pub/au/779 Culture: http://www.disobey.com/ and http://www.gamegrene.com/ aim: akaMorbus / skype: morbusiff / icq: 2927491 / jabber.org: morbus
On 6/18/07, Morbus Iff <morbus@disobey.com> wrote:
I'm all for pushing PHP5, but I'm not sure I want to take such an extreme stance. As I mentioned in my blog post, let's start by making some features PHP5 only. Like, let's rewrite the aggregator module with PHP5's simplified XML parser API. That's disruptive too, but at least we'd be shooting ourselves in the foot, rather than
Dries *was* just using that as an example....but you're right, it's probably about time to bang the drum about this...again. I'm against this, personally - I'd much rather see us standardize around
an external library like SimplePie. RSS and Atom parsers are a very very complicated thing, and our parsing issue queue is quite indicative of that. I'd much rather use a library that is /dedicated/ to that task then the parser we currently have.
+1. And lo, the discussion from many years continues. What if, instead of rewriting aggregator, we simply shove it out of core
(ala archive.module) and recommend people use something like Feed Parser or SimpleFeed, both of which use and require SimplePie, both require a healthy dose of loving, and both offer a much stronger API?
Neither one of those modules is a suitable replacement for aggregator in core (both do feeds-as-nodes and feed-items-as-nodes ... which is great / necessary for some use cases, but bad for others). See http://groups.drupal.org/node/4624 --> Aggregation API as a GSOC project. Side note: core needs love. It needs your love, and everyone's love. Views in core, more of CCK in core, improved / re-factored / API-ized aggregator -- etc. etc. etc. Oh yes, it also needs testing love, issue queue review love, documentation love. Where's the love? -- Boris Mann Office 604-682-2889 Skype borismann http://www.bryght.com
Neither one of those modules is a suitable replacement for aggregator in core (both do feeds-as-nodes and feed-items-as-nodes ... which is great / necessary for some use cases, but bad for others).
Not entirely true - the Feed Parser module does have an "items as aggregator.module" module, but it has fallen into disrepair. I plan to fix that up for my own needs.
Side note: core needs love. It needs your love, and everyone's love. Views in core, more of CCK in core, improved / re-factored / API-ized aggregator -- etc. etc. etc. Oh yes, it also needs testing love, issue queue review love, documentation love. Where's the love?
Is this intended at me? I can scratch only my own itches nowadays.
+1. And lo, the discussion from many years continues.
Unfortunately, yes. Can you sum up why it's never settled? -- Morbus Iff ( dare you overpower my stench of eeeevil? ) Technical: http://www.oreillynet.com/pub/au/779 Culture: http://www.disobey.com/ and http://www.gamegrene.com/ aim: akaMorbus / skype: morbusiff / icq: 2927491 / jabber.org: morbus
On 6/18/07, Morbus Iff <morbus@disobey.com> wrote:
Neither one of those modules is a suitable replacement for aggregator in core (both do feeds-as-nodes and feed-items-as-nodes ... which is great / necessary for some use cases, but bad for others).
Not entirely true - the Feed Parser module does have an "items as aggregator.module" module, but it has fallen into disrepair. I plan to fix that up for my own needs.
Urgh. Double Urgh. The Feed Parser stuck in 4.7 for which there are now better options :( -- /me laments for the non-convergence of forces around aggregation.
Side note: core needs love. It needs your love, and everyone's love.
Views in core, more of CCK in core, improved / re-factored / API-ized aggregator -- etc. etc. etc. Oh yes, it also needs testing love, issue queue review love, documentation love. Where's the love?
Is this intended at me? I can scratch only my own itches nowadays.
No. it's intended at everyone. Just a "feeling" I'm getting.
+1. And lo, the discussion from many years continues.
Unfortunately, yes. Can you sum up why it's never settled?
Dries wrote the parser functions originally to spec, and loves a challenge :P Stuff like Magpie was not really very pretty code in the past, so the whole "we can likely do this better" as well the great XMLRPC external library scare. These days, SimplePie is pretty solid. Other than that....no one has tackled it with actual code. -- Boris Mann Office 604-682-2889 Skype borismann http://www.bryght.com
Not entirely true - the Feed Parser module does have an "items as aggregator.module" module, but it has fallen into disrepair. I plan to fix that up for my own needs.
Urgh. Double Urgh. The Feed Parser stuck in 4.7 for which there are now better options :( -- /me laments for the non-convergence of forces around aggregation.
Eh? I don't know what you're saying. -- Morbus Iff ( i'm the droid you're looking for ) Technical: http://www.oreillynet.com/pub/au/779 Culture: http://www.disobey.com/ and http://www.gamegrene.com/ aim: akaMorbus / skype: morbusiff / icq: 2927491 / jabber.org: morbus
On 6/18/07, Morbus Iff <morbus@disobey.com> wrote:
Not entirely true - the Feed Parser module does have an "items as aggregator.module" module, but it has fallen into disrepair. I plan to fix that up for my own needs.
Urgh. Double Urgh. The Feed Parser stuck in 4.7 for which there are now better options :( -- /me laments for the non-convergence of forces around aggregation.
Eh? I don't know what you're saying.
Projects: * aggregator2 (abandoned) * Aggregation * Leech (sort of agg2, DevSeed are doing good work but I think the foundation has always been flawed) * Feedparser * SimpleFeed That would be what I mean by non-convergence. -- Boris Mann Office 604-682-2889 Skype borismann http://www.bryght.com
Projects:
Yes, and I took a look at all of these. Only FeedParser had a strong enough foot, unfortunately (even in its unworkable state) - SimpleFeed had a lot of the same features, but had a fairly critical design flaw that stopped me from doing any work with it at all: only one module can respond to its hook API (ie., I can't have a "save in database as item" module and a "send out through email" module both working on one feed - it's either one or the other). I did a fairly aggressive code review of the primary module and included upwards of 20 issues for it (a number of which have been fixed), and ended up basically unhappy with it.
* aggregator2 (abandoned)
Shouldn't even be on the list.
* Aggregation
* Leech (sort of agg2, DevSeed are doing good work but I think the foundation has always been flawed)
Items only as nodes, no API to stop node creation, and its own parsers. Leech also gives me a generic feeling of "what the hell is going on?", which is never a good thing. -- Morbus Iff ( rotinom ruoy edisni deppart mi pleH ) Technical: http://www.oreillynet.com/pub/au/779 Culture: http://www.disobey.com/ and http://www.gamegrene.com/ aim: akaMorbus / skype: morbusiff / icq: 2927491 / jabber.org: morbus
Aron is starting to pull these apart and is doing some benchmarking and feature comparisons http://groups.drupal.org/node/4519 http://groups.drupal.org/node/4547 I know he's putting some real time into this the next few weeks, and one idea is a pluggable parser - so you could have one for php4, one for php5, something using simpleXML, another using simplepie. This is at least an idea, and I think he's going to iron them out still further on the group wiki. Ian On Jun 18, 2007, at 1:07 PM, Boris Mann wrote:
On 6/18/07, Morbus Iff <morbus@disobey.com> wrote:
Not entirely true - the Feed Parser module does have an "items as aggregator.module" module, but it has fallen into disrepair. I plan to fix that up for my own needs.
Urgh. Double Urgh. The Feed Parser stuck in 4.7 for which there are now better options :( -- /me laments for the non-convergence of forces around aggregation.
Eh? I don't know what you're saying.
Projects: * aggregator2 (abandoned) * Aggregation * Leech (sort of agg2, DevSeed are doing good work but I think the foundation has always been flawed) * Feedparser * SimpleFeed
That would be what I mean by non-convergence.
-- Boris Mann Office 604-682-2889 Skype borismann http://www.bryght.com
Ian Ward Development Seed Inc. Technology Development and Discovery http://www.developmentseed.org http://www.developmentseed.org/blog developmentseedperu(skype) Tel. 202.250.3633 Fax. 806.214.6218
Morbus Iff wrote:
+1. And lo, the discussion from many years continues.
Unfortunately, yes. Can you sum up why it's never settled?
We have not seen patches (and even when we seen patches, love was missing) to improve the in-core aggregator, the in-core profile module and so on. It is a lot more comfortable to develop in contrib for some cases/people so obviously out of core profile, aggregator, etc solutions emerged. The lone souls who do try to fix up core modules does not get the love, reviews and tests... Eg. http://drupal.org/node/141419 where Dries noted he is fine with that way to go... Gabor
Aggregator is a) a feed parser b) a storage engine So the first action point is to make both pluggable. Then we can have a PHP4 parser (yay we have that!) our own PHP5 parser (to be written), SimpleXML (we have contrib code for it). Then there can be the current KISS storage engine and a contrib storage engine which stores into node. This could end the multitude of parallel feed modules --hopefully. And yes, I will work on this for Drupal 7 _if noone else does_. Comment module turned out better than I feared of, so I hope that if I can show the way someone will run with the aggregator code, too.
Monday, June 18, 2007, 12:03:48 PM, you wrote:
I'm all for pushing PHP5, but I'm not sure I want to take such an extreme stance. As I mentioned in my blog post, let's start by making some features PHP5 only. Like, let's rewrite the aggregator module with PHP5's simplified XML parser API. That's disruptive too, but at least we'd be shooting ourselves in the foot, rather than
I'm against this, personally - I'd much rather see us standardize around an external library like SimplePie.
RSS and Atom parsers are a very very complicated thing, and our parsing issue queue is quite indicative of that. I'd much rather use a library that is /dedicated/ to that task then the parser we currently have.
Especially after Aron's measurements, that show SimplePie being more than 20 times slower than the Drupal's core parser (http://groups.drupal.org/node/4519) I am beginning to lose hope in a one parser for all solution. The said complexity and variety of feed formats and use cases is rather asking for a pluggable architecture where you can chose what parser suits you best.
What if, instead of rewriting aggregator, we simply shove it out of core (ala archive.module) and recommend people use something like Feed Parser or SimpleFeed, both of which use and require SimplePie, both require a healthy dose of loving, and both offer a much stronger API?
In core or not, we I think we need a _common_ solution. Ian just posted it before - I repeat it nevertheless: Aron Novak is working for his SoC project on an aggregation solution that should replace aggregator module in core and provide a fundament for contrib modules. See here: http://groups.drupal.org/rss-aggregation and: http://aggregation.novaak.net Alex -- Alexander Barth Development Seed http://www.developmentseed.org http://www.developmentseed.org/blog lx_barth(skype) alex_b(drupal.org) Tel. 202.250.3633 Fax. 806.214.6218
Especially after Aron's measurements, that show SimplePie being more than 20 times slower than the Drupal's core parser (http://groups.drupal.org/node/4519) I am beginning to lose hope in a one parser for all solution.
I don't want one-parser-for-all-solutions either - see my comments on the wiki, which includes one potential workflow that I personally need: http://groups.drupal.org/node/4624 With that said, speed is not really an issue here, IMO - just like we can (and will) fiddle/scale with the downloading of thousands of feeds in hook_cron, the same thing can be done with processing time. As already mentioned in the comments of his tests, SimplePie does a lot of filtering to ensure the data is sanitized (granted, a lot of what Drupal could do as well, but which wouldn't have been considered part of the original parsing times, I suspect). What SimplePie does provide, however, is a much greater field of correct parsing - something that Drupal's core parser fails to do (helpfully contributing to its appearance of "speed").
See here: http://groups.drupal.org/rss-aggregation and: http://aggregation.novaak.net
Yes, I've been watching. However, this is Drupal 7 material. I can't wait that long. -- Morbus Iff ( notice how he deftly sidesteps the panty issue. ) Technical: http://www.oreillynet.com/pub/au/779 Culture: http://www.disobey.com/ and http://www.gamegrene.com/ aim: akaMorbus / skype: morbusiff / icq: 2927491 / jabber.org: morbus
Tuesday, June 19, 2007, 7:00:39 AM, Morbus If wrote:
With that said, speed is not really an issue here, IMO - just like we can (and will) fiddle/scale with the downloading of thousands of feeds in hook_cron, the same thing can be done with processing time. As already mentioned in the comments of his tests, SimplePie does a lot of filtering to ensure the data is sanitized (granted, a lot of what Drupal could do as well, but which wouldn't have been considered part of the original parsing times, I suspect). What SimplePie does provide, however, is a much greater field of correct parsing - something that Drupal's core parser fails to do (helpfully contributing to its appearance of "speed").
Unfortunately, speed is an issue for our aggregation sites where parsing time adds up accross all Drupal installations on the server. I don't object to that SimplePie may well be the better choice for a general use case - can we agree on a pluggable solution, though?
See here: http://groups.drupal.org/rss-aggregation and: http://aggregation.novaak.net
Yes, I've been watching. However, this is Drupal 7 material. I can't wait that long.
I can understand. But let's not desmiss the new aggregator solution as Drupal 7 only material. A backport to 6 or even 5 could be very easy to do. Moreover, it's pretty likely that Aron actually starts developing the new aggregator on Drupal 5. From a more personal point of view, I really would like to get rid of leech and build on top of common ground as soon as possible - not only when 7's out. Alex -- Alexander Barth Development Seed http://www.developmentseed.org http://www.developmentseed.org/blog lx_barth(skype) alex_b(drupal.org) Tel. 202.250.3633 Fax. 806.214.6218
On 6/19/07, Alexander Barth <alex@developmentseed.org> wrote:
Unfortunately, speed is an issue for our aggregation sites where parsing time adds up accross all Drupal installations on the server.
There are some good arguments to be made for decoupling parsing / aggregating directly within Drupal and moving it outside -- just putting nodes into Drupal. I don't object to that SimplePie may well be the better
choice for a general use case - can we agree on a pluggable solution, though?
Yes. I can understand. But let's not desmiss the new aggregator solution as
Drupal 7 only material. A backport to 6 or even 5 could be very easy to do. Moreover, it's pretty likely that Aron actually starts developing the new aggregator on Drupal 5.
From a more personal point of view, I really would like to get rid of leech and build on top of common ground as soon as possible - not only when 7's out.
Hallelujah. So...if Morbus needs to move, can we start on the foundations of the one, true pluggable aggregation API of doom, together, now? -- Boris Mann Office 604-682-2889 Skype borismann http://www.bryght.com
Didn't notice this thread in development. I'll read it in detail later today. In the meantime, I was pointed to this link by a friend of mine which shows that simplePie is a complete no-no. This actually compares three of the contributed aggregation modules in terms of performance, my aggregation module is included in the comparison. The aggregation module requires PHP 5 (since I use simpleXML), provides an API, and if you take a look at the issue queue you'll find that you don't need any external libraries to get very stable behavior. Aside from the initial wave of bug I got after completing it, I haven't received a bug issue in two months or so. SimplePie is forbiddingly performance terrible, or so I understood from Aron Novak's article which you can view below. http://groups.drupal.org/node/4519 On 6/19/07, Boris Mann <boris@bryght.com> wrote:
On 6/19/07, Alexander Barth <alex@developmentseed.org> wrote:
Unfortunately, speed is an issue for our aggregation sites where parsing time adds up accross all Drupal installations on the server.
There are some good arguments to be made for decoupling parsing / aggregating directly within Drupal and moving it outside -- just putting nodes into Drupal.
I don't object to that SimplePie may well be the better
choice for a general use case - can we agree on a pluggable solution, though?
Yes.
I can understand. But let's not desmiss the new aggregator solution as
Drupal 7 only material. A backport to 6 or even 5 could be very easy to do. Moreover, it's pretty likely that Aron actually starts developing the new aggregator on Drupal 5.
From a more personal point of view, I really would like to get rid of leech and build on top of common ground as soon as possible - not only when 7's out.
Hallelujah. So...if Morbus needs to move, can we start on the foundations of the one, true pluggable aggregation API of doom, together, now?
-- Boris Mann Office 604-682-2889 Skype borismann http://www.bryght.com
received a bug issue in two months or so. SimplePie is forbiddingly performance terrible, or so I understood from Aron Novak's article which
Unfortunately, we can't take these statistics as canon: * there's no instructions on how to duplicate. * the SimplePie result is an estimate ("At SimplePie I have to do an estimate, because the feed download time was accumulated to the measure." * it is unknown whether the other feed parsers are doing the same sanitization that SimplePie does, again, which adds more time to the results. I won't be convinced that the processing time of SimplePie is an accurate and overwhelming negative to it's more proper parsing support until I see repeatable and clean room results (no download, no sanitizing, etc.) -- Morbus Iff ( be realistic. demand the impossible. ) Technical: http://www.oreillynet.com/pub/au/779 Culture: http://www.disobey.com/ and http://www.gamegrene.com/ aim: akaMorbus / skype: morbusiff / icq: 2927491 / jabber.org: morbus
Unfortunately, we can't take these statistics as canon:
* there's no instructions on how to duplicate.
* the SimplePie result is an estimate ("At SimplePie I have to do an estimate, because the feed download time was accumulated to the measure."
* it is unknown whether the other feed parsers are doing the same sanitization that SimplePie does, again, which adds more time to the results.
I have done some quick tests, using the same URL as Aron: http://www.christiannewswire.com/rss/catfeed_2.xml I downloaded this file to my desktop. I will be passing this string into SimplePie instead of allowing SimplePie to download it. The file is 1M: 1027320 Jun 19 11:50 catfeed_2.xml This is the script I used with SimplePie 1.0 b3.2 (20061124): <?php $handle = fopen('./catfeed_2.xml', "r"); $contents = fread($handle, filesize('./catfeed_2.xml')); require './simplepie.inc'; $feed = new SimplePie(); $feed->set_raw_data($contents); $feed->init(); $parsed = $feed->get_items(); ?> With this command line: ~/Desktop > date && php simplepie.php && date Tue Jun 19 12:26:10 EDT 2007 Tue Jun 19 12:26:22 EDT 2007 As you can see, this does confirm the 10 or 12 second parse time -- it is also using all the sanitation that SimplePie does by default. However, SimpleFeed and FeedParser both ship with the latest development version of SimplePie which includes an option to stop this sanitation: $feed->set_stupidly_fast(TRUE); I grabbed today's development version, added the above line before the ->init() in the above script, and reran: ~/Desktop > date && php simplepie.php && date Tue Jun 19 12:28:54 EDT 2007 Tue Jun 19 12:28:55 EDT 2007 You'll notice that it is only 1 second which removes all doubt in my mind that SimplePie is a bad thing comparitively (since one would assume you'd sanitize the data as necessary within Drupal). -- Morbus Iff ( and think about the bad things that I didn't do ) Technical: http://www.oreillynet.com/pub/au/779 Culture: http://www.disobey.com/ and http://www.gamegrene.com/ aim: akaMorbus / skype: morbusiff / icq: 2927491 / jabber.org: morbus
I'm not really sure about the argument to sanitize data. Can't we sanitize it in a little less than 11 seconds? Also, isn't there a possibility the user wants this HTML code to come in as HTML code rather than plain text? I would guess that my module does lack many sanity checks, but at the same time, I do assume that administrators should be responsible as to what feeds they add to their sites. By the way, any sanity gurus who would like to check on my module's sanity checks and help me with additional sanity checks are very welcome and have my full gratitude. Just drop me a line off-list. On 6/19/07, Morbus Iff <morbus@disobey.com> wrote:
Unfortunately, we can't take these statistics as canon:
* there's no instructions on how to duplicate.
* the SimplePie result is an estimate ("At SimplePie I have to do an estimate, because the feed download time was accumulated to the measure."
* it is unknown whether the other feed parsers are doing the same sanitization that SimplePie does, again, which adds more time to the results.
I have done some quick tests, using the same URL as Aron:
http://www.christiannewswire.com/rss/catfeed_2.xml
I downloaded this file to my desktop. I will be passing this string into SimplePie instead of allowing SimplePie to download it. The file is 1M:
1027320 Jun 19 11:50 catfeed_2.xml
This is the script I used with SimplePie 1.0 b3.2 (20061124):
<?php $handle = fopen('./catfeed_2.xml', "r"); $contents = fread($handle, filesize('./catfeed_2.xml'));
require './simplepie.inc'; $feed = new SimplePie(); $feed->set_raw_data($contents); $feed->init(); $parsed = $feed->get_items(); ?>
With this command line:
~/Desktop > date && php simplepie.php && date Tue Jun 19 12:26:10 EDT 2007 Tue Jun 19 12:26:22 EDT 2007
As you can see, this does confirm the 10 or 12 second parse time -- it is also using all the sanitation that SimplePie does by default. However, SimpleFeed and FeedParser both ship with the latest development version of SimplePie which includes an option to stop this sanitation:
$feed->set_stupidly_fast(TRUE);
I grabbed today's development version, added the above line before the ->init() in the above script, and reran:
~/Desktop > date && php simplepie.php && date Tue Jun 19 12:28:54 EDT 2007 Tue Jun 19 12:28:55 EDT 2007
You'll notice that it is only 1 second which removes all doubt in my mind that SimplePie is a bad thing comparitively (since one would assume you'd sanitize the data as necessary within Drupal).
-- Morbus Iff ( and think about the bad things that I didn't do ) Technical: http://www.oreillynet.com/pub/au/779 Culture: http://www.disobey.com/ and http://www.gamegrene.com/ aim: akaMorbus / skype: morbusiff / icq: 2927491 / jabber.org: morbus
I'm not really sure about the argument to sanitize data. Can't we sanitize it in a little less than 11 seconds? Also, isn't there a
That would be up to Drupal - regardless of what parser you're using, how and when you sanitize it depends on the use case. My tests were merely to remove SimpleFeed's sanitation, forcing it into the same baseline as any other "made for Drupal" parser - use Drupal's routines as necessary. -- Morbus Iff ( you're just a copy of an imitation ) Technical: http://www.oreillynet.com/pub/au/779 Culture: http://www.disobey.com/ and http://www.gamegrene.com/ aim: akaMorbus / skype: morbusiff / icq: 2927491 / jabber.org: morbus
Disclaimer: I am not an RSS guru, just a pedant. :-) RSS is XML. The XML spec explicitly says that invalid files should be discarded, not guessed at the way HTML is. Trying to make sense of a broken RSS feed is explicitly contrary to the spec. So, er, why are we spending so much time trying to sanitize? If it doesn't parse correctly, report an error "this site's RSS feed is f*ed up, tell 'em to fix it". Am I missing something here? --Larry Garfield On Tue, 19 Jun 2007 20:12:45 +0300, "Ashraf Amayreh" <mistknight@gmail.com> wrote:
I'm not really sure about the argument to sanitize data. Can't we sanitize it in a little less than 11 seconds? Also, isn't there a possibility the user wants this HTML code to come in as HTML code rather than plain text?
I would guess that my module does lack many sanity checks, but at the same time, I do assume that administrators should be responsible as to what feeds they add to their sites.
By the way, any sanity gurus who would like to check on my module's sanity checks and help me with additional sanity checks are very welcome and have my full gratitude. Just drop me a line off-list.
On 6/19/07, Morbus Iff <morbus@disobey.com> wrote:
Unfortunately, we can't take these statistics as canon:
* there's no instructions on how to duplicate.
* the SimplePie result is an estimate ("At SimplePie I have to do an estimate, because the feed download time was accumulated to the measure."
* it is unknown whether the other feed parsers are doing the same sanitization that SimplePie does, again, which adds more time to the results.
I have done some quick tests, using the same URL as Aron:
http://www.christiannewswire.com/rss/catfeed_2.xml
I downloaded this file to my desktop. I will be passing this string into SimplePie instead of allowing SimplePie to download it. The file is 1M:
1027320 Jun 19 11:50 catfeed_2.xml
This is the script I used with SimplePie 1.0 b3.2 (20061124):
<?php $handle = fopen('./catfeed_2.xml', "r"); $contents = fread($handle, filesize('./catfeed_2.xml'));
require './simplepie.inc'; $feed = new SimplePie(); $feed->set_raw_data($contents); $feed->init(); $parsed = $feed->get_items(); ?>
With this command line:
~/Desktop > date && php simplepie.php && date Tue Jun 19 12:26:10 EDT 2007 Tue Jun 19 12:26:22 EDT 2007
As you can see, this does confirm the 10 or 12 second parse time -- it is also using all the sanitation that SimplePie does by default. However, SimpleFeed and FeedParser both ship with the latest development version of SimplePie which includes an option to stop this sanitation:
$feed->set_stupidly_fast(TRUE);
I grabbed today's development version, added the above line before the ->init() in the above script, and reran:
~/Desktop > date && php simplepie.php && date Tue Jun 19 12:28:54 EDT 2007 Tue Jun 19 12:28:55 EDT 2007
You'll notice that it is only 1 second which removes all doubt in my mind that SimplePie is a bad thing comparitively (since one would assume you'd sanitize the data as necessary within Drupal).
-- Morbus Iff ( and think about the bad things that I didn't do ) Technical: http://www.oreillynet.com/pub/au/779 Culture: http://www.disobey.com/ and http://www.gamegrene.com/ aim: akaMorbus / skype: morbusiff / icq: 2927491 / jabber.org: morbus
Larry Garfield wrote:
Disclaimer: I am not an RSS guru, just a pedant. :-)
RSS is XML. The XML spec explicitly says that invalid files should be discarded, not guessed at the way HTML is. Trying to make sense of a broken RSS feed is explicitly contrary to the spec. So, er, why are we spending so much time trying to sanitize? If it doesn't parse correctly, report an error "this site's RSS feed is f*ed up, tell 'em to fix it". Am I missing something here?
Because most people don't care if a site's XML is a little messed up. They care about the data; throwing away the data for trivial errors will upset the consumers of the data.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Earl Miles schrieb:
Larry Garfield wrote:
Disclaimer: I am not an RSS guru, just a pedant. :-)
RSS is XML. The XML spec explicitly says that invalid files should be discarded, not guessed at the way HTML is. Trying to make sense of a broken RSS feed is explicitly contrary to the spec. So, er, why are we spending so much time trying to sanitize? If it doesn't parse correctly, report an error "this site's RSS feed is f*ed up, tell 'em to fix it". Am I missing something here?
Because most people don't care if a site's XML is a little messed up. They care about the data; throwing away the data for trivial errors will upset the consumers of the data.
Then they should direct their complaints to the providers of said broken data. Drupal has had a rather good record wrt not validating somebody else's broken feeds. Did this change? Cheers, Gerhard -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) iD8DBQFGeCvKfg6TFvELooQRAvGfAJoDZmi2uE3djMLipkvn5AXO9vnqIACdFV9G RKjNLilGDHTt+CK1QFiUbbM= =S7hj -----END PGP SIGNATURE-----
Tuesday, June 19, 2007, 2:15:24 PM, you wrote:
Disclaimer: I am not an RSS guru, just a pedant. :-)
RSS is XML. The XML spec explicitly says that invalid files should be discarded, not guessed at the way HTML is. Trying to make sense of a broken RSS feed is explicitly contrary to the spec. So, er, why are we spending so much time trying to sanitize? If it doesn't parse correctly, report an error "this site's RSS feed is f*ed up, tell 'em to fix it". Am I missing something here?
True. That's one side of the coin. The other side is a world of non compliant feeds that all turn up on your issue queue to haunt you if your parser complains about them :) Alex
--Larry Garfield
On Tue, 19 Jun 2007 20:12:45 +0300, "Ashraf Amayreh" <mistknight@gmail.com> wrote:
I'm not really sure about the argument to sanitize data. Can't we sanitize it in a little less than 11 seconds? Also, isn't there a possibility the user wants this HTML code to come in as HTML code rather than plain text?
I would guess that my module does lack many sanity checks, but at the same time, I do assume that administrators should be responsible as to what feeds they add to their sites.
By the way, any sanity gurus who would like to check on my module's sanity checks and help me with additional sanity checks are very welcome and have my full gratitude. Just drop me a line off-list.
On 6/19/07, Morbus Iff <morbus@disobey.com> wrote:
Unfortunately, we can't take these statistics as canon:
* there's no instructions on how to duplicate.
* the SimplePie result is an estimate ("At SimplePie I have to do an estimate, because the feed download time was accumulated to the measure."
* it is unknown whether the other feed parsers are doing the same sanitization that SimplePie does, again, which adds more time to the results.
I have done some quick tests, using the same URL as Aron:
http://www.christiannewswire.com/rss/catfeed_2.xml
I downloaded this file to my desktop. I will be passing this string into SimplePie instead of allowing SimplePie to download it. The file is 1M:
1027320 Jun 19 11:50 catfeed_2.xml
This is the script I used with SimplePie 1.0 b3.2 (20061124):
<?php $handle = fopen('./catfeed_2.xml', "r"); $contents = fread($handle, filesize('./catfeed_2.xml'));
require './simplepie.inc'; $feed = new SimplePie(); $feed->set_raw_data($contents); $feed->init(); $parsed = $feed->get_items(); ?>
With this command line:
~/Desktop > date && php simplepie.php && date Tue Jun 19 12:26:10 EDT 2007 Tue Jun 19 12:26:22 EDT 2007
As you can see, this does confirm the 10 or 12 second parse time -- it is also using all the sanitation that SimplePie does by default. However, SimpleFeed and FeedParser both ship with the latest development version of SimplePie which includes an option to stop this sanitation:
$feed->set_stupidly_fast(TRUE);
I grabbed today's development version, added the above line before the ->init() in the above script, and reran:
~/Desktop > date && php simplepie.php && date Tue Jun 19 12:28:54 EDT 2007 Tue Jun 19 12:28:55 EDT 2007
You'll notice that it is only 1 second which removes all doubt in my mind that SimplePie is a bad thing comparitively (since one would assume you'd sanitize the data as necessary within Drupal).
-- Morbus Iff ( and think about the bad things that I didn't do ) Technical: http://www.oreillynet.com/pub/au/779 Culture: http://www.disobey.com/ and http://www.gamegrene.com/ aim: akaMorbus / skype: morbusiff / icq: 2927491 / jabber.org: morbus
-- Alexander Barth Development Seed http://www.developmentseed.org http://www.developmentseed.org/blog lx_barth(skype) alex_b(drupal.org) Tel. 202.250.3633 Fax. 806.214.6218
On 6/19/07, Larry Garfield <larry@garfieldtech.com> wrote:
Disclaimer: I am not an RSS guru, just a pedant. :-)
RSS is XML. The XML spec explicitly says that invalid files should be discarded, not guessed at the way HTML is. Trying to make sense of a broken RSS feed is explicitly contrary to the spec. So, er, why are we spending so much time trying to sanitize? If it doesn't parse correctly, report an error "this site's RSS feed is f*ed up, tell 'em to fix it". Am I missing something here?
And this is the point where I dive back in.... Many many many people have argued this. Fact: many non proper XML RSS feeds exist in the wild. Fact: if Drupal doesn't parse it, when other applications do, Drupal looks "broken" Fact: regular people like stuff that "just works" with any RSS feed out there, and will pick that over XML pedantry every day. A checkbox for "discard invalid XML" makes perfect sense....for *some feeds* and *some use cases*. -- Boris Mann Office 604-682-2889 Skype borismann http://www.bryght.com
On Jun 19, 2007, at 3:36 PM, Boris Mann wrote:
On 6/19/07, Larry Garfield <larry@garfieldtech.com> wrote:
Disclaimer: I am not an RSS guru, just a pedant. :-)
RSS is XML. The XML spec explicitly says that invalid files should be discarded, not guessed at the way HTML is. Trying to make sense of a broken RSS feed is explicitly contrary to the spec. So, er, why are we spending so much time trying to sanitize? If it doesn't parse correctly, report an error "this site's RSS feed is f*ed up, tell 'em to fix it". Am I missing something here?
This is another +1 for a pluggable parser approach. Then you (site admin) can decide on a case by case basis whether to use the parser that handles it all or only valid. So maybe validation is done in the main module and your plugin parser can flip that switch on and off whether to proceed at own will when validation fails. Either way, you can do it either way w/ plugability.
And this is the point where I dive back in....
Many many many people have argued this.
Fact: many non proper XML RSS feeds exist in the wild. Fact: if Drupal doesn't parse it, when other applications do, Drupal looks "broken" Fact: regular people like stuff that "just works" with any RSS feed out there, and will pick that over XML pedantry every day.
A checkbox for "discard invalid XML" makes perfect sense....for *some feeds* and *some use cases*.
-- Boris Mann Office 604-682-2889 Skype borismann http://www.bryght.com
Ian Ward Development Seed Inc. Technology Development and Discovery http://www.developmentseed.org http://www.developmentseed.org/blog developmentseedperu(skype) Tel. 202.250.3633 Fax. 806.214.6218
On 6/19/07, Boris Mann <boris@bryght.com> wrote:
On 6/19/07, Larry Garfield <larry@garfieldtech.com> wrote:
Disclaimer: I am not an RSS guru, just a pedant. :-)
RSS is XML. The XML spec explicitly says that invalid files should be discarded, not guessed at the way HTML is. Trying to make sense of a broken RSS feed is explicitly contrary to the spec. So, er, why are we spending so much time trying to sanitize? If it doesn't parse correctly, report an error "this site's RSS feed is f*ed up, tell 'em to fix it". Am I missing something here?
And this is the point where I dive back in....
Many many many people have argued this.
Fact: many non proper XML RSS feeds exist in the wild. Fact: if Drupal doesn't parse it, when other applications do, Drupal looks "broken" Fact: regular people like stuff that "just works" with any RSS feed out there, and will pick that over XML pedantry every day.
A checkbox for "discard invalid XML" makes perfect sense....for *some feeds* and *some use cases*
I strongly agree with Boris. Again, it goes to the point of how big a problem is it and can you afford to ignore it? If a web site sends bad HTML. Should browsers be so uptight as to popup messages for each error that is in the HTML? Or should it try to make the best of what is passed on silently? Guess what browsers do today? The same goes for MS IE and how non-standards compliant it is. Do we ignore it? No, because of its market share, as painful as it is. So, aggregators should do the same: try to make the best out of the data, even if it has some bad elements. -- 2bits.com http://2bits.com Drupal development, customization and consulting.
On Tue, 19 Jun 2007 12:36:50 -0700, "Boris Mann" <boris@bryght.com> wrote:
Fact: many non proper XML RSS feeds exist in the wild. Fact: if Drupal doesn't parse it, when other applications do, Drupal looks "broken" Fact: regular people like stuff that "just works" with any RSS feed out there, and will pick that over XML pedantry every day.
And once again, stupid people make the world a worse place with slower software. Even with that, though, we really shouldn't agonize over being as forgiving of bad RSS as, say, a browser is of bad HTML. If it's not even well-formed XML, then we're doing everyone, including ourselves, a disservice by trying to handle it. 12 seconds worth of cleanup is a waste of 11.8 seconds. --Larry Garfield
Even with that, though, we really shouldn't agonize over being as forgiving of bad RSS as, say, a browser is of bad HTML. If it's not even well-formed XML, then we're doing everyone, including ourselves, a disservice by trying to handle it. 12 seconds worth of cleanup is a waste of 11.8 seconds.
Oh God, not you too. People! Sanitation has /nothing/ to do with whether a feed is valid or well-formed or not! SimplePie is sanitizing feeds similarly to the Drupal equivalents of check_plain, check_markup, filter_xss, etc., etc. It is not fixing broken XML feeds. Please stop thinking that AA knows what he thinks I'm talking about. -- Morbus Iff ( evil is my sour flavor ) Technical: http://www.oreillynet.com/pub/au/779 Culture: http://www.disobey.com/ and http://www.gamegrene.com/ aim: akaMorbus / skype: morbusiff / icq: 2927491 / jabber.org: morbus
Listen, sanitizing as in protecting against XSS and SQL injection is a must. But others seem to include parsing not fully compliant XML feeds and not fully standards compliant feeds as part of sanitization. In my opinion the second is not sanitization and no aggregator needs to waste the code and time on trying to handle non-XML or non-standards compliant feeds. I would be very surprised if I found that SimplePie is wasting 11 seconds out of 12 in preventing XSS or SQL injection attacks alone. But hey, what do I know about SimplePie. Does anyone know what SimplePie actually does within these 11 seconds? I use CURL rather than the drupal function because drupal uses fsockets, fsockets are known to have issues when retrieving from HTTP 1.0 as opposed to HTTP 1.1, this is solved by sending specific headers, these headers are ignored on some Linux systems and that may increase the response time by as much as 5x what CURL would take, normally they would be similar in speed although CURL is slightly faster. In specific situations like firewall presence or failure in DNS resolution fsockets may stall for a few minutes. I read about these issues in a number of articles and I also found them listed in the comments under fsocketopen. CURL just saved me the hassle and probably tens of issues :-) http://www.php.net/manual/en/function.fsockopen.php Finally, with CURL I can support both HTTP and FTP URLs weather they are authenticated with a username and password or not. Only con to using CURL is that it has to be installed on the server machine and enabled in PHP (weather as a module or compiled).
Aside: I was just looking at aggregation module and I like how it keeps XML parsing and parsing into your internal data structure apart. How did you decide on the data structure that you use in aggregation?
Going into details will be too long, but there's an API between my module and the feed-specific handler (where it receives the ready to parse SimpleXML object from my module) and another API between the feed-specific handler and my module (where it passes the parsed data). My module chooses which feed handler to use (RSS,ATOM,etc) based on the user's input (specifying a term) when he creates a feed node. Thus to parse a new XML format all you have to do is add a term to the aggregation vocabulary and add a specific feed-handler file that conforms to the API calls. You can check the existing feed handlers to see how easy it is to do that.
On 6/19/07, Ashraf Amayreh <mistknight@gmail.com> wrote:
I use CURL rather than the drupal function because drupal uses fsockets, fsockets are known to have issues when retrieving from HTTP 1.0 as opposed to HTTP 1.1, this is solved by sending specific headers, these headers are ignored on some Linux systems and that may increase the response time by as much as 5x what CURL would take, normally they would be similar in speed although CURL is slightly faster. In specific situations like firewall presence or failure in DNS resolution fsockets may stall for a few minutes. I read about these issues in a number of articles and I also found them listed in the comments under fsocketopen. CURL just saved me the hassle and probably tens of issues :-)
http://www.php.net/manual/en/function.fsockopen.php
Finally, with CURL I can support both HTTP and FTP URLs weather they are authenticated with a username and password or not. Only con to using CURL is that it has to be installed on the server machine and enabled in PHP (weather as a module or compiled).
Ashraf,
From the comments at the PHP fsockopen page: === EDITORS NOTE: HTTP/1.1 uses persistent connection causing this delay. Use "Connection: close" header to disable it. === and === Just a note to everyone who is using fsockopen and fread / fgets for a HTTP connection.
Unless you specify "Connection: Close" in your headers you will need to wait for the socket to time out before feof($streamPointer) to return true. This has wasted 2 days of my time, grr! === The first example in the page, before the comments, also does the close for this reason. This is detailed in the RFC too http://www.w3.org/Protocols/rfc2616/rfc2616-sec8.html#sec8.1 Are you saying the header Connection: Close is ignored? Any reason why Drupal should not use HTTP/1.1?
opinion the second is not sanitization and no aggregator needs to waste the code and time on trying to handle non-XML or non-standards compliant
It depends entirely on your definition of "aggregator". In your module, you have only one parser, really - PHP's SimpleXML (or whatever it's called) that then sends the loaded data structure to the smaller "do things with it" (ie., RSS20.inc, etc.) subparsers. However, I'd think that it'd be far more flexible to send the raw strings around /as well/ - then one could support, for example, non-XML documents (or, in my particular case, I could write scrapers for sites that don't support feeds [or feeds that contain useful data]) so that I'd be able to hook into the generic aggregating process. Aggregation != just XML, IMO. I'd love, for example, to be able to add a "feed" that points to (pff, making crap outta my ass here) some comic site's "latest comic" HTML, choose a custom-made parser that expects that HTML, and return the same data structure that the aggregation API expects as legit. This /is/ aggregation - pulling disparate sources together.
I would be very surprised if I found that SimplePie is wasting 11 seconds out of 12 in preventing XSS or SQL injection attacks alone. But hey, what do I know about SimplePie. Does anyone know what SimplePie actually does within these 11 seconds?
SimplePie's set_stupidly_fast is a wrapper around: $this->enable_order_by_date(false); $this->remove_div(false); $this->strip_comments(false); $this->strip_htmltags(false); $this->strip_attributes(false); $this->set_image_handler(false); None of those are "fix broken XML". I reran the initial test like so: $feed->set_stupidly_fast(TRUE); $feed->enable_order_by_date(TRUE); i.e. first shutting everything off, then enabling one command: $feed->enable_order_by_date(TRUE) 2 seconds $feed->remove_div(TRUE) 1 second $feed->strip_comments(TRUE); 2 seconds $feed->strip_htmltags(TRUE); 2 seconds $feed->strip_attributes(TRUE); 2 seconds $feed->set_image_handler(TRUE); 1 second -- Morbus Iff ( if god is my witness, god must be blind ) Technical: http://www.oreillynet.com/pub/au/779 Culture: http://www.disobey.com/ and http://www.gamegrene.com/ aim: akaMorbus / skype: morbusiff / icq: 2927491 / jabber.org: morbus
Are you saying the header Connection: Close is ignored?
Any reason why Drupal should not use HTTP/1.1?
Examining the drupal_http_request I found that it actually doesn't use HTTP 1.1 in the first place, so I guess there's no problem in using it at all. But I would still like to maintain the ability to transparently accept feeds from HTTP or FTP as well as providing the users the option to access authenticated URLs. I don't know how widely used these features are, but I'd hate to remove a feature that could help a user out. Seems we were guilty in assuming what SimplePie did during these 11 seconds. Although I still think it's going about it the wrong way. 11 seconds is suicide. I sanitize against the extracted data, rather than the feed string as a whole. That's what I presume SimplePie is doing. I wish I could check it out for myself but my sleep indicators are overloaded. Morbus' suggestion to pass along the string as whole sounds logical, I'll see what I can do about that. Although I really had assumed that aggregation happens from XMLs only so the module would need a considerable amount of change to accommodate non-XML strings. I'll study the option and see what I can do. Anyone care to give me a patch for my next birthday? :-P AA On 6/20/07, Morbus Iff <morbus@disobey.com> wrote:
opinion the second is not sanitization and no aggregator needs to waste the code and time on trying to handle non-XML or non-standards compliant
It depends entirely on your definition of "aggregator". In your module, you have only one parser, really - PHP's SimpleXML (or whatever it's called) that then sends the loaded data structure to the smaller "do things with it" (ie., RSS20.inc, etc.) subparsers. However, I'd think that it'd be far more flexible to send the raw strings around /as well/ - then one could support, for example, non-XML documents (or, in my particular case, I could write scrapers for sites that don't support feeds [or feeds that contain useful data]) so that I'd be able to hook into the generic aggregating process. Aggregation != just XML, IMO.
I'd love, for example, to be able to add a "feed" that points to (pff, making crap outta my ass here) some comic site's "latest comic" HTML, choose a custom-made parser that expects that HTML, and return the same data structure that the aggregation API expects as legit. This /is/ aggregation - pulling disparate sources together.
I would be very surprised if I found that SimplePie is wasting 11 seconds out of 12 in preventing XSS or SQL injection attacks alone. But hey, what do I know about SimplePie. Does anyone know what SimplePie actually does within these 11 seconds?
SimplePie's set_stupidly_fast is a wrapper around:
$this->enable_order_by_date(false); $this->remove_div(false); $this->strip_comments(false); $this->strip_htmltags(false); $this->strip_attributes(false); $this->set_image_handler(false);
None of those are "fix broken XML". I reran the initial test like so:
$feed->set_stupidly_fast(TRUE); $feed->enable_order_by_date(TRUE);
i.e. first shutting everything off, then enabling one command:
$feed->enable_order_by_date(TRUE) 2 seconds $feed->remove_div(TRUE) 1 second $feed->strip_comments(TRUE); 2 seconds $feed->strip_htmltags(TRUE); 2 seconds $feed->strip_attributes(TRUE); 2 seconds $feed->set_image_handler(TRUE); 1 second
-- Morbus Iff ( if god is my witness, god must be blind ) Technical: http://www.oreillynet.com/pub/au/779 Culture: http://www.disobey.com/ and http://www.gamegrene.com/ aim: akaMorbus / skype: morbusiff / icq: 2927491 / jabber.org: morbus
Re: curl vs. drupal_http_request In addition to handling FTP urls and handle http authenticaton, curl can optionally follow redirects and handle things like content disposition headers (with custom code), which I've found to be important when dealing with enclosures. Not sure how flexible/capable drupal_http_request is on those kinds of issues. Scott On 6/20/07, Ashraf Amayreh <mistknight@gmail.com> wrote:
Are you saying the header Connection: Close is ignored?
Any reason why Drupal should not use HTTP/1.1?
Examining the drupal_http_request I found that it actually doesn't use HTTP 1.1 in the first place, so I guess there's no problem in using it at all. But I would still like to maintain the ability to transparently accept feeds from HTTP or FTP as well as providing the users the option to access authenticated URLs. I don't know how widely used these features are, but I'd hate to remove a feature that could help a user out.
Seems we were guilty in assuming what SimplePie did during these 11 seconds. Although I still think it's going about it the wrong way. 11 seconds is suicide. I sanitize against the extracted data, rather than the feed string as a whole. That's what I presume SimplePie is doing. I wish I could check it out for myself but my sleep indicators are overloaded.
Morbus' suggestion to pass along the string as whole sounds logical, I'll see what I can do about that. Although I really had assumed that aggregation happens from XMLs only so the module would need a considerable amount of change to accommodate non-XML strings. I'll study the option and see what I can do. Anyone care to give me a patch for my next birthday? :-P
AA
On 6/20/07, Morbus Iff <morbus@disobey.com > wrote:
opinion the second is not sanitization and no aggregator needs to waste the code and time on trying to handle non-XML or non-standards compliant
It depends entirely on your definition of "aggregator". In your module, you have only one parser, really - PHP's SimpleXML (or whatever it's called) that then sends the loaded data structure to the smaller "do things with it" (ie., RSS20.inc, etc.) subparsers. However, I'd think that it'd be far more flexible to send the raw strings around /as well/ - then one could support, for example, non-XML documents (or, in my particular case, I could write scrapers for sites that don't support feeds [or feeds that contain useful data]) so that I'd be able to hook into the generic aggregating process. Aggregation != just XML, IMO.
I'd love, for example, to be able to add a "feed" that points to (pff, making crap outta my ass here) some comic site's "latest comic" HTML, choose a custom-made parser that expects that HTML, and return the same data structure that the aggregation API expects as legit. This /is/ aggregation - pulling disparate sources together.
I would be very surprised if I found that SimplePie is wasting 11 seconds out of 12 in preventing XSS or SQL injection attacks alone. But hey, what do I know about SimplePie. Does anyone know what SimplePie actually does within these 11 seconds?
SimplePie's set_stupidly_fast is a wrapper around:
$this->enable_order_by_date(false); $this->remove_div(false); $this->strip_comments(false); $this->strip_htmltags(false); $this->strip_attributes(false); $this->set_image_handler(false);
None of those are "fix broken XML". I reran the initial test like so:
$feed->set_stupidly_fast(TRUE); $feed->enable_order_by_date(TRUE);
i.e. first shutting everything off, then enabling one command:
$feed->enable_order_by_date(TRUE) 2 seconds $feed->remove_div(TRUE) 1 second $feed->strip_comments(TRUE); 2 seconds $feed->strip_htmltags(TRUE); 2 seconds $feed->strip_attributes(TRUE); 2 seconds $feed->set_image_handler(TRUE); 1 second
-- Morbus Iff ( if god is my witness, god must be blind ) Technical: http://www.oreillynet.com/pub/au/779 Culture: http://www.disobey.com/ and http://www.gamegrene.com/ aim: akaMorbus / skype: morbusiff / icq: 2927491 / jabber.org: morbus
Curl is frequently not available. I've had to have it installed a few times, but a LOT of people won't even have that as an option. Scott Trudeau wrote:
Re: curl vs. drupal_http_request
In addition to handling FTP urls and handle http authenticaton, curl can optionally follow redirects and handle things like content disposition headers (with custom code), which I've found to be important when dealing with enclosures. Not sure how flexible/capable drupal_http_request is on those kinds of issues.
Scott
On 6/20/07, Ashraf Amayreh <mistknight@gmail.com> wrote:
Are you saying the header Connection: Close is ignored?
Any reason why Drupal should not use HTTP/1.1?
Examining the drupal_http_request I found that it actually doesn't use HTTP 1.1 in the first place, so I guess there's no problem in using it at all. But I would still like to maintain the ability to transparently accept feeds from HTTP or FTP as well as providing the users the option to access authenticated URLs. I don't know how widely used these features are, but I'd hate to remove a feature that could help a user out.
Seems we were guilty in assuming what SimplePie did during these 11 seconds. Although I still think it's going about it the wrong way. 11 seconds is suicide. I sanitize against the extracted data, rather than the feed string as a whole. That's what I presume SimplePie is doing. I wish I could check it out for myself but my sleep indicators are overloaded.
Morbus' suggestion to pass along the string as whole sounds logical, I'll see what I can do about that. Although I really had assumed that aggregation happens from XMLs only so the module would need a considerable amount of change to accommodate non-XML strings. I'll study the option and see what I can do. Anyone care to give me a patch for my next birthday? :-P
AA
On 6/20/07, Morbus Iff <morbus@disobey.com > wrote:
opinion the second is not sanitization and no aggregator needs to waste the code and time on trying to handle non-XML or non-standards compliant
It depends entirely on your definition of "aggregator". In your module, you have only one parser, really - PHP's SimpleXML (or whatever it's called) that then sends the loaded data structure to the smaller "do things with it" (ie., RSS20.inc, etc.) subparsers. However, I'd think that it'd be far more flexible to send the raw strings around /as well/ - then one could support, for example, non-XML documents (or, in my particular case, I could write scrapers for sites that don't support feeds [or feeds that contain useful data]) so that I'd be able to hook into the generic aggregating process. Aggregation != just XML, IMO.
I'd love, for example, to be able to add a "feed" that points to (pff, making crap outta my ass here) some comic site's "latest comic" HTML, choose a custom-made parser that expects that HTML, and return the same data structure that the aggregation API expects as legit. This /is/ aggregation - pulling disparate sources together.
I would be very surprised if I found that SimplePie is wasting 11 seconds out of 12 in preventing XSS or SQL injection attacks alone. But hey, what do I know about SimplePie. Does anyone know what SimplePie actually does within these 11 seconds?
SimplePie's set_stupidly_fast is a wrapper around:
$this->enable_order_by_date(false); $this->remove_div(false); $this->strip_comments(false); $this->strip_htmltags(false); $this->strip_attributes(false); $this->set_image_handler(false);
None of those are "fix broken XML". I reran the initial test like so:
$feed->set_stupidly_fast(TRUE); $feed->enable_order_by_date(TRUE);
i.e. first shutting everything off, then enabling one command:
$feed->enable_order_by_date(TRUE) 2 seconds $feed->remove_div(TRUE) 1 second $feed->strip_comments(TRUE); 2 seconds $feed->strip_htmltags(TRUE); 2 seconds $feed->strip_attributes(TRUE); 2 seconds $feed->set_image_handler(TRUE); 1 second
-- Morbus Iff ( if god is my witness, god must be blind ) Technical: http://www.oreillynet.com/pub/au/779 Culture: http://www.disobey.com/ and http://www.gamegrene.com/ aim: akaMorbus / skype: morbusiff / icq: 2927491 / jabber.org: morbus
-- Sean Robertson Web Developer NGP Software, Inc. seanr@ngpsoftware.com (202) 686-9330 http://www.ngpsoftware.com
On 6/20/07, Sean Robertson <seanr@ngpsoftware.com> wrote:
Curl is frequently not available. I've had to have it installed a few times, but a LOT of people won't even have that as an option
This is another one of those "must be pluggable" or "fallback" areas. For random ass shared hosting (hereafter known as RASH), we can't necessarily support high performance aggregation of content. For sites that do need high performance aggregation of content, they will have control over their environment. And re: curl handling authenticated feeds -- storing user + pass in the database in cleartext as part of the feed URL is not really a great solution..... drupal_socket_request with plugins for different things? Interesting thought.... -- Boris Mann Office 604-682-2889 Skype borismann http://www.bryght.com
I would say that module requirements and minimum PHP 5 version requirements that aim to improve performance and reduce module code are arguably beneficial in that they force developers/users to look for quality hosting rather than RASH (nice one Boris, hehe). Also, when a module is capable of coping with anything by using very slow, inefficient and somewhat non-guaranteed techniques to "make do" then many lazy developers will just plug these modules in without really putting any effort to install the performance boosting "optional" components, and when these very less-than-optimal techniques fail on some crappy feeds it gets your issue queue filled with garbage. Not to mention giving the false impression to anyone who's examining the site that the module is crappy, not the developer who used it. By requiring PHP 5 and CURL support I minimized the number of issues I got. I realize this is a shady area and my argument might not always hold true. I agree with you here Boris, I should find a better way of storing these authentication passwords. MD5 and SHA1 both offer one way encryption so they won't help. mcrypt came to mind but it requires some additional libraries on a Linux machine. I wonder if someone stumbled on PHP code that would provide a simple 2-way encryption/decryption or has any ideas that would help here? On 6/20/07, Boris Mann <boris@bryght.com> wrote:
On 6/20/07, Sean Robertson <seanr@ngpsoftware.com> wrote:
Curl is frequently not available. I've had to have it installed a few times, but a LOT of people won't even have that as an option
This is another one of those "must be pluggable" or "fallback" areas. For random ass shared hosting (hereafter known as RASH), we can't necessarily support high performance aggregation of content.
For sites that do need high performance aggregation of content, they will have control over their environment.
And re: curl handling authenticated feeds -- storing user + pass in the database in cleartext as part of the feed URL is not really a great solution.....
drupal_socket_request with plugins for different things? Interesting thought....
-- Boris Mann Office 604-682-2889 Skype borismann http://www.bryght.com
Quoting Ashraf Amayreh <mistknight@gmail.com>:
I agree with you here Boris, I should find a better way of storing these authentication passwords. MD5 and SHA1 both offer one way encryption so they won't help. mcrypt came to mind but it requires some additional libraries on a Linux machine. I wonder if someone stumbled on PHP code that would provide a simple 2-way encryption/decryption or has any ideas that would help here?
I did that once with mcrypt. I can try to find the code if you like but I'm thinking you want something that isn't mcrypt. Earnie
Just FYI: SimplePie will use the PHP CURL extension, if it is installed. If you want to read SSL-encrypted feeds like I do, CURL is helpful there, too. I'm glad to see all this discussion on aggregation, and heartened to see that after some, uh, disagreement, people seem to be heading in relatively the same direction. Having a good aggregation solution for 99% of Drupal installs would be nearly a "killer app" -- where such a good solution would provide support for needs at various ends of the spectrums involved (fast, thorough, PHP4, etc.). ..chrisxj
RSS is XML. The XML spec explicitly says that invalid files should be discarded, not guessed at the way HTML is. Trying to make sense of a broken RSS feed is explicitly contrary to the spec. So, er, why are we spending so much time trying to sanitize? If it doesn't parse correctly, report an error "this site's RSS feed is f*ed up, tell 'em to fix it". Am I missing something here?
Did you forget Postel's Law? Or the fact that for a feed to be considered "invalid" (as opposed to "not well-formed") would mean that Drupal would have to have a validating document type parser? http://www.w3.org/TR/REC-xml/#dt-valid http://www.w3.org/TR/REC-xml/#dt-wellformed And, honestly, telling people that their RSS is malformed and "pls fix, k thanks" is about as viable as telling someone that their HTML isn't well formed. It just ain't going to happen. -- Morbus Iff ( be realistic. demand the impossible. ) Technical: http://www.oreillynet.com/pub/au/779 Culture: http://www.disobey.com/ and http://www.gamegrene.com/ aim: akaMorbus / skype: morbusiff / icq: 2927491 / jabber.org: morbus
Ahhh... so by sanitizing you mean accepting non-fully standards compliant feeds? If that's what you mean then definitely not. I totally agree with Larry on this. Why waste my processing time on feeds that are not worth a penny? Also, in my issue queue I haven't received one complaint about a feed that would not parse. Not because I do any sanitization, but I think the reason for this is because I parse the feed as an XML and look for the main components, so even if it's not fully conforming it will make do if it has the main components. If it's beyond hope in being called XML in the first place or just totally messed up I don't waste the time and the coding effort that might double or triple my module's code size on it. If that's a type of sanitization then good for me but it definitely does not affect my module's performance :-) Finally, I don't really care to make my module work with everyone and everything. That's why I have clear PHP 5 and CURL requirements. In my opinion, if a person is not prepared to get a good environment for his site or let's something as crappy as a host dictate his platform (let's drop drupal because our host is using php 3.0 looool) then he's definitely not from my target audience. PHP 5 sites have become as common as PHP 4 and very near in price. VPS hosting is becomming cheaper by the day for anyone who's serious about a site. Take a look for yourselves. I use this VPS provider to sharpen my LAMP skills since this provider provides a clean slate installation. I have root access and I can do anything I want there. The comparison between HTML and XML feeds is simply flawed IMHO. http://www.vpslink.com/ On 6/19/07, Morbus Iff <morbus@disobey.com> wrote:
RSS is XML. The XML spec explicitly says that invalid files should be discarded, not guessed at the way HTML is. Trying to make sense of a broken RSS feed is explicitly contrary to the spec. So, er, why are we spending so much time trying to sanitize? If it doesn't parse correctly, report an error "this site's RSS feed is f*ed up, tell 'em to fix it". Am I missing something here?
Did you forget Postel's Law? Or the fact that for a feed to be considered "invalid" (as opposed to "not well-formed") would mean that Drupal would have to have a validating document type parser?
http://www.w3.org/TR/REC-xml/#dt-valid http://www.w3.org/TR/REC-xml/#dt-wellformed
And, honestly, telling people that their RSS is malformed and "pls fix, k thanks" is about as viable as telling someone that their HTML isn't well formed. It just ain't going to happen.
-- Morbus Iff ( be realistic. demand the impossible. ) Technical: http://www.oreillynet.com/pub/au/779 Culture: http://www.disobey.com/ and http://www.gamegrene.com/ aim: akaMorbus / skype: morbusiff / icq: 2927491 / jabber.org: morbus
Ahhh... so by sanitizing you mean accepting non-fully standards compliant feeds? If that's what you mean then definitely not. I totally
No, I don't. I mean protecting the users from some idiot inserting XSS or anything else in his RSS items (knowingly or not). Someone in this thread said they "trust" (hope?) that the consumer of their module "trusts" the RSS feeds they consume. That's uh... foolish. The rest of your email was entirely ignorable. -- Morbus Iff ( keep out of reach of children ) Technical: http://www.oreillynet.com/pub/au/779 Culture: http://www.disobey.com/ and http://www.gamegrene.com/ aim: akaMorbus / skype: morbusiff / icq: 2927491 / jabber.org: morbus
Protecting the user from an XSS or SQL injection attack is one thing, accepting non-standard compliant feeds is another. Did you waste the time to read a couple of threads before mine or did you have this reply tailor made a few days ago? The discussion was on weather to accept non-standard compliant RSS/RDF/ATOM feeds or not sweety. And a little on weather to push for PHP 5 or not. So why don't you stick to that for a change? On 6/19/07, Morbus Iff <morbus@disobey.com> wrote:
Ahhh... so by sanitizing you mean accepting non-fully standards compliant feeds? If that's what you mean then definitely not. I totally
No, I don't. I mean protecting the users from some idiot inserting XSS or anything else in his RSS items (knowingly or not). Someone in this thread said they "trust" (hope?) that the consumer of their module "trusts" the RSS feeds they consume. That's uh... foolish.
The rest of your email was entirely ignorable.
-- Morbus Iff ( keep out of reach of children ) Technical: http://www.oreillynet.com/pub/au/779 Culture: http://www.disobey.com/ and http://www.gamegrene.com/ aim: akaMorbus / skype: morbusiff / icq: 2927491 / jabber.org: morbus
Did you waste the time to read a couple of threads before mine or did you have this reply tailor made a few days ago? The discussion was on weather to accept non-standard compliant RSS/RDF/ATOM feeds or not sweety. And a little on weather to push for PHP 5 or not. So why don't you stick to that for a change?
You're kidding, right? -- Morbus Iff ( damn you richards! i will have my revenge! ) Technical: http://www.oreillynet.com/pub/au/779 Culture: http://www.disobey.com/ and http://www.gamegrene.com/ aim: akaMorbus / skype: morbusiff / icq: 2927491 / jabber.org: morbus
To put this distraction aside, PHP 5 is as readily available a hosting solution as PHP 4. Developers who want to use Drupal need to upgrade to PHP 5 rather than Drupal keeping pace with PHP 4. As to aggregation, as long as a module is not allowing an XSS or an SQL injection attack. I see no need to try to accept feeds from sites that don't even put the effort in authoring feeds that meet the absolute minimal feed standards. Although some may like to aggregate the HTML content in a feed as is, how we could set in dealing with this case is open for debate. This is my opinion on the subject. On 6/20/07, Morbus Iff <morbus@disobey.com> wrote:
Did you waste the time to read a couple of threads before mine or did you have this reply tailor made a few days ago? The discussion was on weather to accept non-standard compliant RSS/RDF/ATOM feeds or not sweety. And a little on weather to push for PHP 5 or not. So why don't you stick to that for a change?
You're kidding, right?
-- Morbus Iff ( damn you richards! i will have my revenge! ) Technical: http://www.oreillynet.com/pub/au/779 Culture: http://www.disobey.com/ and http://www.gamegrene.com/ aim: akaMorbus / skype: morbusiff / icq: 2927491 / jabber.org: morbus
Tuesday, June 19, 2007, 5:31:01 PM, you wrote:
To put this distraction aside, PHP 5 is as readily available a hosting solution as PHP 4. Developers who want to use Drupal need to upgrade to PHP 5 rather than Drupal keeping pace with PHP 4.
As to aggregation, as long as a module is not allowing an XSS or an SQL injection attack. I see no need to try to accept feeds from sites that don't even put the effort in authoring feeds that meet the absolute minimal feed standards.
Somebody will see the need and will come up with a solution for it. That's where in the past this somebody started his/her own new aggregation module from scratch :)
Although some may like to aggregate the HTML content in a feed as is, how we could set in dealing with this case is open for debate. This is my opinion on the subject.
On 6/20/07, Morbus Iff <morbus@disobey.com> wrote:
Did you waste the time to read a couple of threads before mine or did you have this reply tailor made a few days ago? The discussion was on weather to accept non-standard compliant RSS/RDF/ATOM feeds or not sweety. And a little on weather to push for PHP 5 or not. So why don't you stick to that for a change?
You're kidding, right?
-- Morbus Iff ( damn you richards! i will have my revenge! ) Technical: http://www.oreillynet.com/pub/au/779 Culture: http://www.disobey.com/ and http://www.gamegrene.com/ aim: akaMorbus / skype: morbusiff / icq: 2927491 / jabber.org: morbus
-- Alexander Barth Development Seed http://www.developmentseed.org http://www.developmentseed.org/blog lx_barth(skype) alex_b(drupal.org) Tel. 202.250.3633 Fax. 806.214.6218
Tuesday, June 19, 2007, 4:08:30 PM, you wrote:
Ahhh... so by sanitizing you mean accepting non-fully standards compliant feeds? If that's what you mean then definitely not. I totally agree with Larry on this. Why waste my processing time on feeds that are not worth a penny?
I think the very discussion that we are having here is proving that there are use cases where you want to go for speed rather than resilience, and there are equally valid use cases where resilience is more important than speed. I guess we'll find contradicting use cases like these all the way down Aggregation Lane. We shouldn't get caught up in discussing which one's the one. The new aggregator should provide a fundament that does what all aggregators have to do while leaving all doors for improvement and adaption open.
Also, in my issue queue I haven't received one complaint about a feed that would not parse. Not because I do any sanitization, but I think the reason for this is because I parse the feed as an XML and look for the main components, so even if it's not fully conforming it will make do if it has the main components. If it's beyond hope in being called XML in the first place or just totally messed up I don't waste the time and the coding effort that might double or triple my module's code size on it. If that's a type of sanitization then good for me but it definitely does not affect my module's performance :-)
Aside: I was just looking at aggregation module and I like how it keeps XML parsing and parsing into your internal data structure apart. How did you decide on the data structure that you use in aggregation?
Finally, I don't really care to make my module work with everyone and everything. That's why I have clear PHP 5 and CURL requirements.
Agreed. There won't be a "one single monolithic module serves all" solution. I do hope though that there will be a "one fundament/API serves them all" solution. * We are duplicating efforts between all aggregation modules that have nothing to do with the special requirements which justify every single module out there. * A lot of features that are implemented on just one aggregation module would be good to see on all aggregation modules. Alex -- Alexander Barth Development Seed http://www.developmentseed.org http://www.developmentseed.org/blog lx_barth(skype) alex_b(drupal.org) Tel. 202.250.3633 Fax. 806.214.6218
Finally, I don't really care to make my module work with everyone and everything. That's why I have clear PHP 5 and CURL requirements.
Incidentally, I'm curious: why choose curl over drupal_http_request? What were the pros and cons? The nearest I can see (implemented) was support for FTP feeds, which is a relatively rare occurrence. -- Morbus Iff ( that's crazier than a sack of ferrets at a hoe-down. ) Technical: http://www.oreillynet.com/pub/au/779 Culture: http://www.disobey.com/ and http://www.gamegrene.com/ aim: akaMorbus / skype: morbusiff / icq: 2927491 / jabber.org: morbus
Hallelujah. So...if Morbus needs to move, can we start on the foundations of the one, true pluggable aggregation API of doom, together, now?
The "problem" is the SOC project. If we help out on that, we potentially invalidate his application and earnings. If we don't help out on that, we do what? Work on Feed Parser? SimpleFeed? A brand new module? -- Morbus Iff ( be realistic. demand the impossible. ) Technical: http://www.oreillynet.com/pub/au/779 Culture: http://www.disobey.com/ and http://www.gamegrene.com/ aim: akaMorbus / skype: morbusiff / icq: 2927491 / jabber.org: morbus
Tuesday, June 19, 2007, 12:03:28 PM, you wrote:
Hallelujah. So...if Morbus needs to move, can we start on the foundations of the one, true pluggable aggregation API of doom, together, now?
The "problem" is the SOC project. If we help out on that, we potentially invalidate his application and earnings. If we don't help out on that, we do what? Work on Feed Parser? SimpleFeed? A brand new module?
I don't see how we could invalidate Aron's application and earnings by helping out on his SoC project. Quite the contrary, I think everybody's input is very important to make sure that it is a success for the Drupal community. Alex -- Alexander Barth Development Seed http://www.developmentseed.org http://www.developmentseed.org/blog lx_barth(skype) alex_b(drupal.org) Tel. 202.250.3633 Fax. 806.214.6218
On 6/19/07, Alexander Barth <alex@developmentseed.org> wrote:
Tuesday, June 19, 2007, 12:03:28 PM, you wrote:
Hallelujah. So...if Morbus needs to move, can we start on the foundations of the one, true pluggable aggregation API of doom, together, now?
The "problem" is the SOC project. If we help out on that, we potentially invalidate his application and earnings. If we don't help out on that, we do what? Work on Feed Parser? SimpleFeed? A brand new module?
I don't see how we could invalidate Aron's application and earnings by helping out on his SoC project. Quite the contrary, I think everybody's input is very important to make sure that it is a success for the Drupal community.
Morbus meant actually working on code....which he wants/needs to do to scratch his own itch. So: how to work with Aaron's stuff while being able to forge ahead with producing running code? -- Boris Mann Office 604-682-2889 Skype borismann http://www.bryght.com
Tuesday, June 19, 2007, 12:33:11 PM, you wrote:
On 6/19/07, Alexander Barth <alex@developmentseed.org> wrote:
Tuesday, June 19, 2007, 12:03:28 PM, you wrote:
Hallelujah. So...if Morbus needs to move, can we start on the foundations of the one, true pluggable aggregation API of doom, together, now?
The "problem" is the SOC project. If we help out on that, we potentially invalidate his application and earnings. If we don't help out on that, we do what? Work on Feed Parser? SimpleFeed? A brand new module?
I don't see how we could invalidate Aron's application and earnings by helping out on his SoC project. Quite the contrary, I think everybody's input is very important to make sure that it is a success for the Drupal community.
Morbus meant actually working on code....which he wants/needs to do to scratch his own itch.
So: how to work with Aaron's stuff while being able to forge ahead with producing running code?
Thanks for clarification - got that wrong. We were discussing the same thing here at Development Seed. I am afraid we'll have to wait until the baby takes shape and there is actually code to build on. Obviously there are ways to make things faster by helping Aron with vetting solutions and further on with testing, patching and feedback. I got some time set aside for this. When exactly it makes sense to jump on the new platform depends a lot on the deadlines of the project at hand. The SoC's deadlines are published here: http://aggregation.novaak.net/ Alex -- Alexander Barth Development Seed http://www.developmentseed.org http://www.developmentseed.org/blog lx_barth(skype) alex_b(drupal.org) Tel. 202.250.3633 Fax. 806.214.6218
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Dries Buytaert schrieb:
On 18 Jun 2007, at 16:49, Larry Garfield wrote:
That is precisely the intent of the GoPHP5 effort. :-) Change the curve so that it's not 20%-30%, but 50%+. That pushes past the tipping point, and PHP 4 can more quickly go the direction of PHP 3. To do that, though, requires market pressure. We're trying to be that market pressure.
Sure, I understand that much. The question is: what if we fail to bend the curve?
My guess is that all Drupal installations take up at most 0.5% of PHP's total install base. That doesn't make for a lot of "bending power", does it? Add a dozen of other systems, and it might add up to 10% of PHP's total install base?
The power lies in the fact that if the gophp5 effort succeeds, there will be no "other place" for users to run to. This means that they will either have to stick with the latest php compatible version or they have to change hosters in order to find a host that supports php5.
Are we willing to take this bet and to leave most of our users behind when we failed to significantly bend the curve 6 months down the road?
We won't leave them behind. Drupal 6 will still be available.
Clearly, being able to use PHP5 would help us help our users. Not providing an upgrade path for 70-80% of our install base seems to be at odd with that. It certainly has something kamikaze-like.
Will people remember gophp5.org two weeks from now after it fell of the Digg main page? Just asking.
I'm all for pushing PHP5, but I'm not sure I want to take such an extreme stance. As I mentioned in my blog post, let's start by making some features PHP5 only. Like, let's rewrite the aggregator module with PHP5's simplified XML parser API. That's disruptive too, but at least we'd be shooting ourselves in the foot, rather than through the head.
If we feel that in Feb 08 PHP5 is still not as widely supported as we'd like, we could extend the Drupal 6 life cycle. Cheers, Gerhard -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) iD8DBQFGdrGPfg6TFvELooQRAuInAJ4gWOI/DSiQIj/o44B701AkerVzgwCgy5fd 6M6q0f3htMiPuRC0UPQHD58= =wbGC -----END PGP SIGNATURE-----
This is not the time to be timid. We as developers know deep in our hearts that PHP5 is the best thing for Drupal. The amazing resonance that Larry and I have been receiving from the developers on other projects shows that we're not alone in thinking this. A PHP5 based Drupal 7 will not cut off the upgrade path of anybody. Let's be honest about upgrading Drupal. If you follow the instructions then you have to be able to export your database (and potentially import it later if you screw things up) and be able to backup the files on your file system, potentially replacing them if the upgrade fails. These are coincidentally the same steps that are needed to migrate your Drupal site to a new web host. There are already plenty of web hosts that support PHP5. Anybody who doesn't have the skills to migrate to one is fairly unlikely to ever upgrade their Drupal installation anyway. We have a chance right now to act in unison with a majority of the meaningful PHP based FOSS projects. This is an action that has no precedent. Never before have we all spoken with one voice in a coordinated way. I feel that failing to take a position of strong leadership based on what we believe is best for our software is the shot in the head, not the other way around. -Robert Dries Buytaert wrote:
Sure, I understand that much. The question is: what if we fail to bend the curve? ... I'm all for pushing PHP5, but I'm not sure I want to take such an extreme stance. As I mentioned in my blog post, let's start by making some features PHP5 only. Like, let's rewrite the aggregator module with PHP5's simplified XML parser API. That's disruptive too, but at least we'd be shooting ourselves in the foot, rather than through the head.
I'm in agreement with Robert Douglass. Now is the time to be bold, not timid. Dries wrote: "My guess is that all Drupal installations take up at most 0.5% of PHP's total install base." I don't think that measurement (PHP total install base) is relevant. Far more important is what fraction of the top 50 most commonly installed PHP packages Drupal is, and what fraction of the top 50 the other PHP projects who will agree to move to PHP5 comprise. The long tail of PHP applications is huge -- and mostly ignorable. ..chrisxj
As other people have stated, now is a time to lead, not to follow. There will always be myriad reasons to make the safe choice. Others have articulated the advantages of using PHP5, and there is widespread support among the PHP crowd (as articulated by Rasmus at OSCMS) and among other open source projects. Perhaps leadership in this area will allow more people to step up and share the responsibility for articulating why this course makes sense. And even if nobody else stands up, it's still the right thing to do from a strictly technical place. Cheers, Bill Chris Johnson wrote:
I'm in agreement with Robert Douglass. Now is the time to be bold, not timid.
Dries wrote: "My guess is that all Drupal installations take up at most 0.5% of PHP's total install base." I don't think that measurement (PHP total install base) is relevant. Far more important is what fraction of the top 50 most commonly installed PHP packages Drupal is, and what fraction of the top 50 the other PHP projects who will agree to move to PHP5 comprise. The long tail of PHP applications is huge -- and mostly ignorable.
..chrisxj
-- Bill Fitzgerald http://www.funnymonkey.com Tools for Teachers 503.897.7160
On 19 Jun 2007, at 12:19, Robert Douglass wrote:
This is not the time to be timid. We as developers know deep in our hearts that PHP5 is the best thing for Drupal. The amazing resonance that Larry and I have been receiving from the developers on other projects shows that we're not alone in thinking this.
I'm playing devil's advocate to make sure that we're all on the same page, and that we understand the consequences of this bet. We're putting a lot at risk here. Mark my words: a PHP5-only Drupal 7 will leave many, many Drupal users behind. -- Dries Buytaert :: http://www.buytaert.net/
On 6/20/07, Dries Buytaert <dries.buytaert@gmail.com> wrote:
I'm playing devil's advocate to make sure that we're all on the same page, and that we understand the consequences of this bet. We're putting a lot at risk here.
Mark my words: a PHP5-only Drupal 7 will leave many, many Drupal users behind.
It's OK...every full Drupal update leaves many Drupal users behind for a significant portion of time. Ponder that :P -- Boris Mann Office 604-682-2889 Skype borismann http://www.bryght.com
It's OK...every full Drupal update leaves many Drupal users behind for a significant portion of time. Ponder that :P I have one site for a conference in three weeks which is still on 4.7. It goes away after the conference so I am not even considering upgrading to 5.1, both because if it ain't broke, don't fix it, and I don't have the time. It is not a static site with no changes, I update it several times a day as the conference start approaches.
I have another site, not on any CMS, that hasn't had a change in two years. It is for a dead project that may become active at some time in the future so it is interesting only historically at the moment. Yes, I would probably change it occasionally if it were on a CMS and easy to do trivial things, but it is not currently worth the time to convert it to Drupal. Eventually something happens that makes it necessary to upgrade, host goes dead, bad security bug appears, mission changes, etc. There are probably still several million PCs running Win98. That is not bad if it is sufficient for the user. We don't have to worry about people still running Drupal 4.6. They will eventually upgrade or drop out no matter what we do.
On 21 Jun 2007, at 08:38, Boris Mann wrote:
Mark my words: a PHP5-only Drupal 7 will leave many, many Drupal users behind.
It's OK...every full Drupal update leaves many Drupal users behind for a significant portion of time. Ponder that :P
Except that this time, we risk leaving 70% of the install base behind. The amount of intelligent/constructive feedback to this thread has been surprisingly low. Let's stick to facts and real arguments, please. I'd like to see a _real_ discussion here. -- Dries Buytaert :: http://www.buytaert.net/
Web services SOAP/WSDL support in PHP4 (libraries like NuSOAP) is simply not that good. The support in PHP5 is significantly better. We are already building modules which rely on external web services, and supposing Drupal will be used in a SOA manner it becomes important.
On 6/26/07, Dries Buytaert <dries.buytaert@gmail.com> wrote:
Except that this time, we risk leaving 70% of the install base behind. The amount of intelligent/constructive feedback to this thread has been surprisingly low. Let's stick to facts and real arguments, please. I'd like to see a _real_ discussion here.
I rather thought we were having a real discussion. I'm tempted to ask just exactly what "real arguments" are. The facts in the situation are preciously few: * PHP5 provides better support than PHP4 for a bunch of technologies. * Many developers of PHP-based software would like to move to PHP5 but are held back by the installed base on PHP4. * The PHP developers would like the developers and users of PHP-based software to move ahead to later releases (per Rasmus Lerdorf), but are similarly held back. * Hosting service providers have little motivation to upgrade in such a situation. Beyond the above, we have little in the way of "facts" to guide us. It's not because people simply have not stated those missing facts. It's because they don't exist. They don't exist because we are operating in an area of many unkowns and guessing the future. One never has all the facts in such a situation, and even when one does, it is impossible to foretell the future. Let me quote Greg Hudson: "It is important not to let the perfect become the enemy of the good, even when you can agree on what perfect is. Doubly so when you can't." As someone who is migrating a bunch of D4.6 sites to D4.7, and planning migration of 60+ sites from D4.7 to D5 and PHP5 at the same time, I already feel very "left behind" by the Drupal 6 crowd. The amount of support for D5, D4.7 and D4.6 users is rapidly evaporating. For 4.x, it's quickly approaching zero. Temporarily leaving 70% of the install base behind will probably be Good Thing. It will result in increased pressure for better documention and support of past releases. It might even foster some empathy for the numerous sites which just cannot keep up with the 9-month, break-everything release cycles Drupal is famous for. That would beneficial, in my view.
On 6/26/07, Chris Johnson <cxjohnson@gmail.com> wrote:
On 6/26/07, Dries Buytaert <dries.buytaert@gmail.com> wrote:
Except that this time, we risk leaving 70% of the install base behind. The amount of intelligent/constructive feedback to this thread has been surprisingly low. Let's stick to facts and real arguments, please. I'd like to see a _real_ discussion here.
I rather thought we were having a real discussion. I'm tempted to ask just exactly what "real arguments" are.
The facts in the situation are preciously few:
* PHP5 provides better support than PHP4 for a bunch of technologies. * Many developers of PHP-based software would like to move to PHP5 but are held back by the installed base on PHP4. * The PHP developers would like the developers and users of PHP-based software to move ahead to later releases (per Rasmus Lerdorf), but are similarly held back. * Hosting service providers have little motivation to upgrade in such a situation.
Beyond the above, we have little in the way of "facts" to guide us. It's not because people simply have not stated those missing facts. It's because they don't exist.
They don't exist because we are operating in an area of many unkowns and guessing the future. One never has all the facts in such a situation, and even when one does, it is impossible to foretell the future.
Let me quote Greg Hudson:
"It is important not to let the perfect become the enemy of the good, even when you can agree on what perfect is. Doubly so when you can't."
As someone who is migrating a bunch of D4.6 sites to D4.7, and planning migration of 60+ sites from D4.7 to D5 and PHP5 at the same time, I already feel very "left behind" by the Drupal 6 crowd. The amount of support for D5, D4.7 and D4.6 users is rapidly evaporating. For 4.x, it's quickly approaching zero.
Temporarily leaving 70% of the install base behind will probably be Good Thing. It will result in increased pressure for better documention and support of past releases. It might even foster some empathy for the numerous sites which just cannot keep up with the 9-month, break-everything release cycles Drupal is famous for. That would beneficial, in my view.
I'm a bit curious about what happened back when MediaWiki dropped PHP4 support. It came out of the blue for me and I did have to wait almost half a year before I could switch to a host that gave me PHP5, so I could update the software. My MediaWiki is really tiny, with at most 5-10 active users and less than 100 hits a day, so I can afford to stay on an old version for a while. But this was back in 2005, two full years ago, when PHP5 was even less widespread than it will be in 2008. I really wonder how many sites were stuck with no upgrade path because they couldn't get PHP5 support, and how the owners dealt with that. If Drupal takes this step, it would be very good to know what to expect; fortunately, there are examples like MediaWiki to judge this. -- Aran
On 6/26/07, Dries Buytaert <dries.buytaert@gmail.com> wrote:
... The amount of intelligent/constructive feedback to this thread has been surprisingly low. Let's stick to facts and real arguments, please. I'd like to see a _real_ discussion here.
-- Dries Buytaert :: http://www.buytaert.net/
I think there has been a great deal of intelligent/constructive feedback, the best of it _strategic_, as for example in the case of Robert Douglass (sorry Robert, this doesn't make you responsible for the stuff I say :) : ...We as developers know deep in our hearts that PHP5 is the best thing for Drupal. ... A PHP5 based Drupal 7 will not cut off the upgrade path of anybody. Let's be honest about upgrading Drupal. If you follow the instructions then you have to be able to export your database (and potentially import it later if you screw things up) and be able to backup the files on your file system, potentially replacing them if the upgrade fails. These are coincidentally the same steps that are needed to migrate your Drupal site to a new web host. There are already plenty of web hosts that support PHP5. Anybody who doesn't have the skills to migrate to one is fairly unlikely to ever upgrade their Drupal installation anyway. We have a chance right now to act in unison with a majority of the meaningful PHP based FOSS projects. This is an action that has no precedent. Never before have we all spoken with one voice in a coordinated way. I feel that failing to take a position of strong leadership based on what we believe is best for our software is the shot in the head, not the other way around. Because, at a time (a very different situation from 2005) in which Drupal wins a Web 2.0 award in the Publishing category from CNET, alongside Flash, WordPress and Blogger, the real debate is, what is the _real_ Drupal roadmap? What is the _real_ Drupal roadmap? To be another script people can install from Fantastico (above and beyond the frequency of update problem); or a CMS Application Framework? If the latter is the case, we must be moving towards an object-oriented paradigm for most of core. That is why partial "compromise" solutions won't work: the ideas of making Drupal _partly_ compatible with PHP4 is a deal breaker. We need to shift to PHP5 to avoid painting ourselves into a "scripty" corner out of which we will never emerge. In order, too, to save PHP5 itself as a decent framework language. That is what is going on: the fight for a Web 2.0 application framework. This requires a gradual paradigm shift only possible with a relatively short-term move to PHP5. So as to gain time in order to be able to be gradual in our paradigm shift to an object oriented framework (some procedural in the innards of core, but an object oriented API). It is that or oblivion, in the long run. Only we can choose. saludos, Victor Kane http://awebfactory.com.ar
Quoting Dries Buytaert <dries.buytaert@gmail.com>:
On 21 Jun 2007, at 08:38, Boris Mann wrote:
Mark my words: a PHP5-only Drupal 7 will leave many, many Drupal users behind.
It's OK...every full Drupal update leaves many Drupal users behind for a significant portion of time. Ponder that :P
Except that this time, we risk leaving 70% of the install base behind. The amount of intelligent/constructive feedback to this thread has been surprisingly low. Let's stick to facts and real arguments, please. I'd like to see a _real_ discussion here.
How do we quantify the 70%? Does all of that 70% have no choice to move to PHP5 if they want? How can we help those who want to move to Drupal 7 but can't because their stuck on PHP4 find a path? What preparation will early announcement have on the 70% to help reduce that number? Those who don't want to upgrade will never upgrade; should we reduce that number from the 70%? We could add a warning to Drupal 6 that Drupal 7 is going to require PHP5 to prepare the masses who upgrade to it. Unless we approach this as all or nothing IMO we will forever remain at PHP4. It is easy for modules to begin to require PHP5 with a simple try/catch coding phrase. Since try/catch cannot be emulated it is a must to move to PHP5 to use the code. So in order to help reduce the migration to PHP5 I do agree that some modules should begin requiring it for Drupal 6. I find PHP5 faster than PHP4 so therefore remaining at PHP4 makes little sense since we are looking for speed improvements. The issue is, for those who have the need to remain at PHP4 how can we continue to support them? We let them remain with Drupal 4, 5 and 6 until such time they are ready to move forward. Earnie
We could potentially make an extended support cycle for Drupal 6, e.g. still when Drupal 8 comes out, however D7 will have php5 required for some of core, as some of us have said, and D8 will be php5 only (but 6 will still be supported) On Jun 26, 2007, at 7:33 AM, Earnie Boyd wrote:
Quoting Dries Buytaert <dries.buytaert@gmail.com>:
On 21 Jun 2007, at 08:38, Boris Mann wrote:
Mark my words: a PHP5-only Drupal 7 will leave many, many Drupal users behind.
It's OK...every full Drupal update leaves many Drupal users behind for a significant portion of time. Ponder that :P
Except that this time, we risk leaving 70% of the install base behind. The amount of intelligent/constructive feedback to this thread has been surprisingly low. Let's stick to facts and real arguments, please. I'd like to see a _real_ discussion here.
How do we quantify the 70%? Does all of that 70% have no choice to move to PHP5 if they want? How can we help those who want to move to Drupal 7 but can't because their stuck on PHP4 find a path? What preparation will early announcement have on the 70% to help reduce that number? Those who don't want to upgrade will never upgrade; should we reduce that number from the 70%? We could add a warning to Drupal 6 that Drupal 7 is going to require PHP5 to prepare the masses who upgrade to it. Unless we approach this as all or nothing IMO we will forever remain at PHP4.
It is easy for modules to begin to require PHP5 with a simple try/ catch coding phrase. Since try/catch cannot be emulated it is a must to move to PHP5 to use the code. So in order to help reduce the migration to PHP5 I do agree that some modules should begin requiring it for Drupal 6. I find PHP5 faster than PHP4 so therefore remaining at PHP4 makes little sense since we are looking for speed improvements. The issue is, for those who have the need to remain at PHP4 how can we continue to support them? We let them remain with Drupal 4, 5 and 6 until such time they are ready to move forward.
Earnie
First: Kudos to everyone for so heavily weighing the needs of users who clearly can't even participate at the developer level -- I'm sure anyone active on this list would suffer little pain from a php5-only D7. Second: The underlying concensus seems to be that this move should probably happen sooner or later, and the debate is about how soon. IMO, there are few strong arguments for NOT providing a php5-only D7. If, in hindsight, it proves to be a mistake (IMO, very unlikely), we can always continue to support D6 and, in the absolute worst case scenario, abandon php5-only benefits and begin backporting. For the broader push for php5-only OS web apps, I think it would make sense to identify a range of web hosts that already provide php5, those that do not, and provide documentation on how to export a site from the latter set and import a site to the former set -- if we expect a large number of users to want to upgrade from php4-only hosts. Is there any evidence for large numbers of Drupal users who will likely want to upgrade who are truly "stuck" on a php4-only host? This is effectively, btw, an organized boycott of php4-only hosts -- but I see that as a good thing. Hell, perhaps an enterprising consultant or shop could solicit "sponsorships" from php5-capable hosts to draft "importing a Drupal site" to their platform and use some of that sponsorship money to help pay to draft "export" documentation for uncooperative hosts. This decision would be a little easier to gauge if we had numbers describing things like how many D4.6/4.7/5.x/6 sites there out there? How "good" are sites at keeping up to date with major version changes (i.e., how many sites tend to upgrade, how long is the lag between release and upgrade, on average, etc.)? How many sites are currently active on php4-only hosts? Etc. Do we have any sense of those numbers? Is there a good way to poll the wider Drupal community to get their thoughts? I.e., how many people on support know this conversation is happening? FWIW, I also think that it might be a good thing to plan a major change on a nice, long timeline and expect to provide extended support for D6 for longer than usual -- as Chris Johnson points out, it's very difficult to keep up with the rapid-fire Drupal release cycle. Scott On 6/26/07, dmitri <dmitri@dmitrizone.com> wrote:
We could potentially make an extended support cycle for Drupal 6, e.g. still when Drupal 8 comes out, however D7 will have php5 required for some of core, as some of us have said, and D8 will be php5 only (but 6 will still be supported)
On Jun 26, 2007, at 7:33 AM, Earnie Boyd wrote:
Quoting Dries Buytaert <dries.buytaert@gmail.com>:
On 21 Jun 2007, at 08:38, Boris Mann wrote:
Mark my words: a PHP5-only Drupal 7 will leave many, many Drupal users behind.
It's OK...every full Drupal update leaves many Drupal users behind for a significant portion of time. Ponder that :P
Except that this time, we risk leaving 70% of the install base behind. The amount of intelligent/constructive feedback to this thread has been surprisingly low. Let's stick to facts and real arguments, please. I'd like to see a _real_ discussion here.
How do we quantify the 70%? Does all of that 70% have no choice to move to PHP5 if they want? How can we help those who want to move to Drupal 7 but can't because their stuck on PHP4 find a path? What preparation will early announcement have on the 70% to help reduce that number? Those who don't want to upgrade will never upgrade; should we reduce that number from the 70%? We could add a warning to Drupal 6 that Drupal 7 is going to require PHP5 to prepare the masses who upgrade to it. Unless we approach this as all or nothing IMO we will forever remain at PHP4.
It is easy for modules to begin to require PHP5 with a simple try/ catch coding phrase. Since try/catch cannot be emulated it is a must to move to PHP5 to use the code. So in order to help reduce the migration to PHP5 I do agree that some modules should begin requiring it for Drupal 6. I find PHP5 faster than PHP4 so therefore remaining at PHP4 makes little sense since we are looking for speed improvements. The issue is, for those who have the need to remain at PHP4 how can we continue to support them? We let them remain with Drupal 4, 5 and 6 until such time they are ready to move forward.
Earnie
The "organized boycott of php4-only hosts" is a somewhat harsh phrasing for what the GoPHP5 effort is trying to do: Get a lot of projects together to say "we're done with PHP 4 now, thank you" at the same time, and give hosts fair warning of that so they can upgrade. Even though the target date for GoPHP5 is February of next year, most projects won't have an actual release until later in the year. Drupal 7 is likely not going to ship until May of next year at the absolute earliest, so we're discussing something that involves a year's warning at least. (And if we were to wait until Drupal 8, we're talking about a 2 year delay until we can fully embrace PHP 5.) Having permitted "value-add" for PHP 5 in core is, honestly, a nice idea but not feasible in practice. That would mean that the form system, menu system, database system, blocks, nodes, throttle, theming system, watchdog, etc. can't use PHP 5. That leaves maybe Aggregator as a core system that could use PHP 5 and get substantial benefit from it. Dries actually OKed that policy for Drupal 6 about 2 months ago, but nothing has happened with it because there's not much in core that we really could do that to. :-) I had suggested adding a taxonomy to project nodes that could flag a minimum PHP version shortly after DrupalCon, but I like the idea of putting it in the info file better. PHP has bulit-in version comparison utilities. Dries, dww, is that something doable/allowable? Optionally offering extended Drupal 6 support would be fine with me, and that's something we could decide "on the fly" later when we see what the market looks like. I hesitate to do that to poor Gabor, though, since that wasn't the plan when he agreed to be D6 maintainer. :-) Gabor, your thoughts? --Larry Garfield On Tue, 26 Jun 2007 14:58:47 -0400, "Scott Trudeau" <strudeau@umich.edu> wrote:
First: Kudos to everyone for so heavily weighing the needs of users who clearly can't even participate at the developer level -- I'm sure anyone active on this list would suffer little pain from a php5-only D7.
Second: The underlying concensus seems to be that this move should probably happen sooner or later, and the debate is about how soon.
IMO, there are few strong arguments for NOT providing a php5-only D7. If, in hindsight, it proves to be a mistake (IMO, very unlikely), we can always continue to support D6 and, in the absolute worst case scenario, abandon php5-only benefits and begin backporting.
For the broader push for php5-only OS web apps, I think it would make sense to identify a range of web hosts that already provide php5, those that do not, and provide documentation on how to export a site from the latter set and import a site to the former set -- if we expect a large number of users to want to upgrade from php4-only hosts. Is there any evidence for large numbers of Drupal users who will likely want to upgrade who are truly "stuck" on a php4-only host? This is effectively, btw, an organized boycott of php4-only hosts -- but I see that as a good thing.
Hell, perhaps an enterprising consultant or shop could solicit "sponsorships" from php5-capable hosts to draft "importing a Drupal site" to their platform and use some of that sponsorship money to help pay to draft "export" documentation for uncooperative hosts.
This decision would be a little easier to gauge if we had numbers describing things like how many D4.6/4.7/5.x/6 sites there out there? How "good" are sites at keeping up to date with major version changes (i.e., how many sites tend to upgrade, how long is the lag between release and upgrade, on average, etc.)? How many sites are currently active on php4-only hosts? Etc. Do we have any sense of those numbers? Is there a good way to poll the wider Drupal community to get their thoughts? I.e., how many people on support know this conversation is happening?
FWIW, I also think that it might be a good thing to plan a major change on a nice, long timeline and expect to provide extended support for D6 for longer than usual -- as Chris Johnson points out, it's very difficult to keep up with the rapid-fire Drupal release cycle.
Scott
On 6/26/07, dmitri <dmitri@dmitrizone.com> wrote:
We could potentially make an extended support cycle for Drupal 6, e.g. still when Drupal 8 comes out, however D7 will have php5 required for some of core, as some of us have said, and D8 will be php5 only (but 6 will still be supported)
On Jun 26, 2007, at 7:33 AM, Earnie Boyd wrote:
Quoting Dries Buytaert <dries.buytaert@gmail.com>:
On 21 Jun 2007, at 08:38, Boris Mann wrote:
Mark my words: a PHP5-only Drupal 7 will leave many, many Drupal users behind.
It's OK...every full Drupal update leaves many Drupal users behind for a significant portion of time. Ponder that :P
Except that this time, we risk leaving 70% of the install base behind. The amount of intelligent/constructive feedback to this thread has been surprisingly low. Let's stick to facts and real arguments, please. I'd like to see a _real_ discussion here.
How do we quantify the 70%? Does all of that 70% have no choice to move to PHP5 if they want? How can we help those who want to move to Drupal 7 but can't because their stuck on PHP4 find a path? What preparation will early announcement have on the 70% to help reduce that number? Those who don't want to upgrade will never upgrade; should we reduce that number from the 70%? We could add a warning to Drupal 6 that Drupal 7 is going to require PHP5 to prepare the masses who upgrade to it. Unless we approach this as all or nothing IMO we will forever remain at PHP4.
It is easy for modules to begin to require PHP5 with a simple try/ catch coding phrase. Since try/catch cannot be emulated it is a must to move to PHP5 to use the code. So in order to help reduce the migration to PHP5 I do agree that some modules should begin requiring it for Drupal 6. I find PHP5 faster than PHP4 so therefore remaining at PHP4 makes little sense since we are looking for speed improvements. The issue is, for those who have the need to remain at PHP4 how can we continue to support them? We let them remain with Drupal 4, 5 and 6 until such time they are ready to move forward.
Earnie
Min version checking is something that modules *can* implement using hook_status, if I remember correctly. Putting it into the info files makes a lot of sense, though -- it's a basic check that's best centralized. --Jeff On Jun 26, 2007, at 4:47 PM, Larry Garfield wrote:
I had suggested adding a taxonomy to project nodes that could flag a minimum PHP version shortly after DrupalCon, but I like the idea of putting it in the info file better. PHP has bulit-in version comparison utilities. Dries, dww, is that something doable/allowable?
On Jun 26, 2007, at 3:51 PM, Jeff Eaton wrote:
Min version checking is something that modules *can* implement using hook_status, if I remember correctly. Putting it into the info files makes a lot of sense, though -- it's a basic check that's best centralized.
Thing is, if it's in the .info files, then the forthcoming drupalorg.module could parse the .info files and implement another project browsing method on d.o and other stuff like that. If it's in a php hook, there's no chance for d.o itself to access the data, since we will not give everyone with a CVS account PHP access to d.o (which is effectively what would happen if we were trying to get this out of a module hook). Plus, on the client side, you want to know this *before* you try to enable it at admin/build/modules (just like core version) so it's gotta be in the .info file for that to work, too. Really, this isn't that hard. Look at the patch I wrote [1]. This is almost identical. It should be trivial to add php in there if someone wants to do it. My hands are full moving update_status into core, so I'm not volunteering. -Derek (dww) [1] http://drupal.org/node/146910 http://drupal.org/cvs?commit=69762 http://drupal.org/files/issues/drupal_core_compatibility.patch_7.txt
On Tuesday 26 June 2007, Derek Wright wrote:
On Jun 26, 2007, at 3:51 PM, Jeff Eaton wrote:
Min version checking is something that modules *can* implement using hook_status, if I remember correctly. Putting it into the info files makes a lot of sense, though -- it's a basic check that's best centralized.
Thing is, if it's in the .info files, then the forthcoming drupalorg.module could parse the .info files and implement another project browsing method on d.o and other stuff like that. If it's in a php hook, there's no chance for d.o itself to access the data, since we will not give everyone with a CVS account PHP access to d.o (which is effectively what would happen if we were trying to get this out of a module hook). Plus, on the client side, you want to know this *before* you try to enable it at admin/build/modules (just like core version) so it's gotta be in the .info file for that to work, too.
Really, this isn't that hard. Look at the patch I wrote [1]. This is almost identical. It should be trivial to add php in there if someone wants to do it. My hands are full moving update_status into core, so I'm not volunteering.
He did however volunteer to review it for me, so here 'tis: http://drupal.org/node/154949 -- Larry Garfield AIM: LOLG42 larry@garfieldtech.com ICQ: 6817012 "If nature has made any one thing less susceptible than all others of exclusive property, it is the action of the thinking power called an idea, which an individual may exclusively possess as long as he keeps it to himself; but the moment it is divulged, it forces itself into the possession of every one, and the receiver cannot dispossess himself of it." -- Thomas Jefferson
One thing we can do is plan to have synchronized, well-publicized free workshops in every major city in the world, at some point well before the official release of Drupal 7, covering all issues (and popular hosting venues) and giving complete hands-on practice on backing up, and upgrading. Each Drupal Users Group can play a major role here in this major step forward. saludos, Victor Kane http://awebfactory.com.ar On 6/20/07, Dries Buytaert <dries.buytaert@gmail.com> wrote:
On 19 Jun 2007, at 12:19, Robert Douglass wrote:
This is not the time to be timid. We as developers know deep in our hearts that PHP5 is the best thing for Drupal. The amazing resonance that Larry and I have been receiving from the developers on other projects shows that we're not alone in thinking this.
I'm playing devil's advocate to make sure that we're all on the same page, and that we understand the consequences of this bet. We're putting a lot at risk here.
Mark my words: a PHP5-only Drupal 7 will leave many, many Drupal users behind.
-- Dries Buytaert :: http://www.buytaert.net/
Quoting Dries Buytaert <dries.buytaert@gmail.com>:
Mark my words: a PHP5-only Drupal 7 will leave many, many Drupal users behind.
There will always be those that get left behind; that is even true if you stay with PHP4. They can always use an older version or the services of others to upgrade. Only weeks ago I would have joined you in your caution but I believe that the only way out of the whirlwind is to take a different path and the pavement is being laid. Now is the time to jump on the bandwagon driving down the new laid pavement. PHP 6 will be out soon enough and we're still toying with PHP 4. Go PHP 5, Go! Why are there always those that are left behind? One reason is fear. People are of afraid of change and what it might mean to them doing things differently or seeing things with a different view. Another reason is laziness. People setup the product and just don't want to be bothered with changing it. They like the way it is and just don't want to spend time upgrading. There are probably other reasons but as I see it, these are the top two. Earnie
On 6/20/07, Dries Buytaert <dries.buytaert@gmail.com> wrote:
On 19 Jun 2007, at 12:19, Robert Douglass wrote:
This is not the time to be timid. We as developers know deep in our hearts that PHP5 is the best thing for Drupal. The amazing resonance that Larry and I have been receiving from the developers on other projects shows that we're not alone in thinking this.
I'm playing devil's advocate to make sure that we're all on the same page, and that we understand the consequences of this bet. We're putting a lot at risk here.
Mark my words: a PHP5-only Drupal 7 will leave many, many Drupal users behind.
Dries is making the same point I made here: http://lists.drupal.org/pipermail/development/2007-June/024609.html We have to be aware that this is a gamble (with a good cause, good results, and good chance of success). However, what if the world stays back? Just think of it and what the impact is if that (hopefully unlikely) situation arises. -- 2bits.com http://2bits.com Drupal development, customization and consulting.
Quoting Khalid Baheyeldin <kb@2bits.com>:
However, what if the world stays back? Just think of it and what the impact is if that (hopefully unlikely) situation arises.
We cannot concretely deal with a situation that does not yet exist. We can only deal with the situation *if* and *only* if it arises. If we never cut the hole in the dam the we will forever be behind it and scratching the RASH. :) If the situation arises we will can provide a solution as we will then understand the situation. If we try to provide solution before the situation arises we are likely to miss the mark and just burden ourselves with needless soultions. Remaining on PHP 4 isn't the solution. Earnie
The CentOS team is pleased to announce the availability of CentOS 5.0. Major changes in CentOS 5 compared to CentOS 4 include:
These updated software versions: Apache-2.2, php-5.1.6
... RHEL 5 has moved all the way up to PHP 5.1.6 from RHEL 4's mere PHP 4.3.9
We are getting there.
Now there's another catch there. Its far easier to upgrade the PHP/Apache than OS. Lets see: 1 ) Most people manage their servers remotely even people who pay for dedicated servers. So upgrading PHP means if it goes wrong I am still logged on to my server and can revert back. If I upgrade my OS and it goes wrong I am stuck and it goes out of my control as most people cant install OS remotely. 2 ) Most production server will be Ok with upgrading PHP/Apache as its still application than upgrading the OS. You can always have php4 & php5 installed on same machine. 3 ) RHEL5/CentOS5 have recently been launched and there are issue with stability. So most people will wait till its stabilized before upgrading. So as I understand new servers may use RHEL5/CentOS5 older servers are in no rush to upgrade. Now even if RHEL5 becomes prominent platform we are again stuck with PHP 5.1 , while we are planning to target PHP 5.2 . So Jonathan is right that we are moving at pace of RedHat. Although I am in full favor of move to php 5.2 soon but hosting companies, which rely on "yum update" for maintaining their servers, wont come on board esily. ___________________________________________________________________________________ You snooze, you lose. Get messages ASAP with AutoCheck in the all-new Yahoo! Mail Beta. http://advision.webevents.yahoo.com/mailbeta/newmail_html.html
On Wednesday 06 June 2007, Vivek Puri wrote:
Now there's another catch there. Its far easier to upgrade the PHP/Apache than OS. Lets see: 1 ) Most people manage their servers remotely even people who pay for dedicated servers. So upgrading PHP means if it goes wrong I am still logged on to my server and can revert back. If I upgrade my OS and it goes wrong I am stuck and it goes out of my control as most people cant install OS remotely. 2 ) Most production server will be Ok with upgrading PHP/Apache as its still application than upgrading the OS. You can always have php4 & php5 installed on same machine. 3 ) RHEL5/CentOS5 have recently been launched and there are issue with stability. So most people will wait till its stabilized before upgrading. So as I understand new servers may use RHEL5/CentOS5 older servers are in no rush to upgrade.
Now even if RHEL5 becomes prominent platform we are again stuck with PHP 5.1 , while we are planning to target PHP 5.2 .
So Jonathan is right that we are moving at pace of RedHat. Although I am in full favor of move to php 5.2 soon but hosting companies, which rely on "yum update" for maintaining their servers, wont come on board esily.
OS upgrades aren't really a concern for us. We're just interested in the PHP version. Although Red Hat's latest version ships 5.1.6 by default, that version apparently had some nasty bugs anyway from what I have heard. I think the most telling data, though, is this: http://www.nexen.net/chiffres_cles/phpversion/16987-php_stats_evolution_for_... See the last chart especially. A month ago, 5.2 was already within spitting distance of surpassing 5.1 installations, and 5.1 is in slight decline. By next February, at that rate it should have a healthy chunk of the market already, even without us pushing. Not enough that we could abandon PHP 4 without pushing, but enough that it's a safe target, I believe. -- Larry Garfield AIM: LOLG42 larry@garfieldtech.com ICQ: 6817012 "If nature has made any one thing less susceptible than all others of exclusive property, it is the action of the thinking power called an idea, which an individual may exclusively possess as long as he keeps it to himself; but the moment it is divulged, it forces itself into the possession of every one, and the receiver cannot dispossess himself of it." -- Thomas Jefferson
OS upgrades aren't really a concern for us. We're just interested in the PHP version.
true but I was responding to how PHP versions are tied to version of OS. ____________________________________________________________________________________ Sick sense of humor? Visit Yahoo! TV's Comedy with an Edge to see what's on, when. http://tv.yahoo.com/collections/222
I suspect a lot would change very quickly if web hosts started to perceive that they were losing customers to other webhosts who ran modern PHP versions. Vivek Puri wrote:
OS upgrades aren't really a concern for us. We're just interested in the PHP version.
true but I was responding to how PHP versions are tied to version of OS.
____________________________________________________________________________________ Sick sense of humor? Visit Yahoo! TV's Comedy with an Edge to see what's on, when. http://tv.yahoo.com/collections/222
Actually, it almost doesn't matter. IF a substantial portion of the larger php based projects move next year, it doesn't mean that their products that have been developed to work up until that point will cease working. It just means that to do anything moving forward people will need to upgrade. This really isn't different then what happens now. Besides, this strategy has worked successfully for Microsoft for years. It will work for the Open Source community just as well. Announce a date, stick to it, move forward. People who don't upgrade now, are the ones who won't upgrade then. -sp
-----Original Message----- From: development-bounces@drupal.org [mailto:development- bounces@drupal.org] On Behalf Of Robert Douglass Sent: Thursday, June 07, 2007 12:34 AM To: development@drupal.org Subject: Re: [development] Go PHP 5, Go!
I suspect a lot would change very quickly if web hosts started to perceive that they were losing customers to other webhosts who ran modern PHP versions.
Vivek Puri wrote:
OS upgrades aren't really a concern for us. We're just interested in the PHP version.
true but I was responding to how PHP versions are tied to version of OS.
_______________________________________________________________________ _____________
Sick sense of humor? Visit Yahoo! TV's Comedy with an Edge to see what's on, when. http://tv.yahoo.com/collections/222
Announce a date, stick to it, move forward. People who don't upgrade now, are the ones who won't upgrade then.
I definitely agree with that , just trying to be devil's advocate here ;) In a scenario where Drupal moves to PHP 5.2 on Feb 2008, lets say its Drupal 7 then. Will we continue to support Drupal 5.X & Dupal 6.X on PHP 4.X ? Or after the cutoff date all patches and upgrade will be only valid on PHP 5.2 ? ____________________________________________________________________________________ Got a little couch potato? Check out fun summer activities for kids. http://search.yahoo.com/search?fr=oni_on_mail&p=summer+activities+for+kids&c...
I definitely agree with that , just trying to be devil's advocate here ;) In a scenario where Drupal moves to PHP 5.2 on Feb 2008, lets say its Drupal 7 then. Will we continue to support Drupal 5.X & Dupal 6.X on PHP 4.X ? Or after the cutoff date all patches and upgrade will be only valid on PHP 5.2 ?
I think we'll continue to maintain the PHP 4-compatible version as PHP 4-compatible until the release itself becomes obsolete.
On Thursday 07 June 2007, David Strauss wrote:
I definitely agree with that , just trying to be devil's advocate here ;) In a scenario where Drupal moves to PHP 5.2 on Feb 2008, lets say its Drupal 7 then. Will we continue to support Drupal 5.X & Dupal 6.X on PHP 4.X ? Or after the cutoff date all patches and upgrade will be only valid on PHP 5.2 ?
I think we'll continue to maintain the PHP 4-compatible version as PHP 4-compatible until the release itself becomes obsolete.
Yes, the agreement is that new feature releases post that date will assume 5.2. Projects are free to maintain support for older releases however they like for as long as they like. Just like Drupal 5 is/will be maintained as MySQL 3 compatible for as long as Drupal 5 is maintained even though Drupal 6 will require MySQL 4.1. -- Larry Garfield AIM: LOLG42 larry@garfieldtech.com ICQ: 6817012 "If nature has made any one thing less susceptible than all others of exclusive property, it is the action of the thinking power called an idea, which an individual may exclusively possess as long as he keeps it to himself; but the moment it is divulged, it forces itself into the possession of every one, and the receiver cannot dispossess himself of it." -- Thomas Jefferson
Robert Douglass wrote:
I suspect a lot would change very quickly if web hosts started to perceive that they were losing customers to other webhosts who ran modern PHP versions.
This is OT, but I think that Red Hat has made a mistake by shipping 5.1.6 with RHEL5. RHEL5.1 is about to come in Q3, let's push on Red Hat to include 5.2 in separate packages (they can't upgrade now) And yes, i know this would be hard... Jakub
On Thursday 07 June 2007, Jakub Suchy wrote:
Robert Douglass wrote:
I suspect a lot would change very quickly if web hosts started to perceive that they were losing customers to other webhosts who ran modern PHP versions.
This is OT, but I think that Red Hat has made a mistake by shipping 5.1.6 with RHEL5. RHEL5.1 is about to come in Q3, let's push on Red Hat to include 5.2 in separate packages (they can't upgrade now)
And yes, i know this would be hard...
Jakub
Hopefully seeing an effort such as this, which should go public well before Q3, will be a notice to Red Hat as well as to web hosts. Of course, RH is known for upping the version of software they include and patching the hell out of it but not actually changing the version number, so who knows what it it is they actually run. :-) It certainly seems like everyone is in favor of this idea. We need to decide how we're going to formally "sign on". Dries, that's your call to decide how we go about it. -- Larry Garfield AIM: LOLG42 larry@garfieldtech.com ICQ: 6817012 "If nature has made any one thing less susceptible than all others of exclusive property, it is the action of the thinking power called an idea, which an individual may exclusively possess as long as he keeps it to himself; but the moment it is divulged, it forces itself into the possession of every one, and the receiver cannot dispossess himself of it." -- Thomas Jefferson
Larry Garfield wrote:
It certainly seems like everyone is in favor of this idea. We need to decide how we're going to formally "sign on". Dries, that's your call to decide how we go about it.
That's not so hard... we just need a page on Drupal.org that says we're going to do it. This should go live when the website goes live. Having a public page on the project website is the requirement that all projects have to meet in order to be listed as having joined the effort.
On 6/6/07, Vivek Puri <crystalcube@yahoo.com> wrote:
OS upgrades aren't really a concern for us. We're just interested in the PHP version.
true but I was responding to how PHP versions are tied to version of OS.
CentOS has long had the CentOS Plus repository (see http://www.jimohalloran.com/2005/08/30/php-5-for-centos-41/). PHP5 has been available for quite some time. One of the pushes of this process could be banding together and promoting/recommending a certain minimum version. Getting the CentOS Plus maintainers on board to support whatever version of PHP 5.2.x we think is best would really help with adoption... -- Boris Mann Office 604-682-2889 Skype borismann http://www.bryght.com
Drupal has a number of custom fixes (e.g. function drupal_clone()) to deal with differences between PHP 4 and PHP 5 - it would be nice to not have to worry about such things. -Peter On 6/6/07, Sean Robertson <seanr@ngpsoftware.com> wrote:
Are there currently that many PHP4 apps that won't run on PHP5? I thought Drupal already does. Why can't ISPs upgrade without breaking older apps?
I am a huge fan of this idea overall, BTW. I think migrating apps to PHP5 will start forcing ISPs to at least provide the option to upgrade since they'll start losing customers if they don't. At my company, we own our own servers so it'd be relatively easy for us to upgrade.
Larry Garfield wrote:
This is a follow-up to the PHP 5 thread from a week or two ago. It looks like some momentum is building. Ken Rickard, Robert Douglas, and I have been talking with some of the Jooma folks, and have a working draft of the "core statement and justification". That is, what the goal is and why it is. Joomla's development team is discussing the matter and is leaning yes. Based on the earlier thread here I am hoping that there isn't much objection to Drupal participating in the "Go PHP5" effort. :-) So far Joomla is leaning yes, CakePHP is interested, and I had a positive first response from Typo3. Robert Douglas has volunteered himself to setup a web site for it.
I'm not sure how Dries wants to handle the question of Drupal's participation (by vote, by consensus, or by fiat). Dries?
Anyway, here's the working statement. Consider this an official recommendation that Drupal commit to participating in this effort.
------------------------------------ PHP 4 has served the web developer community for seven years now, and served it well. However, it also shows its age. Most of PHP 4's shortcomings have been addressed by PHP 5, released three years ago, but the transition from PHP 4 to PHP 5 has been slow for a number of reasons.
PHP developers cannot leverage PHP 5's full potential without dropping support for PHP 4, but PHP 4 is still installed on a majority of shared web hosts and users would then be forced to switch to a different application. Web hosts cannot upgrade their servers to PHP 5 without making it impossible for their users to run PHP 4-targeted web apps, and have no incentive to go to the effort of testing and deploying PHP 5 while most web apps are still compatible with PHP 4 and the PHP development team still provides maintenance support for PHP 4. The PHP development team, of course, can't drop maintenance support for PHP 4 while most web hosts still run PHP 4.
It is a dangerous cycle, and one that needs to be broken. The open source PHP developer community has decided that it is indeed now time to move forward, together. Therefore, the listed open source PHP projects have all agreed that effective 5 February 2008, any new feature release will have a minimum required PHP version no older than PHP 5.2.0. It is our believe that this will allow web hosts a reason to upgrade and the PHP development team the ability to retire PHP 4 and focus efforts on PHP 5 and the forthcoming PHP 6, all without penalizing any existing project for being "first out of the gate". ------------------------------------
Notes: - I chose the date because I figure that will be 7-8 months after we officially announce this thing, which I believe should be sufficient time for web hosts. It also comes out to 5/2/2008 (using European convention), and I just like inside references like that. :-) - This does not preclude any project from moving before the deadline date, or from supporting older versions for however long they wish to. That's up to each project. - PHP 5.2 is already the most widely installed version of PHP 5, based on the latest published stats. I know at least two web hosts I work with that either have jumped or are in the process of jumping from PHP 4 straight to PHP 5.2. By the target date it will have been out for nearly a year and a half. It also adds a number of new and useful core features (filter_input, json, a stable PDO, etc.). It's a good version to target.
Thoughts? Comments? Support? Rotten tomatoes?
-- Sean Robertson Web Developer NGP Software, Inc. seanr@ngpsoftware.com (202) 686-9330 http://www.ngpsoftware.com
I don't know a number, but that's one of the fears that there may be. A lot of it could be custom code that someone wrote 6 years ago to PHP 4 standards that will have subtle problems in PHP 5, and they're worried about breaking that. It's the same reason so many web hosts still run register_globals On, even though it's a glaring security hole; they have thousands of clients, and if even 50 of them have code that would stop working without register_globals that's enough to make them shy away from enforcing it. Any distributed PHP applications or frameworks that are not PHP 5 compatible at this point I would consider officially abandoned, honestly. --Larry Garfield On Wed, 06 Jun 2007 10:33:39 -0400, Sean Robertson <seanr@ngpsoftware.com> wrote:
Are there currently that many PHP4 apps that won't run on PHP5? I thought Drupal already does. Why can't ISPs upgrade without breaking older apps?
I am a huge fan of this idea overall, BTW. I think migrating apps to PHP5 will start forcing ISPs to at least provide the option to upgrade since they'll start losing customers if they don't. At my company, we own our own servers so it'd be relatively easy for us to upgrade.
Larry Garfield wrote:
This is a follow-up to the PHP 5 thread from a week or two ago. It looks like some momentum is building. Ken Rickard, Robert Douglas, and I have been talking with some of the Jooma folks, and have a working draft of the "core statement and justification". That is, what the goal is and why it is. Joomla's development team is discussing the matter and is leaning yes. Based on the earlier thread here I am hoping that there isn't much objection to Drupal participating in the "Go PHP5" effort. :-) So far Joomla is leaning yes, CakePHP is interested, and I had a positive first response from Typo3. Robert Douglas has volunteered himself to setup a web site for it.
I'm not sure how Dries wants to handle the question of Drupal's participation (by vote, by consensus, or by fiat). Dries?
Anyway, here's the working statement. Consider this an official recommendation that Drupal commit to participating in this effort.
------------------------------------ PHP 4 has served the web developer community for seven years now, and served it well. However, it also shows its age. Most of PHP 4's shortcomings have been addressed by PHP 5, released three years ago, but the transition from PHP 4 to PHP 5 has been slow for a number of reasons.
PHP developers cannot leverage PHP 5's full potential without dropping support for PHP 4, but PHP 4 is still installed on a majority of shared web hosts and users would then be forced to switch to a different application. Web hosts cannot upgrade their servers to PHP 5 without making it impossible for their users to run PHP 4-targeted web apps, and have no incentive to go to the effort of testing and deploying PHP 5 while most web apps are still compatible with PHP 4 and the PHP development team still provides maintenance support for PHP 4. The PHP development team, of course, can't drop maintenance support for PHP 4 while most web hosts still run PHP 4.
It is a dangerous cycle, and one that needs to be broken. The open source PHP developer community has decided that it is indeed now time to move forward, together. Therefore, the listed open source PHP projects have all agreed that effective 5 February 2008, any new feature release will have a minimum required PHP version no older than PHP 5.2.0. It is our believe that this will allow web hosts a reason to upgrade and the PHP development team the ability to retire PHP 4 and focus efforts on PHP 5 and the forthcoming PHP 6, all without penalizing any existing project for being "first out of the gate". ------------------------------------
Notes: - I chose the date because I figure that will be 7-8 months after we officially announce this thing, which I believe should be sufficient time for web hosts. It also comes out to 5/2/2008 (using European convention), and I just like inside references like that. :-) - This does not preclude any project from moving before the deadline date, or from supporting older versions for however long they wish to. That's up to each project. - PHP 5.2 is already the most widely installed version of PHP 5, based on the latest published stats. I know at least two web hosts I work with that either have jumped or are in the process of jumping from PHP 4 straight to PHP 5.2. By the target date it will have been out for nearly a year and a half. It also adds a number of new and useful core features (filter_input, json, a stable PDO, etc.). It's a good version to target.
Thoughts? Comments? Support? Rotten tomatoes?
-- Sean Robertson Web Developer NGP Software, Inc. seanr@ngpsoftware.com (202) 686-9330 http://www.ngpsoftware.com
On 06/06/07, Larry Garfield <larry@garfieldtech.com> wrote:
This is a follow-up to the PHP 5 thread from a week or two ago. It looks like some momentum is building. Ken Rickard, Robert Douglas, and I have been talking with some of the Jooma folks, and have a working draft of the "core statement and justification". That is, what the goal is and why it is. Joomla's development team is discussing the matter and is leaning yes. Based on the earlier thread here I am hoping that there isn't much objection to Drupal participating in the "Go PHP5" effort. :-) So far Joomla is leaning yes, CakePHP is interested, and I had a positive first response from Typo3. Robert Douglas has volunteered himself to setup a web site for it.
Excellent work and a truly noble cause :) Are there plans to perhaps garner more community support via /. or digg? Perhaps a website might be in order. If there's any way I can help out, please let me know. -K
Karthik wrote:
Excellent work and a truly noble cause :) Are there plans to perhaps garner more community support via /. or digg? Perhaps a website might be in order.
If there's any way I can help out, please let me know. -K
A (Drupal based) website is in the works. Thanks for the offer =) There will be plenty of /. and digg action going, when the time comes.
Need any help with the design? I do graphic design (web and print) for a living. Here are some samples of my work: http://www.murtha.org http://www.vetsformurtha.org http://www.johnlewisforcongress.com http://www.ksdp.org http://www.johnconyers.com http://www.ngpsoftware.com/lefty08 All of those are Drupal sites, BTW. Robert Douglass wrote:
Karthik wrote:
Excellent work and a truly noble cause :) Are there plans to perhaps garner more community support via /. or digg? Perhaps a website might be in order.
If there's any way I can help out, please let me know. -K
A (Drupal based) website is in the works. Thanks for the offer =) There will be plenty of /. and digg action going, when the time comes.
-- Sean Robertson Web Developer NGP Software, Inc. seanr@ngpsoftware.com (202) 686-9330 http://www.ngpsoftware.com
I applaud this effort and submit the following for consideration. Be careful in questioning the "why" question with ISPs, I would not lump them into a single "lazy" category. There are many, including Dreamhost, that provide customers a choice between PHP 4 or 5. Perhaps constructive suggestions on how ISPs/Hosts can configure their hosting environment to support the cause and why they should (security, performance, etc.). It's been mentioned by others in this thread already, the problem isn't FOSS compatibility with PHP 5, it's custom app compatibility with PHP 5. If I'm a developer and am not going to get paid to upgrade my customer's PHP 4 site, why should I even bother? Here lies the biggest challenge of this effort, IMHO. I think providing information on upgrading apps from 4 to 5 would be helpful, but probably won't yield great results. Practical transition plans are what needed. Perhaps allowing visitors to share their migration/upgrade experiences would be helpful. - Chad Kieffer On Jun 5, 2007, at 11:25 PM, Larry Garfield wrote:
This is a follow-up to the PHP 5 thread from a week or two ago. It looks like some momentum is building. Ken Rickard, Robert Douglas, and I have been talking with some of the Jooma folks, and have a working draft of the "core statement and justification". That is, what the goal is and why it is. Joomla's development team is discussing the matter and is leaning yes. Based on the earlier thread here I am hoping that there isn't much objection to Drupal participating in the "Go PHP5" effort. :-) So far Joomla is leaning yes, CakePHP is interested, and I had a positive first response from Typo3. Robert Douglas has volunteered himself to setup a web site for it.
I'm not sure how Dries wants to handle the question of Drupal's participation (by vote, by consensus, or by fiat). Dries?
Anyway, here's the working statement. Consider this an official recommendation that Drupal commit to participating in this effort.
------------------------------------ PHP 4 has served the web developer community for seven years now, and served it well. However, it also shows its age. Most of PHP 4's shortcomings have been addressed by PHP 5, released three years ago, but the transition from PHP 4 to PHP 5 has been slow for a number of reasons.
PHP developers cannot leverage PHP 5's full potential without dropping support for PHP 4, but PHP 4 is still installed on a majority of shared web hosts and users would then be forced to switch to a different application. Web hosts cannot upgrade their servers to PHP 5 without making it impossible for their users to run PHP 4-targeted web apps, and have no incentive to go to the effort of testing and deploying PHP 5 while most web apps are still compatible with PHP 4 and the PHP development team still provides maintenance support for PHP 4. The PHP development team, of course, can't drop maintenance support for PHP 4 while most web hosts still run PHP 4.
It is a dangerous cycle, and one that needs to be broken. The open source PHP developer community has decided that it is indeed now time to move forward, together. Therefore, the listed open source PHP projects have all agreed that effective 5 February 2008, any new feature release will have a minimum required PHP version no older than PHP 5.2.0. It is our believe that this will allow web hosts a reason to upgrade and the PHP development team the ability to retire PHP 4 and focus efforts on PHP 5 and the forthcoming PHP 6, all without penalizing any existing project for being "first out of the gate". ------------------------------------
Notes: - I chose the date because I figure that will be 7-8 months after we officially announce this thing, which I believe should be sufficient time for web hosts. It also comes out to 5/2/2008 (using European convention), and I just like inside references like that. :-) - This does not preclude any project from moving before the deadline date, or from supporting older versions for however long they wish to. That's up to each project. - PHP 5.2 is already the most widely installed version of PHP 5, based on the latest published stats. I know at least two web hosts I work with that either have jumped or are in the process of jumping from PHP 4 straight to PHP 5.2. By the target date it will have been out for nearly a year and a half. It also adds a number of new and useful core features (filter_input, json, a stable PDO, etc.). It's a good version to target.
Thoughts? Comments? Support? Rotten tomatoes?
FYI, I just found this quite interesting post on Hiverminds Magazine via PHPDeveloper.org [1] - so this is already drawing quite some attention (which is a Good Thing, IMHO). http://www.hiveminds.co.uk/node/3409 regards, frando [1] http://www.phpdeveloper.org/news/8146 Larry Garfield schrieb:
This is a follow-up to the PHP 5 thread from a week or two ago. It looks like some momentum is building. Ken Rickard, Robert Douglas, and I have been talking with some of the Jooma folks, and have a working draft of the "core statement and justification". That is, what the goal is and why it is. Joomla's development team is discussing the matter and is leaning yes. Based on the earlier thread here I am hoping that there isn't much objection to Drupal participating in the "Go PHP5" effort. :-) So far Joomla is leaning yes, CakePHP is interested, and I had a positive first response from Typo3. Robert Douglas has volunteered himself to setup a web site for it.
I'm not sure how Dries wants to handle the question of Drupal's participation (by vote, by consensus, or by fiat). Dries?
Anyway, here's the working statement. Consider this an official recommendation that Drupal commit to participating in this effort.
------------------------------------ PHP 4 has served the web developer community for seven years now, and served it well. However, it also shows its age. Most of PHP 4's shortcomings have been addressed by PHP 5, released three years ago, but the transition from PHP 4 to PHP 5 has been slow for a number of reasons.
PHP developers cannot leverage PHP 5's full potential without dropping support for PHP 4, but PHP 4 is still installed on a majority of shared web hosts and users would then be forced to switch to a different application. Web hosts cannot upgrade their servers to PHP 5 without making it impossible for their users to run PHP 4-targeted web apps, and have no incentive to go to the effort of testing and deploying PHP 5 while most web apps are still compatible with PHP 4 and the PHP development team still provides maintenance support for PHP 4. The PHP development team, of course, can't drop maintenance support for PHP 4 while most web hosts still run PHP 4.
It is a dangerous cycle, and one that needs to be broken. The open source PHP developer community has decided that it is indeed now time to move forward, together. Therefore, the listed open source PHP projects have all agreed that effective 5 February 2008, any new feature release will have a minimum required PHP version no older than PHP 5.2.0. It is our believe that this will allow web hosts a reason to upgrade and the PHP development team the ability to retire PHP 4 and focus efforts on PHP 5 and the forthcoming PHP 6, all without penalizing any existing project for being "first out of the gate". ------------------------------------
Notes: - I chose the date because I figure that will be 7-8 months after we officially announce this thing, which I believe should be sufficient time for web hosts. It also comes out to 5/2/2008 (using European convention), and I just like inside references like that. :-) - This does not preclude any project from moving before the deadline date, or from supporting older versions for however long they wish to. That's up to each project. - PHP 5.2 is already the most widely installed version of PHP 5, based on the latest published stats. I know at least two web hosts I work with that either have jumped or are in the process of jumping from PHP 4 straight to PHP 5.2. By the target date it will have been out for nearly a year and a half. It also adds a number of new and useful core features (filter_input, json, a stable PDO, etc.). It's a good version to target.
Thoughts? Comments? Support? Rotten tomatoes?
Hi all, Larry, Ken and I have been busy in the background and we're getting close to launching a site that will be dedicated to advancing PHP 5.2. We have a number of big-name PHP projects signed up (not going to ruin the surprise yet ;-) as well as a way to list web hosts that offer accounts with PHP 5.2. What we need now is a logo. In addition, I'd like to have a button that we can offer for people to put on their sites to show support. If anyone can help us come up with these assets, it would be greatly appreciated. The name of the website is gophp5.org and I'll give you access to the site if you're interested in working on the logo. Cheers, Robert
participants (41)
-
Alexander Barth -
Arancaytar Ilyaran -
Ashraf Amayreh -
Bill Fitzgerald -
Boris Mann -
Chad Kieffer -
Chris Johnson -
Chris Kennedy -
David Strauss -
Derek Wright -
dmitri -
Dries Buytaert -
Earl Miles -
Earnie Boyd -
Eric Tucker -
FGM -
Frando (Franz Heinzmann) -
Frederik 'Freso' S. Olesen -
Gabor Hojtsy -
Gerhard Killesreiter -
Ian Ward -
Jakub Suchy -
James Walker -
Jeff Eaton -
Johan Forngren -
Jonathan Lambert -
Karoly Negyesi -
Karthik -
Khalid Baheyeldin -
Konstantin Käfer -
Larry Garfield -
Morbus Iff -
Peter Wolanin -
Robert Douglass -
Scott Trudeau -
Sean Robertson -
sime -
Steven Peck -
Victor Kane -
Vivek Puri -
Walt Daniels