[drupal-devel] Path system scaleability
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I've recently found myself incredibly impressed with the autopath module, but I realize that there is an inherent scaleability issue with how we do paths at the moment. On every page load, the entire url_alias table is loaded into memory, and enabling the autopath module on a site like Drupal.org with thousands upon thousands of nodes, is just impractical. Does anyone have an idea of how we could make the path system easier to use for large sites? - -- Adrian Rossouw Drupal developer and Bryght Guy http://drupal.org | http://bryght.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (Darwin) iD8DBQFCZPIagegMqdGlkasRAsYLAJ0TBqKVHU+3QL9J2TgdVar/r2LHqgCfbmWu zINs3OqZ7AO3lHFYCIMDHOY= =7IEu -----END PGP SIGNATURE-----
I would think that we could employ a system similar to locale.module. It translates a thousand of strings on a large page and seems to perform adequately. -moshe Adrian Rossouw wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
I've recently found myself incredibly impressed with the autopath module, but I realize that there is an inherent scaleability issue with how we do paths at the moment.
On every page load, the entire url_alias table is loaded into memory, and enabling the autopath module on a site like Drupal.org with thousands upon thousands of nodes, is just impractical. Does anyone have an idea of how we could make the path system easier to use for large sites?
- -- Adrian Rossouw Drupal developer and Bryght Guy http://drupal.org | http://bryght.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (Darwin)
iD8DBQFCZPIagegMqdGlkasRAsYLAJ0TBqKVHU+3QL9J2TgdVar/r2LHqgCfbmWu zINs3OqZ7AO3lHFYCIMDHOY= =7IEu -----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I just came upon a solution that I think would work well. We would : 1) add a url_alias column to the node table 2) modify node_load to use the url_alias column, if set, otherwise default to node/view/$node->nid, and set the $node->link variable. 3) change all l('node/view/ . $node->nid) to l($node->link); Then pathauto can just implement the nodeapi hook to set the url_alias to whatever it wants, and the url_alias table will not grow unruly. I will be rolling a patch for this soon. On 19 Apr 2005, at 2:23 PM, Moshe Weitzman wrote:
I would think that we could employ a system similar to locale.module. It translates a thousand of strings on a large page and seems to perform adequately.
-moshe
Adrian Rossouw wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I've recently found myself incredibly impressed with the autopath module, but I realize that there is an inherent scaleability issue with how we do paths at the moment. On every page load, the entire url_alias table is loaded into memory, and enabling the autopath module on a site like Drupal.org with thousands upon thousands of nodes, is just impractical. Does anyone have an idea of how we could make the path system easier to use for large sites? - -- Adrian Rossouw Drupal developer and Bryght Guy http://drupal.org | http://bryght.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (Darwin) iD8DBQFCZPIagegMqdGlkasRAsYLAJ0TBqKVHU+3QL9J2TgdVar/r2LHqgCfbmWu zINs3OqZ7AO3lHFYCIMDHOY= =7IEu -----END PGP SIGNATURE-----
- -- Adrian Rossouw Drupal developer and Bryght Guy http://drupal.org | http://bryght.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (Darwin) iD8DBQFCZU6/gegMqdGlkasRAmbEAJ9b775ZxHDZN9SLNx4AA6pRDNLG0QCfVIH9 HHfsunWsGlY51H44aZahnh0= =fwEw -----END PGP SIGNATURE-----
On Tue, 19 Apr 2005, Adrian Rossouw wrote:
We would : 1) add a url_alias column to the node table 2) modify node_load to use the url_alias column, if set, otherwise default to node/view/$node->nid, and set the $node->link variable. 3) change all l('node/view/ . $node->nid) to l($node->link);
Then pathauto can just implement the nodeapi hook to set the url_alias to whatever it wants, and the url_alias table will not grow unruly.
I don't like that approach. It makes aliases for nodes a special case and does not cater for user profile aliases. I have an alternative idea: Treat the url_aliases table like we treat the variables table. Cache it in the cache table. (this is more or less the same as Moshe suggested.) Cheers, Gerhard
So you want to treat the node view url as a special case? Hmmm, that is going to be a tough sell. Please explain why this must be so in the issue ... Also explain what the real problem is with a large url_alias table. Metrics which illustrate the problem would be candy. -moshe Adrian Rossouw wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
I just came upon a solution that I think would work well.
We would : 1) add a url_alias column to the node table 2) modify node_load to use the url_alias column, if set, otherwise default to node/view/$node->nid, and set the $node->link variable. 3) change all l('node/view/ . $node->nid) to l($node->link);
Then pathauto can just implement the nodeapi hook to set the url_alias to whatever it wants, and the url_alias table will not grow unruly.
I will be rolling a patch for this soon.
On 19 Apr 2005, at 2:23 PM, Moshe Weitzman wrote:
I would think that we could employ a system similar to locale.module. It translates a thousand of strings on a large page and seems to perform adequately.
-moshe
Adrian Rossouw wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I've recently found myself incredibly impressed with the autopath module, but I realize that there is an inherent scaleability issue with how we do paths at the moment. On every page load, the entire url_alias table is loaded into memory, and enabling the autopath module on a site like Drupal.org with thousands upon thousands of nodes, is just impractical. Does anyone have an idea of how we could make the path system easier to use for large sites? - -- Adrian Rossouw Drupal developer and Bryght Guy http://drupal.org | http://bryght.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (Darwin) iD8DBQFCZPIagegMqdGlkasRAsYLAJ0TBqKVHU+3QL9J2TgdVar/r2LHqgCfbmWu zINs3OqZ7AO3lHFYCIMDHOY= =7IEu -----END PGP SIGNATURE-----
- -- Adrian Rossouw Drupal developer and Bryght Guy http://drupal.org | http://bryght.com
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (Darwin)
iD8DBQFCZU6/gegMqdGlkasRAmbEAJ9b775ZxHDZN9SLNx4AA6pRDNLG0QCfVIH9 HHfsunWsGlY51H44aZahnh0= =fwEw -----END PGP SIGNATURE-----
On Tuesday 19 April 2005 01:32 pm, Adrian Rossouw wrote:
We would : 1) add a url_alias column to the node table 2) modify node_load to use the url_alias column, if set, otherwise default to node/view/$node->nid, and set the $node->link variable. 3) change all l('node/view/ . $node->nid) to l($node->link);
Then pathauto can just implement the nodeapi hook to set the url_alias to whatever it wants, and the url_alias table will not grow unruly.
The path module currently lets you define many aliases for a single Drupal system URL. And the aliases can be any module path (user, node, search, etc.). So does this solution just copy over the default alias into the node table or are you proposing something more extreme like dropping the url_alias table? We could also benchmark if storing the alias array in the cache table would provide any speed benefits. It seemed to work for the variables table anyway. Matt
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 19 Apr 2005, at 8:51 PM, Matt Westgate wrote:
We could also benchmark if storing the alias array in the cache table would provide any speed benefits. It seemed to work for the variables table anyway.
Imagine having a alias for every single node on Drupal.org. 20 000 records loaded into memory all the time. And caching them will help, but sooner or later it's going to take up obscene amounts of memory. This would be another way of providing url's for nodes, only where they are needed (ie: when linking to the node). A similar addition could be made to user_load to handle profiles. - -- Adrian Rossouw Drupal developer and Bryght Guy http://drupal.org | http://bryght.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (Darwin) iD8DBQFCZVxcgegMqdGlkasRAg1XAKCYGxndX4mgmJYxkvAkh+vm56ByVgCeITrl BdJ3yLxoI9X92gu5WaVQjS0= =FC6j -----END PGP SIGNATURE-----
I just came upon a solution that I think would work well.
We would : 1) add a url_alias column to the node table 2) modify node_load to use the url_alias column, if set, otherwise default to node/view/$node->nid, and set the $node->link variable. 3) change all l('node/view/ . $node->nid) to l($node->link);
Then pathauto can just implement the nodeapi hook to set the url_alias to whatever it wants, and the url_alias table will not grow unruly.
I will be rolling a patch for this soon.
There is no need to make the node view a special case, as all url alias generation can be intercepted with conf_url_rewrite() and that can easily detect a node/$nid URL (although the other node properties are not available at that point). Goba
I've recently found myself incredibly impressed with the autopath module, but I realize that there is an inherent scaleability issue with how we do paths at the moment.
On every page load, the entire url_alias table is loaded into memory, and enabling the autopath module on a site like Drupal.org with thousands upon thousands of nodes, is just impractical. Does anyone have an idea of how we could make the path system easier to use for large sites?
conf_url_rewrite()? Goba
participants (5)
-
Adrian Rossouw -
Gabor Hojtsy -
Gerhard Killesreiter -
Matt Westgate -
Moshe Weitzman