I had a strange problem while importing a new po file in drupal 4.6.4/5. Finally, I found the solution [3] : memory_limit should be higher than 8Mb. The module causing seems to be locale. As it is in drupal core, I think the "System requirements" page [1] should be updated, because the next line isnt really true in my eyes: "php memory of 8MB for a Drupal core install." I know there is another web page [2] telling sometimes drupal needs more PHP memory, but I fear the exception had become the rule. What do you think ? On one side, a 12Mb requirement may scare users, but on the other side, it would help users to avoid problems. Ch. [1] System requirements http://drupal.org/node/270 [2] My admin > modules page is blank http://drupal.org/node/31819 [3] Locale doesn't work for me http://drupal.org/node/12020#comment-18732
Hi Christophe, On 13 Dec 2005, at 12:13, Christophe Chisogne wrote:
I had a strange problem while importing a new po file in drupal 4.6.4/5. Finally, I found the solution [3] : memory_limit should be higher than 8Mb.
instead of increasing the memory limit, I'd like to see us investigate how we can reduce the memory requirements of the locale module's parser. I haven't looked at the code, but using explicit unset()s might be enough to reduce memory usage. Unsetting a variable will free its memory! Not sure memory is freed when a variable is overwritten or when a local variable goes out of scope when method execution returns to its caller. It is certainly worth exploring ... -- Dries Buytaert :: http://www.buytaert.net/
On 12/13/05 7:18 AM, Dries Buytaert wrote:
Hi Christophe,
On 13 Dec 2005, at 12:13, Christophe Chisogne wrote:
I had a strange problem while importing a new po file in drupal 4.6.4/5. Finally, I found the solution [3] : memory_limit should be higher than 8Mb.
instead of increasing the memory limit, I'd like to see us investigate how we can reduce the memory requirements of the locale module's parser.
while I'm all for optimizations... and Drupal's RAM usage could definitely use some optimizations, the fact is RAM is fairly cheap, and PHP's default 8M is a bit dated. Most larger scale projects these days recommend 16M (in fact I heard rumours that the PHP team is considering up'ing the default).
I haven't looked at the code, but using explicit unset()s might be enough to reduce memory usage. Unsetting a variable will free its memory! Not sure memory is freed when a variable is overwritten or when a local variable goes out of scope when method execution returns to its caller. It is certainly worth exploring ...
I *believe* that only unset will free the memory (until the end of execution). i may well be wrong - so yeah, investigation is in order :) -- James Walker :: http://walkah.net/ :: xmpp:walkah@walkah.net
On 13 Dec 2005, at 12:13, Christophe Chisogne wrote:
I had a strange problem while importing a new po file in drupal 4.6.4/5. Finally, I found the solution [3] : memory_limit should be higher than 8Mb.
instead of increasing the memory limit, I'd like to see us investigate how we can reduce the memory requirements of the locale module's parser.
while I'm all for optimizations... and Drupal's RAM usage could definitely use some optimizations, the fact is RAM is fairly cheap, and PHP's default 8M is a bit dated. Most larger scale projects these days recommend 16M (in fact I heard rumours that the PHP team is considering up'ing the default).
You are technically right. But in the real world, we will be shutting out many users who use shared hosting and have no control on how much memory they can allocate to PHP. I do not like to see more people deprived of using Drupal because of these constraints. We are much better off tuning and optimizing locale, going to split mode, ...etc.
On 12/13/05 10:10 AM, Khalid B wrote:
On 13 Dec 2005, at 12:13, Christophe Chisogne wrote:
I had a strange problem while importing a new po file in drupal 4.6.4/5. Finally, I found the solution [3] : memory_limit should be higher than 8Mb. instead of increasing the memory limit, I'd like to see us investigate how we can reduce the memory requirements of the locale module's parser. while I'm all for optimizations... and Drupal's RAM usage could definitely use some optimizations, the fact is RAM is fairly cheap, and PHP's default 8M is a bit dated. Most larger scale projects these days recommend 16M (in fact I heard rumours that the PHP team is considering up'ing the default).
You are technically right. But in the real world, we will be shutting out many users who use shared hosting and have no control on how much memory they can allocate to PHP.
I do not like to see more people deprived of using Drupal because of these constraints.
We are much better off tuning and optimizing locale, going to split mode, ...etc.
Absolutely. Sites of all sizes win with memory usage optimizations... I'm all for it. While I still maintain split mode is a bit too extreme (and presents some complications beyond it's quite impressive RAM usage numbers) - I think it hints towards what's important here. Drupal loads too much code into RAM per-request. There are lots of ways to cut this down.. (long help text removed .module files, libraries / helper functions moved to .incs, etc - locale has already done some of this, but clearly has work still to do). I was merely pointing out that hitting PHP's 8M limit is not at all uncommon, and not the end of the world. -- James Walker :: http://walkah.net/ :: xmpp:walkah@walkah.net
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Tue, 13 Dec 2005 10:06:13 -0500 James Walker <walkah@walkah.net> wrote:
On 12/13/05 7:18 AM, Dries Buytaert wrote:
Hi Christophe,
On 13 Dec 2005, at 12:13, Christophe Chisogne wrote:
I had a strange problem while importing a new po file in drupal 4.6.4/5. Finally, I found the solution [3] : memory_limit should be higher than 8Mb.
instead of increasing the memory limit, I'd like to see us investigate how we can reduce the memory requirements of the locale module's parser.
while I'm all for optimizations... and Drupal's RAM usage could definitely use some optimizations, the fact is RAM is fairly cheap, and PHP's default 8M is a bit dated. Most larger scale projects these days recommend 16M (in fact I heard rumours that the PHP team is considering up'ing the default).
ram isn't cheap on a VPS hosting setup, and African standards of cheap are very different from say western standards. in fact I've been having a very hardtime keep a mildly popular drupal website online due to memory usage and would greatly appreciate reducing it. cheers, Alaa - -- http://www.manalaa.net "context is over-rated. who are you anyway?" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2 (GNU/Linux) iD8DBQFDn/6JvQ4s8A97qKQRAjt2AKCc4kEOAJTSp6Sv7LcBB+sMy8Fe+wCfX879 2IWciAvpObsvUj33zihGthM= =fhtP -----END PGP SIGNATURE-----
Dries Buytaert wrote:
Hi Christophe,
On 13 Dec 2005, at 12:13, Christophe Chisogne wrote:
I had a strange problem while importing a new po file in drupal 4.6.4/5. Finally, I found the solution [3] : memory_limit should be higher than 8Mb.
instead of increasing the memory limit, I'd like to see us investigate how we can reduce the memory requirements of the locale module's parser.
I haven't looked at the code, but using explicit unset()s might be enough to reduce memory usage. Unsetting a variable will free its memory!
Won't help in this case. The whole PO file is parsed into a huge array and then postprocessed to upload the single translations. Cheers, Gerhard, with enough memory.
Won't help in this case. The whole PO file is parsed into a huge array and then postprocessed to upload the single translations.
And that one array is 5 MB or so? The parsing routine, _locale_import_read_po(), has only one call site so eliminating that huge array, and making the parser store the strings in the database, might be a viable option. Just a thought. -- Dries Buytaert :: http://www.buytaert.net/
Dries Buytaert wrote:
Won't help in this case. The whole PO file is parsed into a huge array and then postprocessed to upload the single translations.
And that one array is 5 MB or so? The parsing routine, _locale_import_read_po(), has only one call site so eliminating that huge array, and making the parser store the strings in the database, might be a viable option. Just a thought.
As I hinted I am in the process of preparing to upgrade Drupal.hu to 4.7 (from 4.5), and I especially planned to take the time and do some performance improvements on the PO importing code. If anyone beats me to it, then it is even better, but I am surely going to tackle it before the release. (First in my sight is the update process, which has some flows when upgrading from 4.5) Goba
participants (8)
-
Alaa Abd El Fattah -
Christophe Chisogne -
Dries Buytaert -
Dries Buytaert -
Gabor Hojtsy -
Gerhard Killesreiter -
James Walker -
Khalid B