[drupal-support] how does the throttle module works?

[Abalieno]

>> I can still use my modified module by replacing it when the new
>> release will be out or there are other dependences?
>
> Possibly.  It depends on how you've implemented it, and how many
> core modules you're willing to patch.  Any core modules that use the
> throttle module would have to be updated to recognize multiple
> throttle levels

Sigh.. I cannot. I cannot mantain an alternate version since I know zero of 
PHP.

But it would be possible to just modify the throttle.module to add more 
dynamic settings to it? My idea isn't to change every module, but just to 
add fuctions on top of what Drupal will do with the next version.

> Yes, things like this benefit from having multiple throttle levels.
> But it was decided that the configuration and concepts were too
> difficult, thus it was simplified:
> http://lists.drupal.org/archives/drupal-devel/2004-08/msg00486.html

I rather dislike this continuous loss of features to add "simplification". 
Designing a good system doesn't mean removing its potential but just to 
better organize and show it. But this is a personal opinion that is not 
important.

The original patch suggested by Njivy was *wonderful* because it allowed to 
finally customize all the levels but then Dries asked to simply wipe what 
matters to remove directly *all* the strengths of the module. Great work.

You know what could work wonderfully? To still have each of the six levels 
(0-5) and then let the user to type directly the number of pages for each. 
This would allow to use all the five levels or just the first and the last. 
Depending on what the admin chooses.

Anyway, how does the new system work? I tried to follow your links but I 
didn't grasp on what the new switch is based. The current throttle checks 
the hits from time to time depending on how it's set. The link tells that 
the new module (4.6.0) doesn't use anymore the access log, so what does it 
use?

Side-question: why a spider bot is counted as "many different users" and not 
just as the same one requesting different pages? If I'm an anonymous user 
and if I keep refreshing the page I'm still counted as "1" in the "Who's 
online" block. So why Dries wants to base the throttle on the number of 
anonymous users and NOT on the number of pages served? I don't understand 
how it could work, that block doesn't seem consistent. If there's just one 
users that is savagely spamming my site the method Dries proposes won't 
detect that. I also do not understand what he writes in that link, it 
actually sounds like the OPPOSITE should happen. If the throttle checks how 
many pages are served in a minute it is supposed to easily locate the spam, 
while if it is based on the "Who's online" block it will simply ignore an 
emergency.

In the first case the site will resist, in the second case it will be 
swamped.

" The past months drupal.org often got hit by Evil Crawlers. Unlike Good 
Crawlers (i.e. Googlebot) they don't adapt their request rate to the sender, 
and request thousands of pages like there is no tomorrow. Clearly, they can 
put a lot of pressure on the server. "

That's why it *makes sense* to enable the throttle if the spider is putting 
stress on the site. Who cares if it's not a "normal" load to cause the 
switch? I care for the server, I want my site to survive. Why the throttle 
should let the spider do its work unaffected if that damages the server?

>> Something happens, my refreshes are counted till a point, then
>> they go ignored.
>
> This is by design, well, the old design.  Essentially, at throttle
> level 5 the throttle tuned itself to no longer perform database
> queries.  See here:
> http://lists.drupal.org/archives/drupal-devel/2003-12/msg00218.html

I've read that message but so how the throttle know when to switch back at 
level 4? If no more checks are performed why the site doesn't stay at level 
5 forever once it reaches it?

It wouldn't be better to add a setting to the throttle module in order to 
flag the "emergency status" on a time-based period?

This is my proposal:
Let the admin set the number of hits before the throttle switches on, then 
add a field where the admin can choose for how long the emergency module 
will remain active. No matter of usage. When this timeout is over (for 
example after ten minutes) a new check is performed to decide if there are 
the conditions to go back at the "normal" mode.

This would allow to completely remove the overhead because the checks are 
performed only between the timeouts.

Basing the throttle module on the "Who's online" block is NOT CONSISTENT.

> In any case, this was determined to be too complex, and is no longer
> true in 4.6+.

Drupal 4.6.0 is due out next week. I guess the system should be already well 
tested and defined, or not?

> Once a page is cached, performance for anonymous guests should be
> significantly better.  If you're making lots of modifications to the
> core code, this may no longer be true.

Yes, but I don't want by default to always use the cache on my site. This is 
why I asked if it would be useful to dynamically switch it on and off by 
checking the throttle status. That's what I coded myself into the current 
module, I hope it will work:

  if ($throttle_new < $throttle) {
    variable_set('throttle_level', $throttle - 1);
    variable_set('default_nodes_main', $safenet + 5);
    watchdog($type, t('Throttle: %hits hits in past minute; throttle 
decreased to level %level.', array('%hits' => "<em>$hits</em>", '%level' => 
'<em>'. ($throttle - 1) .'</em>')));
    if ($throttle_new == 0) {
               variable_set('default_nodes_main', 25);
               variable_set('cache', 0);
     }
 }

  if ($throttle_new > $throttle) {
    variable_set('throttle_level', $throttle + 1);
    variable_set('default_nodes_main', $safenet - 5);
    watchdog($type, t('Throttle: %hits hits in past minute; throttle 
increased to level %level.', array('%hits' => "<em>$hits</em>", '%level' => 
'<em>'. ($throttle + 1) .'</em>')));
           if ($throttle_new == 5) {
               variable_set('default_nodes_main', 5);
               variable_set('cache', 1);
     }
  }

-HRose / Abalieno