How to implement rate limiting
A rate limit is essentially a cap on how quickly a visitor can load content on your site. This can be an effective measure against dumb scrapers that try to load as many pages as possible, as quickly as possible.
In this tutorial, we'll see how to implement a policy that stops visitors from visiting pages too quickly and enforces a cooldown before they're allowed access again.
The setup for this is quite simple. Since our rate limit will apply to all users across all pages, we don't need to create a visitor group or page group.
Go to the policies page and click the "New policy" button. Our rate limiting configuration will consist of a single policy.
Keep in mind that the time window for our rate limit also serves as the cooldown time for offenders.
Consider the following scenario:
A robot visits 100 times in 10 minutes. On the 100th visit, they are denied by our "rate limit" policy.
Since the "rate limit" policy checks in 24 hour increments, this visitor will not be allowed access again until their total visit count for the past 24 hours is less than 100 -- roughly 23 hours and 50 minutes in this case.
If you'd like to deny access for a longer period of time, see how to automatically ban a visitor with an expiration time.