How to blacklist visitors
Sometimes, you want to ban misbehaving visitors, which is different from denying them access to a specific page. For example, suppose there is a robot visiting your site, trying to download restricted content (e.g. content that you have excluded via robots.txt). In addition to denying the bot access to those pages, you might want to deny the bot access to other pages that are not excluded via robots.txt -- we call that banning an IP address. There are two steps to bans: creating a blacklist and then adding an IP address to the blacklist.
In this tutorial, we'll cover how to create a blacklist policy that will automatically deny all blacklisted visitors.
First, create a visitor group for blacklisted IP addresses. Go to the visitor groups page and click the "New visitor group" button. We can add in the IP addresses we want to blacklist later.
Next, create a policy to deny blacklisted visitors. Go to the policies page and click the "New policy" button.
The amount of time is not important, but the number of visits should always be set to 1. This ensures that this policy will trigger immediately when a blacklisted visitor visits your site.
From here, add offending IP addresses to the blacklisted IP addresses visitor group to deny access. You can add visitors through the web interface or web API.
curl "https://api.nettoolkit.com/v1/gatekeeper/visitor-groups/$ID/visitors" \ -X POST \ -H "X-NTK-KEY: $YOUR_API_KEY" \ -d "visitor=126.96.36.199"
If you don't like visitors coming through Tor or as identified by FireHOL's abuse list, you can add those groups to this blacklisted visitor group.
The blacklist is a simple, but effective pattern for dealing with the worst of your visitors. For the next step in banning an IP address, see how to automatically ban misbehaving visitors.