How to Block Ahrefs, Moz, and Majestic Crawlers From Your Website
Three reliable methods that work together — robots.txt, server-level rules, and IP blocks — to keep competitors out of your SEO data.
Competitors use Ahrefs, Moz, and Majestic to reverse-engineer your backlink profile, spy on your ranking keywords, and copy your content strategy. If you want to hide your SEO data from competitors, you can block Ahrefs and Moz and Majestic crawlers at multiple levels. This guide shows you three reliable methods that work together.
Why Block SEO Crawlers
These three tools power most competitive SEO research on the web. When AhrefsBot, rogerbot, or MJ12bot crawl your site, they feed your data into public indexes that anyone with a subscription can query. Blocking them delivers three concrete benefits:
- Backlink privacy. Competitors cannot map your link building campaigns or contact your referring domains.
- Keyword protection. Your ranking keywords stay out of public databases like Ahrefs Site Explorer.
- Server resource savings. Aggressive third-party bots consume bandwidth and crawl budget you would rather spend on Googlebot.
The tradeoff is that you also lose your own data inside these tools for your domain. Make sure you accept that limitation before you deploy the blocks.
Method 1: Block Through robots.txt
The fastest way to block Ahrefs and Moz and Majestic is through your robots.txt file. Add these directives to the root of your domain at yourdomain.com/robots.txt:
User-agent: AhrefsBot Disallow: / User-agent: AhrefsSiteAudit Disallow: / User-agent: rogerbot Disallow: / User-agent: dotbot Disallow: / User-agent: MJ12bot Disallow: /
AhrefsBot handles the main Ahrefs index. AhrefsSiteAudit powers their site auditing feature. rogerbot and dotbot both belong to Moz. MJ12bot is the Majestic crawler.
Robots.txt works because all three companies publicly commit to respecting the standard. However, robots.txt is a request, not enforcement. Malicious or renamed bots can ignore it, so you need additional layers for full protection.
Method 2: Block at the Server Level
For guaranteed enforcement, block these crawlers directly in your web server configuration. This method rejects the requests before they reach your application.
Apache (.htaccess)
For Apache servers, add this to your .htaccess file:
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} (AhrefsBot|rogerbot|dotbot|MJ12bot|AhrefsSiteAudit) [NC]
RewriteRule .* - [F,L]
Nginx
For Nginx servers, add this to your server block:
if ($http_user_agent ~* (AhrefsBot|rogerbot|dotbot|MJ12bot|AhrefsSiteAudit)) {
return 403;
}
The server returns a 403 Forbidden response, and the crawler receives no content. This approach works even if the bot ignores robots.txt because the block happens at the HTTP request level.
Method 3: Block by IP Range
Some advanced users prefer to block Ahrefs, Moz, and Majestic by their published IP ranges. Ahrefs publishes their crawler IP list at ahrefs.com/robot. Majestic publishes IPs through their support documentation. Moz confirms rogerbot IPs on request.
IP blocking catches bots that spoof user agents to bypass the previous two methods. Combine IP rules with a firewall like Cloudflare or your server firewall for cleaner management. Cloudflare users can create a WAF rule that matches the known user agents or ASN ranges and issues a block action in seconds.
Verify Your Blocks Work
After you deploy any of these methods, test them with curl. Run this command from a terminal:
curl -A "AhrefsBot" -I https://yourdomain.com
A successful block returns HTTP 403 or an empty response for the disallowed paths. Repeat the test with rogerbot, dotbot, and MJ12bot user agents to confirm each crawler receives the same treatment.
Check your Ahrefs and Moz profiles after 30 to 60 days. Your domain data will gradually disappear from public reports as their indexes refresh.
Which Method to Choose
Use all three layers for maximum protection. Robots.txt handles compliant crawlers instantly. Server rules catch anything that ignores robots.txt. IP blocks defend against user-agent spoofing. Together, they close every gap that lets competitors monitor your SEO performance.
Remember that Googlebot, Bingbot, and other search engine crawlers stay unaffected because none of the blocking rules target them. Your organic rankings continue to work normally while you keep competitors out of your data.