One of the fundamentals in SEO is SEO off page, which is based basically on the quantity and quality of the links to your web pages.
Although some of this information comes from Google Webmaster Tools, many SEOs trust tools like aHrefs or Majestic SEO to perform tasks such as links auditing or just checking new links to the page.
However, many websites are blocking commercial tool robots such as aHrefs or Majestic SEO because they are overflooded with notifications to delete links, or just because they are selling links and they do not want them to be detected as low quality sites.
In many occasions this can mean that if we perform an analysis of backlinks we find that there has been a great loss of links, not knowing exactly why. Apparently the link is present on the web page.
These sites are blocking the SEO commercial tools not via the file robots.txt nor the tag meta robots, since this would be very easy to detect, but from the server, which returns an error code.
This makes things a little harder. We could use a tool that allows us to surf with different user-agents to check that, in fact, the website is blocking the access to the SEO tool.
Are there reasons to block robots to my site?
SEO blocking tools are nothing new. For years a lot of websites have done this with the aim of hiding their links networks-including from Google. If we enter data and information on the links, many of the tools can tell us which sites are linked to it. Taking into account that Google has tools to report users for spamming, websites do not want to give clues to the competition, especially if they are crossing the red line.
This has of course a very negative side. If a competitor decides to do negative SEO and redirect many low quality links to another one, the latter will have it very hard to detect them.
What can we do about this?
It is clear that tools for commercial links are crucial in any SEO campaign, since they will provide information a lot quicker and more detailed in comparison with the tool from Google.
However, the most important thing is what appears under Google Webmaster Tools, and in the case that Google carried out a manual or algorythmical action on our site, we should check the tool to revert the situation and delete the ban.
The truth is that there is not much more we can do, and the only option left would be to choose another tool and hope it is not blocked, or use the command “link:” from Google. Unfortunately this has many limitations and in most cases it only shows a very small part of the real backlinks.