The file robots.txt is included under the root level of the web and its function is to tell search engines the steps to follow when crawling, enabling or disabling access to some parts of the web. But can we use them for something else?
Some pages use the file robots to include funny or promotional messages, or event to find talented SEO.
Here we leave you a compilation of the funniest robots.txt files we have found.
Finding SEO talent in robots.txt:
Some sites know how important the robots file is for SEO. Therefore, they use this file to find the best SEO candidates that have voluntarily found or checked this file within their pages.
One such example of this kind of recruitment is what TripAdvisor‘s robots.txt does:
Another website that uses this technique is the SEO agency Señor Muñoz:
“Si estás leyendo esto es por algo. ¿Por qué no me lo cuentas?” (If you are reading this, why don’t you tell me?)
You can read other funny messages like this if you access the file http://www.senormunoz.es/robots.txt (in Spanish).
Advertising on robots.txt:
We can also find advertising messages or references to the brand under the robots file. This is the case of www.vodafone.es, which uses its file to display an advertisement:
Nike also includes a slogan referring to its brand taking their renowned “Just do it.”
Blocking funny information on robots.txt:
One of the best known robots.txt is that of the website Casa Real española, which after the accusations against Iñaki Urdangarin used the file to block everything related to him in the website, which was very commented in the SEO industry. Up to the day this content remains blocked. You can check it out at http://www.casareal.es/robots.txt
The Spanish copyright association SGAE also blocked content they did not like. In this case they decided to block everything that related them with the keywords thieves, mafia or monopoly.
Important files on robots.txt:
A very well-known practice is to block those files that are important for a web page but without changing their striking name. This is what happens with pages such as accessdnsi.com, where you can see that the file /passwords.txt is blocked. But you better go there and have a look.
Another such a mistake comes from a Chinese web page that sells subtitles for movies and series that charges around 25 euros per subtitle. But they forgot to change the name of the files blocked in the robots.
Other web pages use this file to include odd things and get the attention of the users within their robots.txt files. This is the case of web pages such as www.last.fm/robots.txt or www.minijuegos.com/robots.txt that use their robots to refer to the three laws in robotics by Isaac Asimov.
People also comment that before launching a product on the market, Google blocks it on the robots file. In fact, many fans of Google check Google.com/robots.txt regularly to find new launches.
Wikipedia also includes under its robots file funny messages for users. They complain that everybody copies them.
The robots file of the web page by our SEO colleague Lino Uruñuela, uses his website mecagoenlos.com to show funny comments and a little robot drawn in the file.
Another funny file is that from robots.txt itself, since it is redundant http://www.robotstxt.org/robots.txt
The BBC also uses the robots file to block news that they want to share but that they do not want to index, for example www.bbc.co.uk/news/uk-england-suffolk-20780472
Last of all, we will talk about an initiative from some Spanish people who created a txt file for humans where they advise creating the file humans.txt where to include the name of the actual “humans” that are behind the creation of the different webs. The idea is to present the people behind a web. In fact, Google itself included this file under its main domain where it indicates that there are so many people working at Google that it is impossible for them to include the names of all their employees. This is what they say:
“Google is built by a large team of engineers, designers, researchers, robots, and others in many different sites across the globe. It is updated continuously, and built with more tools and technologies than we can shake a stick at. If you’d like to help us out, see google.com/careers.”
If you know about more funny robots.txt, leave a comment with the link and let your imagination run free to create an original and funny robots file in your own website.
You may also like:
Latest posts by Internet República (see all)
- YouTube SEO - 26 November, 2018
- Introduction to search engines - 20 December, 2017
- 5 reasons why you should create a blog for your website - 15 December, 2017