Robots.txt Easter Eggs – Make Your Robots Directives Fun For Humans!

Robots.txt Easter Eggs - Hero Image

Easter eggs are generally associated with items you find during a hunt, after spending time looking for them in unknown places, without clear guidance on where to look. Because the search requires some time and effort, it’s rewarding to find them ultimately. 

All of this is also true for easter eggs found in tech products. 

Today, I want to look at easter eggs in robots.txt files. I will explain the concept of easter eggs and robots.txt files and why developers may include easter eggs in them. 

I also went on my own easter egg hunt to determine how many forms easter eggs in robots.txt files can take. As it turns out – many! 

Read on and be sure to get to the end of this article – I will have a task for you!

What are easter eggs in tech?

Easter eggs in tech are understood as undocumented features or pieces of software that are not necessary for the product to work. These features are not openly revealed or shared with the users who need to search for them without much guidance.

Easter eggs are commonly found in video games, programs, apps, software, and parts of websites. 

They come in many forms, such as: 

  • cheat codes in a game that uncover additional features or levels, 
  • cultural references appearing on hidden pages or other web documents,
  • games or apps located in software made for other purposes, e.g., writing and editing text documents, 
  • messages left behind by developers, e.g., ones that credit the creators of the product. 

The list goes on, and there are no limits to what easter eggs can be. 

But there is more to it.

For me, easter eggs are a human touch in a machine environment characterized by abundant logic and concrete rules. 

Easter eggs break out of these strict rules. They are a way for creators to add a unique element with a human face to their technological product. Including easter eggs makes the whole experience of using a given tech product more fun and could be a treat for die-hard fans, depending on the product. 

Let’s move on to easter eggs we can find in sites’ robots.txt files. 

Easter eggs and robots.txt files

Finding easter eggs in robots.txt files is less complicated than in many other cases – you need to simply access a site’s robots.txt. 

There are numerous examples of easter eggs in robots.txt files. These files are meant to be read and followed by robots; hence easter eggs should be added in a way that is understandable only to humans. 

Let’s clarify what robots.txt files are and if adding easter eggs won’t impact how robots read the file.

What are robots.txt files?

A robots.txt file lives at the root of your domain and contains a set of directives for search engine crawlers, telling them how to crawl pages on a website. 

Each directive blocks or allows access for a specific crawler to a file path on a site. Unless specified otherwise in robots.txt, robots can crawl all files on a website.

Directives in a robots.txt file are a way to maximize your crawl budget, so Googlebot does not waste it on irrelevant pages on your site.

Can adding easter eggs hurt your robots.txt file?

Robots.txt files follow a specific format which you can read more about in Google’s guidelines on adding robots.txt rules.

Search engines ignore any symbols or commands that appear in the robots.txt file after a hashtag, so including easter eggs preceded by a hashtag will not affect how robots understand these files. 

Easter eggs also tend to be placed in documents or locations that cannot be accessed by users easily, and that is true about robots.txt files. They are generally not linked to from other pages, and users can’t navigate to them by simply clicking on a button on a website. 

Why should you add an easter egg to your robots.txt file?

Since you can add text to robots.txt files that can be readable only for humans, why not use this opportunity to create exclusive content to supplement your marketing endeavors?

You can add easter eggs to your robots.txt file for many reasons beyond providing entertainment to users.

For example, the contents of your robots.txt can also: 

  • Increase your brand awareness 
  • Showcase your employer branding

This can be achieved in several ways. You can:

  • Implement your own branding, e.g., include your logo or slogan.
  • Refer to the services or products you offer or to your company as a work environment. 
  • Encourage visitors to apply to your job offers – this way, you target only a selected group of people who visit your robots.txt. You can do it especially to invite geeks to join your dev or SEO team!
  • Create relevant and original content. It can thematically match the purpose of the robots.txt file, e.g., address the robots, include humorous references to them, search engines, or other topics, like pop culture. Including lighthearted content like that can make your brand seem more approachable and user-oriented.

Choose an approach that will show your brand in the best light and be a unique asset to how users perceive your brand.

Examples of easter eggs in robots.txt 

Let’s look at examples of easter eggs in robots.txt files and see the various types of messages we can find in them.

Google’s killer-robots.txt

Google’s killer-robots.txt file was a memorable easter egg created for the 20th anniversary of the robots.txt directive being available to webmasters. It has since been removed but remains one of the most famous examples of easter eggs in robots.txt files.

Here is the text that Google included:

User-Agent: T-1000

User-Agent: T-800

Disallow: /+LarryPage

Disallow: /+SergeyBrin

T-800 is the Terminator portrayed by Arnold Schwarzenegger in the famous movie. T-1000 is the robot assassin from the movie Terminator 2. The directives included in this file prevent T-800 and T-1000 from killing Google founders Larry Page and Sergey Brin.

Nike

Nike incorporates their branding into the robots.txt file by including a humorous take on its brand slogan. It’s combined with a reference to search engines:

A fragment of Nike.com's robots.txt file

Nike takes these brand features even further by incorporating their logo:

Nike's logo is included in the robots.txt file at Nike.com

Cloudflare

Cloudflare addresses the robots personally. It begins like this:

A robots.txt easter egg at cloudflare.com

And ends with the robot responding to the request and agreeing to follow it:

A robot easter egg in the robots.txt file at cloudflare.com

Youtube

Youtube references robots and refers to a dystopian future where they are dominant:

Youtube has an easter egg related to robots' domination in the future

Glassdoor

Glassdoor is a website that lets employees review companies, view salaries they offer, search for jobs on the platform, and much more. Their robots.txt file follows the job theme. The directives in the file encourage SEO geeks who venture into their robots.txt file to apply for one of their positions:

Glassdoor has an easter egg in robots.txt that contains an encouragement to visit their careers page

Etsy

Etsy’s message appears to be Italian for “for goodness’ sake!” and includes a simple image shaped like … a robot, of course:

Etsy has a robots.txt easter egg that says "Per dindirindina", which translates to "My goodness!" and a robot

Yelp

Yelp quotes a famous science fiction author, Isaac Asimov, and his Three Laws of Robotics. Asimov introduced these laws in a 1942 short story and later used them as a basis for his novels. 

The laws are an ethical system for humans and robots, and a robots.txt file could be seen as precisely that – a way for humans and robots to interact and mutually benefit from:

Yelp has a robots.txt easter egg that quotes Isaac Asimov's human-robot Laws

Tripadvisor

Similarly to Glassdoor, Tripadvisor values people’s curiosity and asks the easter egg hunters to apply and become members of their SEO team:

TripAdvisor has an easter eggs in robots.txt that is a job offer for SEOs

Wikipedia

Wikipedia’s robots.txt file contains several messages – here are just some of them:

Wikipedia has several robots.txt easter eggs
Wikipedia has several robots.txt easter eggs
Wikipedia has several robots.txt easter eggs

These directives comment on the robots’ behavior – since only humans can read them, it’s a way to joke about robots. It shows the fun, lighthearted side of Wikipedia, which comes in contrast to what we expect from it – serious, fact-checked articles. 

Merriam Webster

Merriam Webster reminds anyone who has access to the file to “edit with care”:

Merriam Webster has a robots.txt easter egg that reminds developers to be extra careful when modifying robots.txt files

This is a double reminder that any directives included in robots.txt files should be carefully planned and placed in the file – after all, mistakes can prevent robots from crawling the site correctly.

Screaming Frog 

Screaming Frog included humorous directives relating to the company’s name: 

Screaming Frog has a robots.txt easter egg that plays on the company's name

But there is more to it than just being a funny reference. Including these directives helps build brand awareness. Mentioning the “protection of frog team” could be viewed as an example of employer branding.

Further in the robots.txt file, we can find an invitation to join the company’s SEO team:

Screaming Frog has a robots.txt easter egg with the logo of the company

Search Engine Land

Search Engine Land addresses people who can read the following message – because robots can’t – recognizing that there can be non-robots that visit their robots.txt file. There is also a reference to Battlestar Galactica:

Search Engine Land has a robots.txt easter egg with a Battlestar Galactica reference

Humans.txt

The Internet should be about humans, not just robots. 

If there is a robots.txt file for robots, there should be one for humans as well.

This is the thought behind humans.txt, which was created as a counter-initiative to robots.txt files. As we read on its official website:

“It’s an initiative for knowing the people behind a website. It’s a TXT file that contains information about the different people who have contributed to building the website.”

A .txt file is not intrusive with the code, and it’s an easy way to prove the authorship of a site. 

Here is the text that Google’s humans.txt file contains:

“Google is built by a large team of engineers, designers, researchers, robots, and others in many different sites across the globe. It is updated continuously and built with more tools and technologies than we can shake a stick at. If you’d like to help us out, see careers.google.com.”

Other websites use it too and often get more creative – look at Netflix’s and Medium’s humans.txt files.

Next steps

Now that you’ve seen examples of easter eggs found in robots.txt files, do you feel inspired to leave an easter egg for your users?

We did!

Hi! I’m Bartosz, founder and Head of SEO @ Onely. Thank you for trusting us with your valuable time and I hope that you found the answers to your questions in this blogpost.

In case you are still wondering how to exactly move forward with fixing your website Technical SEO – check out our services page and schedule a free discovery call where we will do all the heavylifting for you.

Hope to talk to you soon!