What Are Meta-search Engines?

Unless we are talking about booking a holiday for a fair price or illegally downloading the latest superhero movie via torrents, the term meta-search isn’t likely to come up in a conversation in 2019. 

However, this wasn’t the case in the early days of online search.

What does a meta-search engine do?

We could define a meta-search engine as an advanced search tool that aggregates other engines’ data to compile its own results pages. It provides users with all the answers they could be looking for elsewhere, all in one place.

Metasearch was particularly popular in the early 2000s, but it dates back to the late 90s when engines like Ixquick or Dogpile popped up as alternatives to Google and Yahoo!.

As of 2019, metasearch is pretty much dead and gone.

However, while the heyday of the meta-search engine is long over, the federated search technology continues to be used by travel aggregators and torrent search engines. 

The story of its demise offers insight into today’s market and a valuable lesson about what we, as users, want from search engines.

Find ten differences

Let’s take a look at the comparison between these two search result pages, one from a meta-search engine and one from Google, and see how they fare against each other.

Here’s eTools.ch SERP when I searched for “battle of hastings.” It looks comprehensive, lists the sources used, and the results are even divided by category! If I had the time, I could deeply educate myself in an organized fashion. 

Now, here’s Google’s SERP for the same query:

The first search result is a Wiki page offering crucial bits of information, along with the knowledge graph box on the right. There is also the People also ask box, helping me to specify what I might be looking for. 

Google feeds me what I need right away and then offers more if I’m not full yet. This is how search looks nowadays, and meta-search engines are far behind.

Why didn’t meta-search engines work?

On paper, the technology seems reasonable and useful:

• You get compiled results from several major engines in one place, and you can be sure that you don’t miss out on anything. 
• You get a quick overview of the answers to your query, or dig in and take a closer look at any of them. 
• All of this while feeling secure and private, as many of the popular meta-search engines would hide the user’s IP address from the queried search engines.

However, there are several challenges metasearch systems were bound to address and didn’t manage to overcome fully. 

Implementing a scoring system to provide a meaningful response is a huge problem when including data from several major search engines, which might have a completely different relevance score for the same website.

The response time also comes to mind. The more indices a meta-search engine wants to include, the more likely it is that one of them will slow the search down. Some meta-search engines dealt with the problem by making the results page load dynamically as more data comes in, but the low-quality user experience remained an issue.

The biggest problem, however, is that we don’t care about extensiveness. 

Search engines are like fast-food chains – we, the customers, vote on what we do and don’t like by using their services every single day. Our actions are what drive them to continually adjust or go out of business if they fail to do so.

When we search the web, we don’t want thousands of answers to our query – we want the one answer that we were looking for, and on the first page. When we buy, we want the products to appear right away, ready for purchase. When we research, the information we need has to be in front of us within seconds.

That’s why Google continues to enjoy the lion’s share of the market.

What happened to meta-search engines?

A list of existing relevant meta-search engines would have to be short. The one that remarkably continues to stand against time is Dogpile, soon to be 23 years old. DuckDuckGo, launched in 2008, enjoys a steady top 10 position on the market largely thanks to its emphasis on privacy protection and is expected to continue to grow. Several others, like the previously mentioned eTools.ch, enjoy the status of specialized engines serving netizens with particular needs.

Most, however, either collapsed due to lack of traction or adjusted by changing the nature of their service altogether.

Let’s take the example of one of the oldest meta-search engines — Mamma.com. Now, it provides a business review service instead.

Startpage.com, formerly known as IxQuick, now uses the Google database for search and acts as a proxy to protect the users’ privacy.

Is there a future for meta-search?

Market research suggests that metasearch can no longer compete against the giants search industry. 

Source: https://gs.statcounter.com/search-engine-market-share

However, while Google remains unchallenged with over 90% market share, customers interested in protecting their online privacy made DuckDuckGo grow rapidly in the past twelve months. The company has managed to double its search volume, reaching 36 million searches a day in 2019. DuckDuckGo’s considerable success might encourage others to follow in their footsteps. 

On the other hand, alternative privacy options, such as VPNs, can turn potential customers away from metasearch engines promising greater security.

Wrapping up

Whatever their future, meta-search engines have made a significant contribution to the history of the web. They provided an alternative to the mainstream, and this healthy competition resulted in the overall improvement of the search service.

However, as it often happens in any industry, you either adjust your services to best meet the customers’ needs, or you go down in history as a failed alternative to the ones who did.