Are you in full control of your website? Find out how Verizon – the 16th biggest company in the US – is paying the price for not optimizing its website with over 55% of its product pages not indexed by Google.
Have you ever used an SEO crawler?
These days, they offer incredible functionality, providing you with both an overview of your website’s structure, as well as a closer look into potential problems that may damage the business.
One of the particularly useful features to give you an eagle’s view of your website architecture is the crawl map.
Looking at this graph, we can see how various parts of the domain are interconnected, how many clicks away they are from the homepage, or how many links the crawler found within a particular section.
It’s particularly useful to inspect the crawl map when you are trying to optimize the website’s structure towards search engine crawlers. When you have a large website and you want it to be regularly visited and indexed by Google, it’s important to pay particular attention to and optimize internal linking, so that new pages are easy to find, and the most important pages that you want to rank well get a strong internal signal boost.
Take a closer look at the two crawl maps that I created using Sitebulb (thanks, Sitebulb!), using the crawl data from two websites of similar size. The node colors represent the number of clicks a given URL is distanced from the homepage, going from green to purple. Light green represents one click, orange is three, blue is six, and so on.
Which website’s architecture would you say is better? The second one is the clear winner, right? Most of the dots are green, yellow, or orange, so the majority of URLs are only a couple clicks away from the homepage – Googlebot will have no problem finding those.
The first map, on the other hand, contains many blue, and even purple dots. Search engine crawlers will have a hard time reaching those links, although they will find them eventually, too. What’s more important is that cross-linking between pages on that website is really weak, and that means that many pages won’t rank as well as they could if they got some more link equity sent their way.
Now, the biggest problem here is, this is the same website. Crawled with and without rendering JavaScript.
The website I crawled to create these graphs belongs to Verizon, the largest wireless network provider in the US with over 35% market share in 2018.
In Verizon’s case, the difference between an HTML crawl and a JavaScript crawl is staggering.
Here’s how the website is crawled when JavaScript isn’t rendered:
And here’s how the data looks when JavaScript is rendered by the crawler:
Can you see how much it’s moved to the left, towards the homepage? Remember, the farther a page is from the homepage, the harder it is for the search engines to find. And if the search engines can’t find it, they won’t have anything to show to the users.
You might think that this doesn’t mean that the website is necessarily underperforming in search. After all, Verizon is doing well, and they are a tech company, so they should know how to build a website that’s optimized for the search engines.
However, using our tools, we’ve checked the indexing status of a random sample of URLs found in the sitemap. It turns out that over 55% of Verizon’s product pages are not indexed by Google.
To give you some perspective, Verizon gets about 25% of their web traffic from search – that’s about 15 million monthly visits, according to the traffic data provided by SEMrush. Can you imagine how much more traffic they would gain if they had all of their product pages indexed?
Are you missing out on traffic too?
It doesn’t have to be like this. There is a clear correlation between how Verizon is using JavaScript on their website and the problems they have getting indexed.
Solving such problems is what we do at Onely — contact us for technical SEO services or read our ultimate guide to JavaScript SEO.
