Index Coverage (Page Indexing) is a report in Google Search Console that shows the crawling and indexing status of all URLs that Google has discovered for your website.
It helps you track your website’s indexing status and keeps you informed about technical issues preventing your pages from being crawled and indexed correctly.
Checking the Index Coverage (Page Indexing) report regularly will help you spot and understand issues and learn how to address them.
In this article, I will describe:
- What the Index Coverage (Page Indexing) report is,
- When and how you should use it,
- The statuses shown in the report, including types of issues, what they mean, and how to fix them.
Google’s indexing pipeline
Before digging into the report’s specifics, let’s discuss the steps Google needs to take to index and, eventually rank web pages.
For a page to be ranked and shown to users, it needs to be discovered, crawled, and indexed.
Google needs to first discover a page to be able to crawl it.
Discovery can happen in a few ways.
Crawling consists of search engines exploring web pages and analyzing their content.
An essential aspect of crawling is the crawl budget, which is the amount of time and resources that search engines can and want to spend on crawling your site. Search engines have limited crawling capabilities and can only crawl a portion of pages on a website. Read more about crawl budget optimization.
During indexing, Google evaluates the pages and adds them to the index – a database of all web pages that Google can use to generate search results. This stage also consists of rendering, which helps Google see the pages’ layout and content. The information Google gathers about a page helps it decide how to show it in search results.
But, just because Google can find and crawl your page, doesn’t mean it will be indexed.
Getting indexed by Google has been getting more complicated. This is mainly because the web is growing, and websites are becoming heavier.
But here is the crucial indexing aspect to remember: you shouldn’t have all of your pages indexed.
Instead, ensure the index only contains your pages with high-quality content valuable to users. Some pages can have low-quality or duplicate content and, if search engines see them, it may negatively affect how they view your site as a whole.
That’s why it’s vital to create an indexing strategy and decide which pages should and shouldn’t be indexed. By preparing an indexing strategy, you can optimize your crawl budget, follow a clear indexing goal and fix any issues accordingly.
If you want to learn more about indexing, start by exploring our guide to Indexing SEO.
Pages that are indexed can be ranked and appear in search results for relevant queries.
Google decides how to rank pages based on numerous ranking factors, such as the amount and quality of links, page speed, mobile-friendliness, content relevance, and many others.
How to use the Index Coverage (Page indexing) report?
To get to the Index Coverage (Page indexing) report, log in to your Google Search Console account. Then, in the menu on the left, select “Pages” in the Index section:
You will then see the report. By ticking both statuses, you can choose what you want to visualize on the chart:
“All known pages”, “All submitted pages” vs. “Unsubmitted pages only”
In the upper left corner, you can select whether you want to view:
- “All known pages”, which is the default option, showing URLs that Google discovered by any means,
- “All submitted pages”, including only URLs submitted in a sitemap, or
- “Unsubmitted pages only,” including only URLs not present in the sitemap that Google discovered through links.
You should find a stark difference between the status of “All submitted pages” and “All known pages” – “All known pages” normally contain more URLs and more of them are reported as Not indexed. That’s because sitemaps should only contain indexable URLs while most websites contain many pages that shouldn’t be indexed. One example is URLs with tracking parameters on eCommerce websites. Search engine bots like Googlebot may find those pages by various means, but they should not find them in your sitemap.
So always be mindful when opening the Index Coverage (Page indexing) report and make sure you’re looking at the data you’re interested in.
Inspecting the URL statuses
To browse the URLs that are indexed within your website, go to the View data about indexed pages section, just below the chart.
Here you can see the timeline of how the number of your indexed pages changed over time on a sorted chart.
Below the chart, you can explore the list of your indexed pages. But remember that you may not see all of them as:
- The report shows up to 1,000 URLs, and
- New URL may have been added after the last crawl.
To receive more information, you can inspect each URL by choosing the URL from the list and clicking Inspect URL on the right panel.
Not indexed pages
To see the details on the issues found as Not indexed, look below the chart in the Page indexing report:
This section displays the reason behind a given status, the source of it (whether your website or Google causes the issue), and the number of affected pages.
You can also see the validation status – after fixing an issue, you can inform Google that it has been addressed and ask to validate the fix.
This is possible at the top of the report after clicking on the issue:
The validation status can appear as “fixed”. But it can also show “failed” or “not started” – you should prioritize fixing issues that respond with these statuses.
You can also see the trend for each status – whether the number of URLs has been rising, dropping, or staying on the same level.
After clicking on one of the types, you will see which URLs respond with this issue. In addition, you can check when each URL was last crawled – however, this information is not always up-to-date due to possible delays in Google’s reporting.
There is also a chart showing the dates and how the issue changed over time.
Here are some important considerations you should be aware of when using the report:
- Always check if you’re looking at all submitted pages or all known pages. The difference between the status of the pages in your sitemap vs all pages that Google discovered can be very stark.
- The report may show changes with a delay, so whenever you release new content, give it at least a few days to get crawled and indexed.
- Google will send you email notifications about any particularly pressing issues encountered on your site.
- Your aim should be to index the canonical versions of the pages you want users and bots to find.
- As your website grows and you create more content, expect the number of indexed pages in the report to increase.
How often should you check the report?
You should check the Index Coverage report regularly to catch any mistakes in crawling and indexing your pages. Generally, try to check the report at least once a month.
But, if you make any significant changes to your site, like adjusting the layout, URL structure, or conducting a site migration, monitor the results more often to spot any negative impact. Then, I recommend visiting the report at least once a week and paying particular attention to the Not indexed status.
URL Inspection tool
Before diving into the specifics of each status in the Index Coverage (Page indexing) report, I want to mention one other tool in the Search Console that will give you valuable insight into your crawled or indexed pages.
URL inspection tool provides details regarding if:
- The page is indexed,
- The page is indexed but has issues (e.g., struggles with structured data-related problems), or
- The page isn’t indexed.
You can find it in Google Search Console in a search bar at the top of the page.
Simply paste a URL that you want to inspect – you will then see the following data:
You can use the URL inspection tool to:
- Check the index status of a URL and, in case of issues, see what they are and troubleshoot them,
- Learn if a URL is indexable,
- View the rendered version of a URL,
- Request indexing of a URL – e.g., if a page has changed,
- See what enhancements a URL is eligible for – e.g., based on the implementation of structured data and whether the page is mobile-friendly.
If you encounter any issues in the Index Coverage (Page indexing) report, use the URL inspection tool to verify them and test the URLs to better understand what should be fixed.
Here’s what you can do now:
- Contact us.
- Receive a personalized plan from us to deal with your indexing issues.
- Enjoy your content in Google’s index!
Still unsure of dropping us a line? Read how technical SEO services can help you improve your website.
Not indexed status in Index Coverage (Page indexing) report and types of issues
It’s time to look at the Not indexed status in the report and:
- Discuss the specific issue types that it can show,
- What causes these issues, and
- How you should address them.
You may find that many URLs in this section have been excluded for the right reasons. But it’s important to regularly check which URLs are not indexed and why to ensure your critical URLs are not kept out of the index.
Excluded by ‘noindex’ tag
Googlebot found a page and could not index it because of a noindex tag or header in the HTTP response. It’s worth routinely going through these URLs to ensure the right ones are blocked from the index.
Blocked by page removal tool
These URLs have been blocked from Google using Google’s Removals tool. However, this method works only temporarily, and, typically after 90 days, Google may show them in search results again. If you want to block a page permanently, you can remove or redirect it or use a noindex tag.
Server error (5xx)
As indicated by the name, it refers to server errors with 5xx status codes, such as 502 Bad Gateway or 503 Service Unavailable.
You should monitor this section regularly, as Google will have trouble indexing pages with server errors. You may need to contact your server administrator to fix these errors or check if they are caused by any recent upgrades or changes on your site.
Check out Google’s suggestions on how to fix server errors.
Page with redirect
Redirects transfer search engine bots and users from an old URL to a new one. They are usually implemented when old URLs change, or their content doesn’t exist anymore.
If a redirect is detected, the URL is marked with the “Page with redirect” status.
Blocked by robots.txt
Robots.txt is a file containing instructions on how robots should crawl your site. If this URL should be indexed, Google needs to crawl it first, so you should go through URLs blocked by robots.txt and check if you intended to block them.
Remember that using robots.txt directives is not a bulletproof way to prevent indexing pages. Google may still index a page without visiting it, e.g., if other pages link to it.
The 401 Unauthorized status code means that a request cannot be completed because it’s necessary to log in with a valid user ID and password. Googlebot cannot index pages hidden behind logins – this tends to occur in staging environments. In this case, either remove the authorization requirement or verify Googlebot so it can access the pages.
If these URLs shouldn’t be indexed, this status is fine. However, to keep these URLs out of Google’s reach, ensure your staging environment cannot be found by Google. For example, remove any existing internal or external links pointing to it.
Crawled – currently not indexed
Googlebot has crawled a URL but is waiting to decide whether it should be indexed.
If you want to learn about what could be causing this status and how to address any issues, be sure to read our article on how to fix “Crawled – currently not indexed”.
Discovered – currently not indexed
This means that Google has found a URL – for example, in a sitemap – but hasn’t crawled it yet.
Keep in mind that in some cases, it could simply mean that Google will crawl it soon. This issue can also be connected with crawl budget problems when Google may view your website as low quality.
If you want to learn more about this status ‒ read our article on how to fix “Discovered – currently not indexed”.
Alternate page with proper canonical tag
This URL is a duplicate of a canonical page marked by the accepted tag. Canonical tags are used to specify a URL that represents the primary version of a page.
In most cases, this status doesn’t need to be fixed. However, if you want to make sure your canonical tags are correct, you should check our guide on how to fix “Alternate page with proper canonical tag” in Google Search Console.
Duplicate without user-selected canonical
There are duplicates for this page, and no canonical version is specified. It means that Google doesn’t view the specified URLs as canonical.
You can use the URL inspection tool to learn which URL Google chose as canonical. For more tips, check our article about the “Duplicate without user-selected canonical” status.
Duplicate, Google chose different canonical than user
You chose a canonical page, but Google selected a different page as canonical.
The page you want to have as canonical may not be as strongly linked internally as a non-canonical page, which Google may then choose as the canonical version.
If you want to learn more about possible causes and solutions for the status, read our guide on how to fix the Duplicate, Google chose different canonical than user issue.
Not found (404)
404 error pages indicate that the requested page could not be found because it changed or was deleted. Error pages exist on every website and, generally, a few of them won’t harm your site. But, whenever a user encounters an error page, it may lead to a negative experience.
If you see this issue in the report, go through the affected URLs and check if you can fix the “Not found (404)” errors.
Page with redirect
Pages with the “Page with redirect” status are redirecting, so they haven’t been indexed. Pages here would generally not require your attention.
For permanently redirecting a page, be sure you implemented a 301 redirect to the closest alternative page. Redirecting 404 pages to the homepage can result in Google treating them as soft 404s.
Soft 404 issue means a page returns a 200 OK status, but its contents make it look like an error, e.g. because it’s empty or contains thin content. Or, it may be custom 404 pages containing user-friendly content directing to other pages, but still returning a 200 OK HTTP code.
To fix soft 404 errors, you can:
- Add or improve the content on these URLs,
- 301 redirect them to the closest matching alternatives, or
- Configure your server to return proper 404 or 410 codes.
Also, as a follow-up, read our article on what are soft 404s in SEO.
Blocked due to access forbidden (403)
The 403 Forbidden status code means the server understands the request but refuses to authorize it. You can either grant access to anonymous visitors so Googlebot can access the URL or, if this is not possible, remove the URL from sitemaps. And if Google shouldn’t access these URLs, it’s better to use a noindex tag.
Blocked due to other 4xx issue
Your URLs may not be indexed due to 4xx issues not specified in other error types. 4xx status codes errors generally refer to problems caused by the client ‒ check these pages to learn what the error is.
You can learn more about what is causing each problem by using the URL Inspection tool. Fix the problems according to the specific code that appears. If you cannot resolve the error, remove the URL from your sitemap.
To learn more about this status, read our article on how to fix “Blocked due to other 4xx issue” in Google Search Console.
The Index Coverage (Page indexing) report shows a detailed overview of your crawling and indexing issues and points to how they should be addressed, making it a vital source of SEO data.
Your website’s crawling and indexing status is not straightforward – not all of your pages should be crawled or indexed. Ensuring such pages are not accessible to search engine bots is as crucial as having your most valuable pages indexed correctly.
The report reflects the fact that your indexing status is not either black or white. It highlights the range of states that your URLs might be in, showing both serious errors and minor issues that don’t always require action. If you’re struggling to understand what action you should take to improve your website’s indexing, contact us for technical SEO services.
Ultimately, you should regularly browse Google’s Index Coverage (Page indexing) report and intervene when it doesn’t align with your indexing strategy.