How to Make JavaScript Websites Successful in Google

How to Make JavaScript Websites Successful in Google?
A few words about myself
I’m an SEO guy. I am not your enemy 😉
I like programming
With my colleagues: We have literally spent years on: Maria Cieślak Bartosz Góralewicz – researching the JavaScript SEO thing – helping multiple clients to ensure their JS websites rank high in Google
My Ultimate Guide to JavaScript SEO was extremely viral
So…
Client: This is our website. We don’t rank in Google. Can you guys help us?
We will help you.
A JavaScript loader was the bad guy. Finally, we spotted the issue! LOADING…
Now they are successful in Google!
We need to go deeper
Then we started researching The JavaScript SEO topic
Yo! Here is a list of simple Hello World Projects utilising different JS frameworks used for JS crawling tests JSSEO.expert experiment
“But I am a developer. Why should I care about traffic from Google?”
“But I am a developer. Why should I care about traffic from Google?” ● Traffic from Google = Money
We SHOULD care about traffic from Google. It’s REAL. ● Traffic from Google = Money ● Poor JavaScript SEO = Google Ranking Drops
We SHOULD care about traffic from Google. It’s REAL. ● Traffic from Google = Money ● Poor JavaScript SEO = Google Ranking Drops ● Google Ranking Drops = Less money = Angry boss
We SHOULD care about traffic from Google. It’s REAL. ● Traffic from Google = Money ● Poor JavaScript SEO = Google Ranking Drops ● Google Ranking Drops = Less money = Angry boss …and it makes cats grumpy
A few examples of famous JavaScript website drops in Google
Example #1
Example #2 sports.yahoo.com
Example #3 answers.yahoo.com
The question…
The answer…
What if I told you that Google’s not perfect at rendering JS?
Client-side rendering – the default technique It’s like a cooking recipe – Google and users have to render it on their own. It can be problematic for Google
Yes… Recently, Google did a really good job with updating their Web Rendering Service BEFORE the 7th of May AFTER the 7th of May Google has been using an “ancient” 4-year old browser for rendering – Chrome 41. It wasn’t even supporting ES6! Google Web Rendering Service is based on the most recent version of Chrome It’s great news, but things are more complicated than that…
…Google is still not perfect at rendering JavaScript
Google is still not perfect at rendering JavaScript
Google crawls JavaScript websites much slower than HTML Pages
Two waves of indexing
But how long do you HAVE to wait? Google’s John Mueller: “…usually it’s on the order of days to a few weeks even.”
Google is still not perfect at rendering JavaScript
The risk: Google may decide not to fetch some JavaScript resources
The Risk: Google may decide not to fetch some JavaScript resources I think we DON’T need to download this JSON file
Google is still not perfect at rendering JavaScript
Googlebot doesn’t act like a real user It doesn’t click the buttons It doesn’t scroll! Watch out for your infinite scrolling!
There are still many websites that implement infinite scrolling incorrectly… In the case of these websites Googlebot cannot access the second page of pagination.
Google is still not perfect at rendering JavaScript
Google’s rendering of JavaScript is not perfect TIMEOUTS
Google limits: TIMEOUTS Remember: There are 130 trillion other documents waiting for Googlebot’s visit
Googlebot is based on the newest version of Chrome. BUT it’s not the same! ● Googlebot declines user permission requests ● Cookies, local & session storage are cleared across page loads ● Browser always download all the resources For instance: Googlebot may not…
Always check if Google has no issues with rendering your website Best tool: URL Inspection Tool (a part of Google Search Console). Avoid using Chrome for it.
Google’s rendering errors may be a big deal Google encounters rendering errors Google stops executing JavaScript No content indexed No money
Angular.io had JavaScript SEO issues in the past
Igor Minar from Angular.io: “Given that we haven’t changed the problematic code in 8 months (…) I believe that something has changed in crawlers during this period of time which caused most of the site to be de-indexed, which then resulted in the traffic loss.” The issue: Google deindexed many pages of Angular.io due to rendering errors
Let’s go back to our experiment… JSSEO.expert
Initial results: Google DOESN’T support Angular 2
Initial results: Google DOESN’T support Angular 2
Google didn’t index our Angular experiment because of… errors in the official Angular documentation
Now we know about Google’s limitations in rendering JavaScript
JavaScript SEO Taming the Beast in 5 Easy Steps
Step 1 Ensure that Google can render your content Use the URL inspection tool provided by Google ● It’s free
Step 1 Ensure that Google can render your content Use the URL inspection tool provided by Google ● It’s free ● It shows rendering errors
Ask yourself the following questions
Ask yourself the following questions ● Is the main content visible? ● Can Google see the user-generated comments? ● Can Google access areas like similar articles and products? ● Can Google see other crucial elements of your page?
What if Google cannot render your page properly? ● You blocked crucial JavaScript files for Googlebot ● Some errors occurred while rendering Here are some possible reasons: ● Google encountered timeouts while rendering
Step 2 Make sure you didn’t block your crucial JavaScript files in robots.txt What is robots.txt?
Step 2 Make sure you didn’t block your crucial JavaScript files in robots.txt Google’s URL inspection tool comes to the rescue!
Step 3 Use the URL Inspection tool for spotting JavaScript errors
Step 4 Check if your content has been indexed in Google How do you know if content is really indexed in Google?
Step 4 Check if your content has been indexed in Google site:URL “fragment”
Step 5 Make sure Google can discover your internal links Google needs proper <a href> links to discover the URLs on your website
If you followed these 5 steps and it still doesn’t work…
Consider dynamic rendering or hybrid rendering Example Recommended by Dynamic rendering YouTube Hybrid rendering (also called “universal JS” or “isomorphic JS”) Airbnb
First option: Use dynamic rendering USERS SERVER RENDERERGooglebot Initial HTML+JS Static HTML BUT a static version for Googlebot ● Serving an interactive version for users
First option: Use dynamic rendering Officially recommended by Google for rapidly changing websites
Most popular dynamic rendering solutions Renderton Puppeteer Prerender.io Free Free Service/Free
But dynamic rendering is not a silver bullet
Disqus.com
What went wrong?
We informed Disqus about the issue
Second option: Hybrid rendering How it works Initial, server-side rendered HTML is sent to users and search engines. Then JavaScript is loaded on top of that.
Hybrid rendering? Which brands use
If you use for hybrid rendering …tryNext.js is the next… BIG thing
The State of JS 2018 Survey: 35% of developers heard of Next.js and want to learn it.
It’s actively maintained
Friends of Next.js Vue.js Angular
So many choices…
Relatively static website Dynamic website You can consider Client-side rendering ● Hybrid rendering, or ● Dynamic rendering
It’s not only about Google! Other search engines struggle with JavaScript. Facebook, Twitter THE SAME!
Technically, Bing can render JavaScript. But struggles with rendering JavaScript at scale
Bing struggles with rendering JavaScript at scale
Bing struggles with rendering JavaScript at scale
Can Bing index the content of Angular.io?
Bing struggles with rendering JavaScript at scale Can Bing index the content of Vimeo?
Can Bing index the content of Vimeo?
Let’s talk about… Twitter
Angular.io is doing fine on Twitter
Angular.io is NOT doing fine on Twitter Vue.js is doing MUCH better.
Key takeaways ● It’s the HTML JS Era (lucky you!) ● Rendering delays are still a problem ● Even after Chromium update, Google still doesn’t support many modern JavaScript features (i.e. no service workers, no geolocation API) ● You should always ensure that Google can render and index your content (choose a sample of pages)
Worth reading ● Ultimate Guide to JavaScript SEO https://www.onely.com/blog/ultimate-guide-javascript-seo/ ● How to Diagnose and Solve JavaScript SEO Issues in 6 Steps by Tomek Rudzki https://moz.com/blog/diagnose-and-solve-javascript- seo-issues ● How to prepare for a JS migration by Maria Cieślak https://searchengineland.com/how-to-prepare-for-a-js-migration-308795
Thank you, everyone! @TomekRudzki /TomekRudzki