Read the Transcript
Martin Splitt: Word of mouth, ads, okay, that’s two ways of getting people on the website, you can also sit at a computer and just go on your website or something, I don’t know.
So I’m really, really happy that we have our next speaker here, named Bartosz, who is being – constantly challenging my life, basically. By basically constantly running tests and going like, “Hey! Quick question! Is this expected behavior that we should see from Google search?” And I’m like, “Yes. Can you explain it to me? Oh, wow, OK!”
So he asks me the really, really hard questions, but he’s a really, really good presenter in answering the hard questions.
So please give it up for Bartosz.
Bartosz Goralewicz: Thank you so much, Martin, for such a nice intro. So we’re most – usually challenging Martin or John [Mueller], so that’s – most of our job is basically annoying Googlers.
Martin (offscreen): We have started the project I believe, and we are maintaining it, but just, just for the record: Angular people are really, really nice people. They come to us and say, “Martin can you help us with this thing?” And I’m like “Nope! There’s the open public forum where you can ask those questions and get the same support.”
Bartosz: So Angular, what’s actually – something that probably Martin wouldn’t say, but Angular was one of the most problematic frameworks.
Martin (offscreen): That’s true!
Bartosz: Which is not something that you would expect and the website Angular.io wasn’t indexed in Google for a few years.
If you look at Wish.com and their visibility chart, you can see that this relationship is not going well.
And desktop is dead.
And this is the data from Germany, and if you look at Switzerland it’s actually very, very surprising because if you look, again, Germany 23% for Google.de comes from desktop. So just 20% of traffic comes from desktop devices right now. In Switzerland this is shockingly – you guys still have quite a lot of desktop traffic which I totally don’t understand.
Like, in Europe you’re, like, actually one of the exceptions. But most countries, you guys, are not – most countries actually don’t have that much desktop traffic any more.
This is – this is how their traffic looked after that. They’ve lost overnight – they lost more than 40%, then within the next upcoming weeks, they’ve lost all together around 70% of the traffic. It goes so bad that you couldn’t find any of the Hulu shows in Google and they were only available at Hulu.com. So if you want to watch any show from Hulu.com the only website that would show up in Google were torrents.
This is how bad it looks like and it’s still not fully fixed until today. But let’s move forward.
So what exactly went wrong? There was no data at that time and we figured, okay, if there is no data then we need to get it somehow. I need to figure we’re gonna create a dream team to basically build an experiment.
The idea was extremely, extremely geeky. A simple query – they were very, very simple. We basically built a website where every single subpage is a different framework. So we would see, OK, how is Google, and other search engines we’re gonna talk about that in a second, how are they dealing with indexing all the different frameworks?
And we found quite a lot of different anomalies. We played with the data and we basically found a video from 2015 from Jeff Whelpley, who’s somehow involved in the Angular project, saying that if you care about SEO, you still need to have server-rendered content. And this got us thinking how our other sections was dealing with that. And if you look at that, only Google and Ask. Ask is not saying that publicly, but Ask is using Google’s servers and basically database, so Ask is just simply Google. It’s not publicly announced but we fight with that, and it is.
Before I move forward with that, I’m guessing you’re pretty technical so I’m not sure this slide is important, but just for those of you who aren’t: HTML is just like a ready-to-go cake, so all the bots, all the users, just basically get the content they seek.
We’ve been actually waiting for that for – for two years, from like – the main problem we had is that developers didn’t really believe us, even though we had experiments – because we are the only ones talking about that. So this I don’t even remember that was a tweet or a hangout, this sentence from John Mueller is something we almost tattooed on our foreheads for over a year because we finally had something to talk about with developers. And his advice helped in general, so he’s – his help was really appreciated.
And I know it works both ways, I know that Google actually used that experiment as well. So after failing at the beginning with John Mueller, I reached out to most, like, like, we actually reached out to, like, 20 different Googlers, but Ilya Grigorik was the one who responded. A lot of them were emails, tweets and whatever. And we invited Ilya Grigorik to our Google Doc with every single thing we found out and he started commenting, commenting like crazy for a day, and then he disappeared.
Someone offscreen: Scale!
In our experiment, in our experience, it can take up to forever, which is – So we have websites from 2016 that are still not indexed.
Just to show you it’s something that’s extremely, extremely interesting here is that – Oh! So we’re not gonna finish on time. We have 100 slides and, yeah. So mobile Google bot and basically Chrome looks at the content from a different point of view, at that time, because they were using Chrome41. And we saw something that’s called Casual Indexing – actually, I came up with that after we found that first.
So all the content that wasn’t available for mobile Google bot wasn’t indexed. So if you see that was available only for basically browsers newer than Chrome 41, and it wasn’t available for mobile Google so this comment wasn’t indexed at all, but at the same time – I’m sorry this is like geek humor – at the same time, if you look at anything that’s available in both those screens, is fully indexed.
They indexed only the static page of their website, which is hilarious to me because it took them like one-half year or something like that . . . But I’m guessing it’s not funny for Hulu. Anyways, any content here, again, wouldn’t be indexed in Google. So they indexed just part of their website.
I’m guessing they either built new servers or hired more developers. I have no idea what they did.
We probably need to get Martin drunk and find out. By the way, it’s every SEO’s dream just to get a Googler drunk. It’ll never happen. So we’re still waiting, Martin.
[Martin speaks offscreen]
The crawler budget repeatedly in 2019, if you remember the explosion here, is gone. So basically the test websites we’ve created a few months ago, a few weeks ago . . . they all work.
Again, random sample, the system likes new content – it’s just like that. So if two waves of indexing – there’s one more thing, as I mentioned before you need to expect some delays indexing your content and this is the most shocking slide I have in my deck.
I’m guessing some of you will get that. We will see that as extremely interesting here. So that’s a geek test. If you find it interesting, sorry.
Allrecipes, 14% is also quite good. But, for example, I thought 50% of the content is still indexed after 10 days.
But we’re going to talk about a little bit of black hat, and again, I’m super sorry, Martin. This is not like a link-building black hat, this is geeky black hat. So cloaking 2019, this is something that we find extremely interesting. And we have a few websites, we have like Pepe, we have guns, we have Trump.
Today we’re gonna talk about guns because I know that you guys have guns in Switzerland. So we use this example. This is actually a website that’s cached by Google. So Google sees, OK, gun control – we’re all for that.
But if you open that website in your browser you’re gonna see completely different content.
And this is not only about the yes or no, but the whole content is completely different.
One last thing this is my second last slide – almost.
So once they do that Bing is gonna be a massive, massive problem – they’re gonna actually struggle with that. Suggested to do for you, because I believe that everything should be actionable if you’re choosing a framework please leave these don’t go with, for example, Knockout, because we have clients who implemented Knockout a year ago, if you look at Twitter, the last tweet is from three years ago, so we can assume that this framework is completely bad.
Martin: Bing is a fine good search engine. It’s not like they’re – we’re actually working a lot of with Bing because fundamentally we want the same thing . . . we all want the web to win and they’re doing fantastic things. And I’m not so sure that they don’t catch up quickly and they’re actually doing great stuff behind the scenes.