Tag Archive for: search off the record

Google Experts Share The Two Most Common Crawling Issues

,

Crawling and indexing issues are one of the most damaging SEO issues a site can have. Not only do they hurt your rankings, making your business and products less visible in search results. These types of issues can completely prevent pages or entire sections of your site from being properly added to Google’s search indexes.

Now, two of Google’s most well-known representatives have shed light on the two biggest crawling issues the search engine encounters regularly

In a recent Search Off the Record podcast, Gary Illyes and Martin Splitt went into detail on the biggest crawling challenges Google faces in 2025, including the two biggest issues Google sees. 

According to Illyes and Splitt, faceted navigation and action parameters account for approximately 75% of crawling issues that Google encounters. 

Both issues create crawling problems that can overload your server, slow your site down significantly, and create infinite crawling loops. If this happens, Google’s crawlers can expend a massive amount of energy and server bandwidth that can bring your website to a screeching halt and even make your site entirely inaccessible in some cases. 

The Two Biggest Crawling Issues 

Gary Illyes says the two biggest issues account for 75% of crawl challenges.

  • Faceted Navigation – This accounts for 50% of issues according to Google. This is a navigation strategy (typically used on e-commerce sites) to allow users to filter and navigate items based on specific details like price, manufacturer, or size. The issue is that this system can generate a seemingly endless number of URL patterns if it creates a URL for every single combination of filters. Without careful management, this can lead to Google crawlers expending crawl budgets on URLs with negligible search value, duplicate content issues, and slower site performance. 
  • Action Parameters – These account for 25% of challenges. These are URL parameters used to trigger or track specific user actions, such as adding an item to a cart or saving an item to a wishlist. Importantly, these sorts of parameters don’t tend to meaningfully change page content, creating widespread duplicate content issues on your site. 

Additionally, Illyes mentioned a few other, less common crawling challenges:

  • Irrelevant Parameters – These account for 10% of issues. These problems pop up when crawlers notice strings of parameters (typically used to track session ID numbers or UTM parameters) attached to content that Google’s systems deem irrelevant to the actual navigation or page content. 
  • WordPress Plugins or Widgets – Approximately 5% of crawling issues come from WordPress plugins and other tools for sites using similar CRM’s. In some cases, these widgets may modify URLs for event tracking. Google can struggle to understand when this happens, because there is not an established system or pattern that these tools follow. 
  • “Weird Stuff”: – Lastly, Illyes attributed approximately 2% of problems to rare technical issues that pop-up. In the podcast, he cited times when URLs may be double-encoded. This means that when the crawler decodes the URL, it is still left with an unusable encoded string instead of a functional URL. 

In the discussion, Illyes and Splitt go more into detail about these issues, what causes them to arise, and how to prevent them. For more, listen to the full episode here.

/0 Comments

Google Says To Optimize For Humans, Not For AI

, ,

Since AI overviews have taken over search results and reduced the amount of clicks going to traditional search results, a debate has emerged within the SEO community. While one half of the community insists that the rise of AI calls for an entirely new approach to website optimization, others have cautioned that there is no need to reinvent the wheel. 

Now, two leading figures from Google have chimed in with a lengthy discussion on AI, how it is impacting search, and whether you need to be doing anything new to improve your visibility in AI overviews. 

Do You Need To Be Doing Anything New To Optimize For AI Search?

In a recent episode of the Search Off The Record podcast, Google’s Danny Sullivan and John Mueller spoke about AI and how it has changed search (and how it hasn’t). While they concede that it feels like search has changed significantly over the past couple years, the rise of artificial intelligence in search doesn’t actually call for any changes to how you optimize your website. 

Instead, they say that AI overviews use many of the same signals that were being used by the search engine.

To start the discussion, Jonn Mueller asked:

“So everything kind of around AI, or is this really a new thing? It feels like these fads come and go. Is AI in fad? How do you think?”

To which, Danny Sullivan said:

“Oh gosh, my favorite thing is that we should be calling it LMNOPEO because there’s just so many acronyms for it. It’s GEO for generative engine optimization or AEO for answer engine optimization and AIEO. I don’t know. There’s so many different names for it.

I used to write about SEO and search. I did that for like 20 years. And part of me is just so relieved. I don’t have to do that aspect of it anymore to try to keep up with everything that people are wondering about.

And on the other hand, you still have to kind of keep up on it because we still try to explain to people what’s going on. And I think the good news is like, There’s not a lot you actually really need to be worrying about.

It’s understandable. I think people keep having these questions, right? I mean, you see search formats changing, you see all sorts of things happening and you wonder, well, is there something new I should be doing? Totally get that.

And remember, we, John and I and others, we all came together because we had this blog post we did in May, which we’ll drop a link to or we’ll point you to somehow to it, but it was… we were getting asked again and again, well, what should we be doing? What should we be thinking about?

And we all put our heads together and we talked with the engineers and everything else. So we came up with nothing really that different.”

Google’s Systems Prioritize The Best Content For Humans

According to Danny Sullivan, the reason you don’t need new strategies to optimize for overviews is that Google’s AI systems aim to surface the best content for human users – the same as Google’s traditional search systems. 

As Sullivan explained:

“And when it comes to all of our ranking systems, it’s about how are we trying to reward content that we think is great for people, that it was written for human beings in mind, not written for search algorithms, not written for LLMs, not written for LMNO, PEO, whatever you want to call it.

It’s that everything we do and all the things that we tailor and all the things that we try to improve, it’s all about how do we reward content that human beings find satisfying and say, that was what I was looking for, that’s what I needed. So if all of our systems are lining up with that, it’s that thing about you’re going to be ahead of it if you’re already doing that.

To whereas the more you’re trying to… Optimize or GEO or whatever you think it is for a specific kind of system, the more you’re potentially going to get away from the main goal, especially if those systems improve and get better, then you’re kind of having to shift and play a lot of catch up.

So, you know, we’re going to talk about some of that stuff here with the big caveat, we’re only talking about Google, right? That’s who we work for. So we don’t say what, anybody else’s AI search, chat search, whatever you want to kind of deal with and kind of go with it from there. But we’ll talk about how we look at things and how it works.”

Optimizing For Humans Is The Key To Google Success

While Sullivan’s comments are limited to Google’s own AI systems, they make it clear that optimizing for anything or anyone other than the end user is a mistake. Instead, it is crucial to do everything possible to deliver the best content and most value for real human visitors to your site. 

This makes sense in the big scheme of things. Ultimately, Google’s goal is to provide the best experience for real human users and delivering websites optimized primarily for artificial systems isn’t likely to win over many users. If you keep your goals aligned with providing the most value and best experience for human users, you’re better positioned to be the type of site Google wants to highlight. 

For more from Danny Sullivan’s discussion with John Mueller, listen to the full Search Off The Record episode here.

/0 Comments

What’s Changing The Updated SEO Starter Guide From Google?

,

The Google SEO Starter Guide is designed to help individuals and organizations quickly learn the most important steps necessary for getting their websites ranking within Google Search. 

While the guide reportedly maintains a 91% approval rating, it has largely gone without updates for several years but that will be changing soon.

In a recent episode of Google’s “Search Off The Record” podcast, the company’s Search Relations team discussed plans to update the SEO Starter Guide, including talking about what would and would not be included in the revised document. 

Discussions like this are great for seeing how SEO is talked about within the search engine and learning what the company prioritizes when ranking sites along with identifying SEO myths that might lead you astray when optimizing your own site. 

So, what’s changing in the revised SEO Starter Guide?

HTML Structure

One topic the group discussed was the importance (or lack thereof) of HTML structure when it comes to online rankings.

While the team agreed that using proper HTML structure can help with online rankings, they indicated the guide will clarify that these are not all that important in the grand scheme.

As Google’s Gary Ilyes said:

“Using headings and a good title element and having paragraphs, yeah, sure. It’s all great, but other than that it’s pretty futile to think about how the page… or the HTML is structured.”

Branded Domain Names vs Keyword Rich Domain Names

SEO experts have been increasingly debating whether it is better to focus on your existing branding when establishing a domain name, or if domains perform better when including specific keywords.

According to the Google team, the new guide will clarify this by indicating that brands should focus on including branding in their domains over using keywords. The thought process shared by those in the discussion was that establishing a memorable brand will have a more long-term impact than trying to optimize your domain specifically for search engines. 

Debunking SEO Myths

Lastly, the group said one thing they want to improve in the document was how it addressed widespread SEO myths and misconceptions. 

For example, everyone agreed that the SEO Starter Guide should specifically debunk the idea that using Google products while creating or optimizing your site will improve search rankings. 

They indicated they would address this myth and several others to prevent people from optimizing their site based on misinformation found elsewhere online. 

For more from the discussion, listen to the entire episode of “Search Off The Record” here.

/0 Comments