New directions given to Google’s quality raters tell them to look for signs that a page’s main content is AI-generated or otherwise made using automated tools. If a page is found to be primarily made with AI, according to Google’s John Mueller, raters are asked to rate it as “lowest quality.”
Though this policy shift was apparently part of the January 2025 Search Quality Rater Guidelines update, Mueller first publicly revealed it this week while speaking at Search Central Live Madrid.
Why This Matters
Though Search Quality Raters do not directly affect Google’s search results, their work is used to improve Google’s algorithms. The way they are asked to rank pages typically reflects Google’s overall internal guidelines.
Here’s what Google had to say when they updated the raters’ guidelines in January:
“As a reminder, these guidelines are what are used by our search raters to help evaluate the performance of our various search ranking systems, and their ratings don’t directly influence ranking. The guidelines share important considerations for what content is helpful for people when using Google Search. Our page on how to create helpful, people-first content summarizes these concepts for creators to help them self-assess their own content to be successful in Google Search.”
If Google is instructing its raters to give AI-generated content, it is a sign that the company is hardening its stance on AI content and moving to reduce its presence in search results.
AI content has always been a risky prospect when it comes to SEO, but this is the one of the most significant signs we’ve seen from Google itself that AI-generated content may be unwelcome in search results.
Google Search Liaison Danny Sullivan spent time during a recent Google Search Central Live NYC event to give advice to small, independent websites wanting to rank in the search engine against big brand competition.
Does Google Favor Big Brands?
According to a report from Search Engine Journal’s Roger Montti, Danny Sullivan spent part of the event answering questions from attendees. Sullivan’s advice to small brands was inspired by one suh question asking if Google was moving towards primarily showing a shrinking number of websites – specifically, big websites.
While Sullivan said he understood the perception that Google Search is designed to favor big brands or that big brands will always outrank smaller sites, he emphatically stated this isn’t the case.
In fact, Danny Sullivan told the crowd that Google is specifically working to improve how it handles smaller sites and give them more opportunities to be successful.
“We’ve been spending a lot of time (and we’re going to continue to spend a lot of time) to understand how can we do a better job on better understanding and perhaps guiding some of the smaller creators and small independent sites so they can be successful. It has been like a huge chunk of my time over the past year. And I’m not alone in it.”
Why It Feels Like Google Favors Big Websites
During his response to the question, Sullivan spoke at length about what leads big brands to rank well and why it is difficult to detach that from how search works.
“And I’ve seen where people do research and say, ‘I’ve figured out that if you have a lot of branded searches…’ That’s kind of valid in some sense.
But it’s not like you have a lot of big branded searchers or small branded searchers or whatever and you’re finding that correlates to your traffic. What it’s saying is that people have recognized you as a brand, which is a good thing. We like brands. Some brands we don’t like, but at least we recognize them, right?
So if you’re trying to be found in the sea of content and you have the 150,000th fried chicken recipe, it’s very difficult to understand which ones of those are necessarily better than anybody else’s out there.
But if you are recognized as a brand in your field, big, small, whatever, just a brand, then that’s important.
That correlates with a lot of signals of perhaps success with search. Not that you’re a brand but that people are recognizing you. People may be coming to you directly, people, may be referring to you in lots of different ways… You’re not just sort of this anonymous type of thing.
So, one thing I would encourage anybody, but especially to smaller and independent ones that are kind of feeling like the big brands are kind of getting it all is, are you making sure that people understand who you are?”
What Small Sites Can Do To Compete
Reaching the crux of the discussion, Sullivan said that helping users and search engines understand who you are and what sets you apart are crucial for competing against bigger brands.
“Anytime you ever have a question about what you should be doing to be successful in Google search and your answer is to ask if it’s a good thing for your readers, if you do that, you are aligning with the things we’re trying to do because we’re trying to send people to satisfying content so that they go, ‘This was great! This is wonderful, I loved it!’
So when they wind up on your website, probably for the first time and they don’t know you from anything and they’re coming from this crazy world where they don’t even know where the profiling for the author is, make it easy for them. Make it easy for them to come into the site and know exactly what you’re about.
I know the travel bloggers, you all have the thing on the side that says, ‘we love travelling the world…’ It’s like, OK, that’s fine and at least people know to expect that from travel bloggers and you’ve got it there.
But help them understand what’s unique or different about you, that makes you a brand. And that is a really good thing.”
It is never easy to be David challenging a Goliath in your industry, but there are ways to overcome. By defining who you are clearly and what sets you apart from the bigger names, you give Google a better chance of understanding your website and why it should be prioritized over well-established international brands.
https://www.tulsamarketingonline.com/wp-content/uploads/2021/02/Google-Rankings-and-Blog-Comments1.png5761200Taylor Ballhttps://www.tulsamarketingonline.com/wp-content/uploads/2018/07/TMO-Logo.pngTaylor Ball2025-03-28 17:37:062025-03-28 17:37:08Danny Sullivan From Google Explains How Small Sites Can Compete Against Giants
In a wide-ranging interview with The New York Times, Google CEO Sundar Pichai said he expects that search will “change profoundly” in 2025 led by advancements in AI and increasing competition from AI search, social media, and hardware advancements.
Below, we’ve collected highlights from the interview that may give us a peak at Google’s plans for 2025 and beyond.
Google Aims To Be a Leader With AI Development
When asked about where Google is today in comparison to the rest of the market, Pichai emphasized that the company is in the early stages of developing radically powerful new AI tools. Additionally, he emphasized that AI developments that may not seem connected to the company are largely built on the back of research and development made possible with Google’s open-sourced technologies.
“Look, it’s a such a dynamic moment in the industry. When I look at what’s coming ahead, we are in the earliest stages of a profound shift. We have taken such a deep full stack approach to AI.
…we do world class research. We are the most cited, when you look at gen AI, the most cited… institution in the world, foundational research, we build AI infrastructure and when I’m saying AI infrastructure all the way from silicon, we are in our sixth generation of tensor processing units. You mentioned our product reach, we have 15 products at half a billion users, we are building foundational models, and we use it internally, we provide it to over three million developers and it’s a deep full stack investment.
We are getting ready for our next generation of models, I just think there’s so much innovation ahead, we are committed to being at the state of the art in this field and I think we are. Just coming today, we announced groundbreaking research on a text and image prompt creating a 3D scene. And so the frontier is moving pretty fast, so looking forward to 2025.”
Using AI To Enhance Search Instead of Replace It
Increasingly, AI is viewed by many as a competitor to traditional search, leading the interviewer to ask what Google is doing to protect the “blue link economy” in order to not “hurt or cannibalize” its search engine and the market around it.
In response, Pichai discussed how AI has been a major part of Google’s development for longer than most people realize. Going back as far as 2012, AI has been part of Deep Neural Networks used to identify speech and images. Since then, artificial intelligence has been a core part of the search engine’s development.
“The area where we applied AI the most aggressively, if anything in the company was in search, the gaps in search quality was all based on Transformers internally. We call it BERT and MUM and you know, we made search multimodal, the search quality improvements, we were improving the language understanding of search. That’s why we built Transformers in the company.
So and if you look at the last couple of years, we have with AI overviews, Gemini is being used by over a billion users in search alone.”
Where Is Search Going In 2025?
Looking forward, Pichai says he believes Search will be radically changing – and soon. While he says that advancement is becoming more difficult because the easiest innovations have already been done, he still believes that people will be surprised at how much is coming in just the first part of 2025.
“And I just feel like we are getting started. Search itself will continue to change profoundly in 2025. I think we are going to be able to tackle more complex questions than ever before. You know, I think we’ll be surprised even early in 2025, the kind of newer things search can do compared to where it is today…
I think the progress is going to get harder when I look at 2025, the low hanging fruit is gone.
But I think where the breakthroughs need to come from where the differentiation needs to come from is is your ability to achieve technical breakthroughs, algorithmic breakthroughs, how do you make the systems work, you know, from a planning standpoint or from a reasoning standpoint, how do you make these systems better? Those are the technical breakthroughs ahead.”
Will AI Replace Traditional Search?
As increasing numbers of people seem to be relying on AI tools to get quick answers instead of using traditional search tools, some have suggested that AI could eventually replace search as we know it. At the same time, there are concerns that AI may be delivering less reliable or accurate answers, which Pichai believes will ensure that Google’s search tools remain relevant if not more valuable.
“In a world in which you’re flooded with like lot of content …if anything, something like search becomes more valuable. In a world in which you’re inundated with content, you’re trying to find trustworthy content, content that makes sense to you in a way reliably you can use it, I think it becomes more valuable.
To your previous part about there’s a lot of information out there, people are getting it in many different ways. Look, information is the essence of humanity. We’ve been on a curve on information… when Facebook came around, people had an entirely new way of getting information, YouTube, Facebook, Tik… I can keep going on and on.
…I think the problem with a lot of those constructs is they are zero sum in their inherent outlook. They just feel like people are consuming information in a certain limited way and people are all dividing that up. But that’s not the reality of what people are doing. “
The full interview touches on several other questions including potential upcoming regulations, how the search engine views its responsibility towards creators, and other Google platforms like YouTube’s future direction. You can watch it here or below:
https://www.tulsamarketingonline.com/wp-content/uploads/2020/01/GoogleRankings.png360640Taylor Ballhttps://www.tulsamarketingonline.com/wp-content/uploads/2018/07/TMO-Logo.pngTaylor Ball2024-12-20 17:30:372024-12-20 17:30:39Google’s CEO Gives Wide-Ranging Interview On The Future of Google and Search
A lead Google spokesperson gave a surprising response to claims that the search engine stole content from a publisher without providing any benefit to the publisher’s website.
Google’s rich search results have been controversial since their launch, as some feel that these results simply copy information from other websites instead of sending users to that content where it was originally posted.
The search engine has largely ignored these criticisms by saying that rich results improve the search experience and include links to the original content.
That’s what makes it so surprising that Google Search Liaison Danny Sullivan recently publicly responded to one publisher’s complaints directly.
The Original Complaint
In several recent tweets, a representative for travel brand Travel Lemming posted:
How is this legal?
Google is now stealing Travel Lemming's own brand searches (even via site search).
They take our list — INCLUDING MY ORIGINAL PHOTOS 📸 — and present it in a rich result so people don't click through.
They are doing this across all travel searches – unbranded and branded alike.
Example: "Mexico Travel Tips" – they have an AI answer & also a rich result that basically just re-creates an entire blog post, including our stolen photos.
“Google is now stealing Travel Lemming’s own brand searches (even via site search).
They take our list — INCLUDING MY ORIGINAL PHOTOS 📸 — and present it in a rich result so people don’t click through.
I am literally IN that Red Rocks photo!…”
They are doing this across all travel searches – unbranded and branded alike.
Example: “Mexico Travel Tips” – they have an AI answer & also a rich result that basically just re-creates an entire blog post, including our stolen photos.
Again, I am IN that Mexico packing photo!
Like how is it legal for Google to just essentially create entire blog posts from creators’ content and images?
I literally have a law degree from the top law school in the world, and even I can’t figure it out!
Fair use does NOT apply if you’re using the content to compete directly against the creator, which they clearly are.
I can’t sit outside a movie theatre, project the movie on a wall, earn money from it, and claim fair use.
I spent SO much time taking those photos in Denver.
It was 10+ full days worth of work for me and partner Clara, going around the city to photograph everything. $100s of money spent in attraction admission fees, gas, parking.
Now Google just gets to extract all that value?
How much does Google get to take before creators say “enough is enough”?
How hard does the water have to boil before the frog jumps?
The comments show it is a prisoner’s dilemma as long as Google has a monopoly on search …”
Google’s Response
Danny Sullivan, Google’s Search Liaison, provided a lengthy response that delves specifically into what is happening, why, and ways they are hoping to improve the situation.
Not only does Sullivan give insight into the company’s perspective, but also their own opinions about the function. Importantly, Sullivan doesn’t disregard Travel Lemming’s complaints and is sympathetic to how rich search results impact publishers:
Hey Nate, this got flagged to my attention. I'll pass along the feedback to the team. Pretty sure this isn't a new feature. Elsewhere in the thread, you talk about it being an AI answer, and I'm pretty sure that's not the case, either. It's a way to refine an initial query and… pic.twitter.com/rO2dyaISbT
I appreciate your thoughts and concerns. I do. The intention overall is to make search better, which includes ensuring people do indeed continue to the open web — because we know for us to thrive, the open web needs to thrive. But I can also appreciate that this might not seem…
“Hey Nate, this got flagged to my attention. I’ll pass along the feedback to the team. Pretty sure this isn’t a new feature. Elsewhere in the thread, you talk about it being an AI answer, and I’m pretty sure that’s not the case, either. It’s a way to refine an initial query and browse into more results.
With the example you point out, when you expand the listing, your image is there with a credit. If you click, a preview with a larger view comes up, and that lets people visit the site. Personally, I’m not a fan of the preview-to-click.
I think it should click directly to the site (feedback I’ve shared internally before, and I’ll do this again). But it’s making use of how Google Images operates, where there’s a larger preview that helps people decide if an image is relevant to their search query. Your site is also listed there, too. Click on that, people get to your site.”
If you don’t want your images to appear in Google Search, this explains how to block them:
I suspect you’d prefer an option to not have them appear as thumbnails in particular features. We don’t have that type of granular control, but I’ll also pass the feedback on.
I appreciate your thoughts and concerns. I do. The intention overall is to make search better, which includes ensuring people do indeed continue to the open web — because we know for us to thrive, the open web needs to thrive.
But I can also appreciate that this might not seem obvious from how some of the features display.
I’m going to be sharing these concerns with the search team, because they’re important.
You and other creators that are producing good content (and when you’re ranking in the top results, that’s us saying it’s good content) should feel we are supporting you.
We need to look at how what we say and how our features operate ensure you feel that way.
I’ll be including your response as part of this.”
I doubt Sullivan is going to change many minds about Google’s rich search results, but this rare interaction is revealing to how Google sees the situation and is trying to walk a tightrope between providing a seamless search experience while sustaining the sites it relies on.
https://www.tulsamarketingonline.com/wp-content/uploads/2021/02/Google-Rankings-and-Blog-Comments1.png5761200Taylor Ballhttps://www.tulsamarketingonline.com/wp-content/uploads/2018/07/TMO-Logo.pngTaylor Ball2023-12-21 19:12:132023-12-21 19:12:15Google’s Danny Sullivan Responds To Claims That Rich Results Steal Content
Google has released its annual Year in Search list of trends and popular searches, but this year’s list includes a special twist.
To celebrate 25 years as a search engine, this year’s Year In Search includes a ton of interesting fun facts, milestones, and a time capsule that helps users see how far we’ve come.
25 Years of Digital Search
To open the latest Year in Search report, Google included a film that highlights how our interests and technology have changed in the last two and a half decades.
The video includes all sorts of gigantic accomplishments from popular figures such as Taylor Swift and iconic brands like Pokemon, along with massive social and scientific developments like nuclear fusion, and the increasing acceptance of marriage equality.
The Time Capsule
The Google Trends Time Capsule shows the most popular searches for certain categories from each year.
For example, the graph below lets you see Pokemon’s gradual fall out of the top five card games and eventual return to popularity 17 years later.
You can explore the featured categories, gain a new perspective of the most powerful trends, and even take an interactive quiz to see which year’s search trends were most interesting to you.
Most Searched Playground
Another new part of the Year in Search report is the inclusion of an interactive Google Doodle/game that highlights many of the most popular places, people, and events from the past 25 years.
Zoom in and explore to find over 1,700 figures, easter eggs, and other surprising finds while touring Google’s history as a search engine.
The Year In Search Report
Of course, we have the annual search trends report itself. The Year in Search 2023 report highlights the most popular searches of the past year for several categories. You can explore the global trends that have shaped the world or filter the trends by country.
Local Year In Search
If you are looking for more localized trends, you can also explore the top trends from specific cities in Google’s Local Year in Search.
The section includes trends for each city including entertainment, “near me” searches, and relevant area-specific searches.
Even More
This year’s report includes even more cool trivia and interactive information, including Google Search Milestones, notable Facts About Google, and a statement from CEO Sundar Pichai’s view for Google’s future.
https://www.tulsamarketingonline.com/wp-content/uploads/2020/05/GoogleRisingProductCategories.png400800Taylor Ballhttps://www.tulsamarketingonline.com/wp-content/uploads/2018/07/TMO-Logo.pngTaylor Ball2023-12-12 20:57:552023-12-12 21:02:02Celebrate Google’s 25 Years In Search Trends Report and Time Capsule
Google is making some big changes to how it ranks results that aim to deliver more personalized search results and increase the prevalence of “first-hand knowledge”.
The search engine announced the changes earlier this month while spotlighting two specific updates that have recently come to users.
Cathy Edwards, Vice President of Search at Google, says these updates will better connect humans with the topics and content that are most relevant to their interests and needs:
“Search has always been about connecting human curiosity with the incredible expanse of human wisdom on the net. These advancements will help users find the most helpful information just for them, no matter how specific their questions may be.
Bringing First-Hand Knowledge To The Surface
Google has made adjustments to its ranking algorithm to show more first-person perspectives higher in search results. While the company didn’t tell us exactly how it tweaked the algorithm, Edwards emphasizes that it will help people find new individual experiences, advice, and opinions when searching.
With this change, the company says it will hopefully show fewer repetitive pieces of content that don’t bring new perspectives or opinions in the first pages of results.
The announcement says:
“As part of this work, we’ve also rolled out a series of ranking improvements to show more first-person perspectives in results, so it’s easier to find this content across Search.”
Follow Topics For More Curated Results
Google is giving you the ability to curate your own search results by following topics that are important to you.
By following topics in search results, such as a favorite football team, style of restaurant, or genre of music, you can stay in touch with these topics naturally while you are searching.
Follows not only impact what you see in typical search results but help highlight important topics in Discover and other areas of Google.
You can see an example of how this can shape your search results below. The first image shows what search results looked like before this update rolled out, and after.
Like most changes to the search results, however, it is unclear exactly how this affects optimization strategies going forward. We will know more as we get more data in the coming weeks.
Personalization Is The Future
Google has been increasingly customizing search results for users based on numerous factors including location, age, gender, demographics, and more. These latest updates continue this effort to ensure that the search results you see aren’t just the most relevant sites for anyone. They are the most relevant search results for you.
https://www.tulsamarketingonline.com/wp-content/uploads/2020/01/GoogleRankings.png360640Taylor Ballhttps://www.tulsamarketingonline.com/wp-content/uploads/2018/07/TMO-Logo.pngTaylor Ball2023-11-28 23:11:592023-11-28 23:12:01Google Updates Bring More Personalized Results and More First-Hand Knowledge
— Kristine (@schachin on Threads/Spoutible) 🇺🇦 (@schachin) November 10, 2023
The statement came during a live talk, where Sullivan reportedly told the crowd to “buckle up” because major changes were on the way.
As the public voice for Google’s Search team, Sullivan is uniquely positioned to speak on what the search engine’s developers are working on behind the scenes. For businesses, this means that he is one of the only people who can give advance notice about upcoming shifts to search results that could impact your online visibility and sales.
What Did Sullivan Say?
Since it wasn’t livestreamed or recorded, there’s been some discussion about exactly what Sullivan told the crowd. Posts on X agree on a few details though.
While attendees agree Sullivan specifically used the phrase “buckle up”, a few users provided longer versions of the quote that paint a slightly different picture.
One person, Andy Simpson, says the entire quote was “There’s so much coming that I don’t want to say to buckle up because that makes you freak out because if you’re doing good stuff, it’s not going to be an issue for you.”
His exact words were…"There's so much coming on and i don't want to say buckle up because that makes you freak out because if you're doing good stuff, it's not going to be an issue to you."
This is likely the case, as Sullivan has since clarified:
“I was talking about various things people have raised where they want to see our results improve, or where they think ‘sure, you fixed this but what about….’ And that these things all correspond to improvements we have in the works. That there’s so much coming that I don’t want to say buckle up, because those who are making good, people-first content should be fine. But that said, there’s a lot of improvements on the way.”
I was talking about various things people have raised where they want to see our results improve, or where they think "sure, you fixed this but what about…." And that these things all correspond to improvements we have in the works. That there's so much coming that I don't want…
Either way, it is important for businesses to take note of these statements and watch their site’s search results performance for any signs of major shifts in the near future.
https://www.tulsamarketingonline.com/wp-content/uploads/2021/05/Google-Page-Experience-Desktop1.jpg343720Taylor Ballhttps://www.tulsamarketingonline.com/wp-content/uploads/2018/07/TMO-Logo.pngTaylor Ball2023-11-14 19:28:082023-11-14 19:28:09Google Spokesperson Says To “Buckle Up” For Major Changes Coming To Search Results
Think using blogs to get to the top of the search engines is a thing of the past? Don’t be so quick to ditch your brand’s blog because a new study suggests that blog posts are the most common type of content found in the top 5 Google search results (excluding homepages).
Even with low-quality AI-generated blog content on the rise, BrightEdge says that blogs are the leading type of content returned by Google – a strong indication that blogs with well-crafted content are one of the strongest search engine optimization tools available to brands today.
About The Study
For the study, BrightEdge analyzed results for a dataset of 10,000 keywords of varying intent across 10 specific industries:
Banking
Insurance
Retail
Software
Higher Education
Real Estate
Advertising and Marketing
Manufacturing
Travel and Hospitality
Industrial
Using data collected during August of this year, the study then analyzed the content types of 23,785 pages ranking in the top 10 search positions.
While the leading type of page found in the top search results were homepages, these were excluded because these are essentially the default type of page Google returns when it believes a site may be relevant but it does not know which specific page to recommend.
Once homepages have been accounted for and excluded, the leading type of content in top search results was blog posts – accounting for 19% of the top 10 search results. When you narrow the focus to just the top 5 search results, that climbs to 23% of search results.
The Takeaway
Many brands have been moving away from traditional brand blogs because of a misguided notion that blogs were becoming irrelevant compared to more interactive or visual media like videos or user-generated content. This trend has only accelerated with the recent surge in lower-quality content pumped out by generative AI systems.
As Jim Yu, founder of BrightEdge and executive chairman says, however, well-maintained blogs are still an essential tool for raising the visibility of your brand and educating consumers:
“The future is not just AI – it’s AI and human symbiosis. AI can inform and assist, but human creativity, expertise and skill sets are necessary to add the voice and trust of your brand. Success lies in the fusion of AI and human expertise throughout any content creation process,”
Brands struggling to make progress in this area should likely re-evaluate their content and ensure their strategy is focused on delivering relevant, useful, and interesting information to your target market.
https://www.tulsamarketingonline.com/wp-content/uploads/2023/10/pexels-anete-lusina-47931541.jpg12821920Taylor Ballhttps://www.tulsamarketingonline.com/wp-content/uploads/2018/07/TMO-Logo.pngTaylor Ball2023-10-26 17:32:342023-10-26 17:32:36Google Puts Blogs In Top Search Results More Than Any Other Type Of Content
After months of rumors and speculation, Google’s AI-powered generative search experience is here – sort of.
The new conversational search tool is available to users as a Google Labs experiment only accessible by signing up for a waitlist. That means it is not replacing the current version of Google Search (at least, not yet), but it is the first public look at what is likely to be the biggest overhaul to Google Search in decades.
Though we at TMO have been unable to get our hands on the new search experience directly, we have gathered all the most important details from those who have to show you what to expect when the generative search experience becomes more widely available.
What The AI-Powered Google Generative Search Experience Looks Like
The new Google search experience is present at the very top of Google search results, giving context, answering basic questions, and providing a conversational way to refine your search for better results.
Notably, any AI-generated search information is currently tagged with a label that reads Generative AI is experimental.
Google will also subtly shade AI content based on specific searches to “reflect specific journey types and the query intent itself.” For example, the AI-created search results in the shopping-related search below are placed on a light blue background.
Where Does The Information Come From?
Unlike most current AI-powered tools, Google’s new search experience cites its sources.
Sources are mentioned and linked to, making it easier for users to keep digging.
Additionally, the AI tools can pull from Google’s existing search tools and data, such as Google Shopping product listings and more.
Conversational Search
The biggest change that comes with the new AI-powered search is the ability to follow up queries with follow-ups using context from your previous search. As the announcement explains:
“Context will be carried over from question to question, to help you more naturally continue your exploration. You’ll also find helpful jumping-off points to web content and a range of perspectives that you can dig into.”
What AI Won’t Answer
The AI-powered tool will not provide information for a range of topics that might be sensitive or where accuracy is particularly important For example, Google’s AI tools won’t give answers about giving medicine to a child because of the potential risks involved. Similarly, reports suggest the tool won’t answer questions about financial issues.
Additionally, Google’s AI-powered search will not discuss or provide information on topics that may be “potentially harmful, hateful, or explicit”.
To understand and rank websites in search results, Google is constantly using tools called crawlers to find and analyze new or recently updated web pages. What may surprise you is that the search engine actually uses three different types of crawlers depending on the situation with web pages. In fact, some of these crawlers may ignore the rules used to control how these crawlers interact with your site.
In the past week, those in the SEO world were surprised by the reveal that the search engine had begun using a new crawler called the GoogleOther crawler to relieve the strain on its main crawlers. Amidst this, I noticed some asking “Google has three different crawlers? I thought it was just Googlebot (the most well-known crawler which has been used by the search engine for over a decade).”
In reality, the company uses quite a few more than just one crawler and it would take a while to go into exactly what each one does as you can see from the list of them (from Search Engine Roundtable) below:
Googlebot: The first type of crawler is easily the most well-known and recognized. Googlebots are the tools used to index pages for the company’s main search results. This always observes the rules set out in robots.txt files.
Special-case Crawlers: In some cases, Google will create crawlers for very specific functions, such as AdsBot which assesses web page quality for those running ads on the platform. Depending on the situation, this may include ignoring the rules dictated in a robots.txt file.
User-triggered Fetchers: When a user does something that requires for the search engine to then verify information (when the Google Site Verifier is triggered by the site owner, for example), Google will use special robots dedicated to these tasks. Because this is initiated by the user to complete a specific process, these crawlers ignore robots.txt rules entirely.
Why This Matters
Understanding how Google analyzes and processes the web can allow you to optimize your site for the best performance better. Additionally, it is important to identify the crawlers used by Google and ensure they are blocked in analytics tools or they can appear as false visits or impressions.
https://www.tulsamarketingonline.com/wp-content/uploads/2022/05/Google-Inspecting-Links.png6751200Taylor Ballhttps://www.tulsamarketingonline.com/wp-content/uploads/2018/07/TMO-Logo.pngTaylor Ball2023-04-25 20:18:052023-04-25 20:18:07Explaining The 3 Types of Crawlers Google Uses To Understand The Web
