AI Search Optimization (AI SEO): What Google Actually Says You Need

Ishant

Ishant

Published : September 29, 2026 at 10:30 am

Updated : October 16, 2026 at 12:14 pm

AI search optimization cover with anonymous hands comparing search visibility and AI-assisted work cards beside an illustrative laptop display.

Key Observations

  • AI SEO means two different things and almost nobody says which one they are selling. It means optimizing so AI assistants cite you, and it means using AI tools to do SEO work. Of 23 ranking pages we read in full, zero lay the two meanings out side by side.
  • Google’s own guide has a section headed “Mythbusting generative AI search: what you don’t need to do”. Of the thirteen third-party pages we read on this search result, only one mirrors those myths at all, and none of the thirteen names the tension between them and its own recommendations.
  • Google states plainly: “Structured data isn’t required for generative AI search, and there’s no special schema.org markup you need to add.” Seven of the thirteen third-party pages we read recommend schema anyway.
  • Ahrefs tested 1,885 pages that added JSON-LD against about 4,000 controls and found plus 2.4 percent and plus 2.2 percent, both statistically indistinguishable from zero, and a statistically significant 4.6 percent decline on AI Overviews that it leaves unexplained. No uplift anywhere.
  • OpenAI documents four separate crawlers. OpenAI states that sites opted out of OAI-SearchBot will not be shown in ChatGPT search answers. GPTBot is the training crawler. Only one of the fourteen pages we read names a single actual user agent.
  • Google’s own documentation says you do not need llms.txt, in the words “You don’t need to create new machine readable files, AI text files, or markup to appear in these features.”
  • Pew Research tracked 900 panellists across 68,879 searches and found an 8 percent click rate when an AI summary was present against 15 percent without, with 1 percent clicking a link inside the summary.
  • None of the fourteen pages we read mentions that AI assistants strip the referrer, which is the reason your analytics cannot see this properly. Seven of them publish a measured result. None states an attribution model.
  • Eight of the fifty three autocomplete suggestions we pulled for these terms are people asking what this discipline is even called, and Google’s own People Also Ask block asks it back. Two of the fourteen pages answer it.
  • Do AI SEO tools improve rankings? The correlation has been measured twice and is weak. The causal question, whether using the tool raises your rankings, has never been publicly tested by anyone, vendors included. Surfer’s own study reports 0.28 across the top 100; Ahrefs measured within a single results page across 20 keywords and five tools and found weak correlations everywhere. A 100-position range produces correlation mechanically, which is why the two disagree.
  • The only genuine controlled test we found ran the same AI content rewrite for the same client in two markets. The US arm gained 12.6%. The Australian arm lost 3.3% with the confidence interval crossing zero. Same content, opposite results.
  • 2,000 unedited AI articles across 20 new domains produced 1,381 clicks in 16 months. That is 0.7 clicks per article. Everyone cites this study. Nobody divides clicks by articles.
  • Google’s spam policy names “scaled content abuse” and was updated on 28 August 2026. Of the 23 pages we read, zero name the policy and one quotes any spam policy at all. The method is not the violation; volume without purpose is.
  • You cannot optimize for RankBrain, BERT or MUM. Google has said MUM was never used for ranking, and told SEOs years ago to stop optimizing for RankBrain.

Table of Contents

  1. What is AI search optimization?
  2. What is AI search optimization also known as?
  3. What does “AI SEO” actually mean?
  4. What does Google say you do not need to do?
  5. Does schema markup help AI search?
  6. Does llms.txt do anything?
  7. Which AI crawlers actually decide whether you get cited?
  8. How do you check ChatGPT traffic in GA4?
  9. Why can’t you measure this properly?
  10. What is the difference between AI Overviews and AI Mode?
  11. Which claims about AI search have nothing behind them?
  12. How do you use AI to do SEO work?
  13. How do AI search optimization tools improve SERP rankings?
  14. How do you tell a real AI SEO tool from a wrapper?
  15. What happens if you use AI to publish at scale?
  16. Does machine learning in SEO mean anything?
  17. What should you actually do first?
  18. Why choose Hustle Marketers or Ishant Sharma for AI search optimization
  19. About the author
  20. AI search optimization FAQs

AI SEO, or AI search optimization, is the work of getting found by AI assistants rather than only by Google’s blue links. If you are trying to work out what it actually involves, you will find no shortage of pages telling you to add schema, publish llms.txt and earn mentions.

Google’s own documentation contradicts two of those three.

That is not a gotcha. It is the state of the subject. This page works through what the platforms themselves document, what the independent testing shows, and which widely repeated claims turn out to have nothing behind them. Where the honest answer is that nobody knows, it says so.

What is AI search optimization?

AI search optimization is the work of getting your pages used as a source when an AI system answers a question, rather than only ranking in a list of links.

That covers two different surfaces which behave differently, and the difference matters more than most pages admit.

Google’s AI Overviews and AI Mode are built on Google’s index. If Google cannot crawl and index the page, it cannot appear. Assistants like ChatGPT, Perplexity and Claude retrieve at the moment of asking, using their own crawlers and in some cases a third party search index, and can describe your brand without linking to you at all.

So the levers are not identical. For Google’s AI features, your existing SEO carries most of the weight. For assistants, crawler access and what other sites say about you carry more of it. We take the longer view in our piece on the future of SEO.

What is AI search optimization also known as?

This is the biggest gap between what people search and what pages answer anywhere in our data set.

Of the fifty three autocomplete suggestions we pulled across these search terms, eight are people asking what the discipline is called. On the “what is” phrasing specifically, six of the ten suggestions are naming questions. Google asks it back too: “What is AI search optimization called?” sits in the People Also Ask block on the head term. Two of the fourteen pages we read answer it.

The names in circulation:

NameWhat people usually mean by it
AI search optimizationThe general practice, both Google AI features and assistants
AEO, answer engine optimizationAimed at answer surfaces, snippets, People Also Ask, voice
GEO, generative engine optimizationAimed at what a model synthesises and recommends
AI SEOUsed loosely, sometimes meaning using AI tools to do SEO
LLMO, LLM optimizationOptimising for large language models specifically

They overlap heavily and most of the underlying work is shared. They are not identical, because AEO aims at extraction from an index while GEO aims at synthesis inside a model, and the two are measured differently.

If you want the distinction argued out properly, our generative engine optimization guide covers GEO on its own terms.

One warning worth carrying. Google’s John Mueller posted on Bluesky in August 2025: “The higher the urgency, and the stronger the push of new acronyms, the more likely they’re just making spam and scamming.” Anyone selling you three acronyms as three products is the case he is describing.

AI SEO proposal comparison separating optimizing for AI search from using AI to do SEO, with goals, work and evidence for each scope.
AI SEO can describe search visibility work or using AI to produce ordinary SEO work. Ask which job a proposal covers, what will be delivered and how it will be checked. If both are included, request separate scopes and prices.

What does “AI SEO” actually mean?

It means two completely different things, and the person selling it to you often will not say which. One is optimizing your site so AI assistants cite you. The other is using AI tools to do ordinary SEO work faster. They share a name, they cost different amounts, and they solve different problems.

We read 23 pages ranking for AI SEO and its close variants. Not one of them puts the two meanings side by side and says what each purchase actually buys. That omission is the single most useful thing missing from this topic, so it goes near the top of this page rather than buried in a glossary.

Meaning A: optimizing for AI searchMeaning B: using AI to do SEO
The problem it solvesChatGPT, AI Overviews and Perplexity do not mention youSEO work takes too long or costs too much
What you buyContent restructuring, crawler access, earned mentions, entity claritySoftware subscriptions, or an agency using them
Who sells itAgencies and consultantsTool vendors, mostly
What the evidence looks likePlatform documentation, a handful of studies, no disclosed algorithmTwo correlation studies that disagree, and no causal test at all
Typical failure modePaying for markup and files that platforms say they do not usePublishing at volume and losing traffic to a spam update
Also calledGEO, AEO, LLMO, AI visibilityAI content optimization, AI-assisted SEO

The confusion is not ours. It runs through the pages that currently rank. Search Engine Land’s guide states that AI SEO is not about using AI tools. Nightwatch’s page on the same term carries the heading “Using AI for Search Rankings”. Onely treats AI SEO as the umbrella term containing GEO, AEO and LLMO. Those three definitions are mutually incompatible and all three pages rank for the same query.

The search data shows buyers are just as split. Searches for how to use AI for SEO, artificial intelligence SEO and machine learning in SEO are asking about the toolset. Searches for AI search SEO and AI SEO strategy are asking about the outcome. Plenty of questions conflate both in one sentence.

Worth knowing. Before you accept any AI SEO proposal, ask one question: are you helping AI assistants find and cite me, or are you using AI software to produce my SEO work faster? If the answer covers both, ask for them to be scoped and priced separately. A proposal that cannot separate them has not thought about either.

The rest of this page covers meaning A first, because that is what most people arriving here want and it is where the platform documentation lives. Meaning B gets its own block further down, with the evidence on whether those tools do anything.

What does Google say you do not need to do?

Google publishes a guide to optimizing for generative AI features with a section headed “Mythbusting generative AI search: what you don’t need to do”. Of the thirteen third-party pages we read, only HubSpot’s mirrors these myths at all, and none of the thirteen names the tension between them and its own recommendations.

Two lines from it do most of the damage to the standard advice.

Structured data isn’t required for generative AI search, and there’s no special schema.org markup you need to add.

However, seeking inauthentic “mentions” across the web isn’t as helpful as it might seem.

Google’s separate page on AI features and your website goes further on the file question:

You don’t need to create new machine readable files, AI text files, or markup to appear in these features. There’s also no special schema.org structured data that you need to add.

Set that against what the pages on this search result recommend. Seven of the thirteen third-party pages recommend schema, and an eighth hedges it to “may help”. Seven of them tell you to earn mentions, and arguably nine, since two more mention it only in a line describing their own service. Only one page in either group, HubSpot’s, cites Google’s guidance at all, and even it does not name the tension with its own recommendations.

That does not automatically make them wrong. It does mean a page selling you schema for AI search owes you an explanation of why Google says otherwise, and none of them offers one.

So what does Google actually name?

Two lists, and both are short.

The affirmative one is unglamorous: allow crawling in robots.txt and at your hosting layer, use internal links so pages are discoverable, keep important content in text rather than behind JavaScript, support it with quality images and video where relevant, make any structured data match what is visible, and keep Merchant Center and Business Profile current.

The other list is the suppression controls:

To limit the information shown from your pages in Search, use nosnippet, data-nosnippet, max-snippet, or noindex controls.

If you have those applied anywhere by accident, you are suppressing yourself. That is a real audit item, and it is more concrete than most of what gets published on this topic.

The honest answer is no for pages that are already visible, and genuinely unknown for entities the system does not recognize.

Here is the evidence rather than the opinion.

Ahrefs tracked 1,885 pages that added JSON-LD against roughly 4,000 control pages. The measured differences were plus 2.4 percent on AI Mode and plus 2.2 percent on ChatGPT, both of which Ahrefs reports as statistically indistinguishable from zero, and minus 4.6 percent on AI Overviews, which it reports as statistically significant and leaves unexplained. No uplift anywhere.

Mark Williams-Cook ran a falsification test using deliberately invalid JSON-LD containing invented properties. ChatGPT and Perplexity both returned an address that appeared only inside that markup and nowhere in the visible content, and Perplexity stated it had read “the page’s embedded structured data”. Read honestly, that cuts both ways. These systems do parse JSON-LD, so a blanket claim that they ignore it is wrong. They also parse markup no validator would pass, so validity is not the lever it gets sold as.

One thing none of this means. Schema still earns rich results in classic search, which is separately and well evidenced, so this is not an argument for stripping it out. Our guides to Magento schema markup and restaurant schema markup cover the implementation where it does pay.

There is a fair counter-argument, and it is the strongest thing schema’s defenders have. Ahrefs tested pages that already had at least a hundred AI Overview citations. Schema’s job is disambiguating entities a system struggles to identify. Nobody has tested that case. So the correct answer for an unknown brand is that we do not know.

What we can say is that the most repeated justification for schema in AI search does not survive checking. The claim that “Microsoft confirmed schema markup helps their LLMs” traces to one person’s paraphrase of a spoken remark at a conference in March 2025, reported second hand. No Microsoft document says it. The only substantive quote directly attributed to the Microsoft speaker in that coverage concerns freshness and IndexNow.

There is a sharper illustration. Search “does schema help ai search” and Google’s own AI Overview answers yes, and asserts that Google and Microsoft have confirmed structured data gives crawlers an advantage. It cites a trade publication. Google’s own documentation, updated in July 2026, says the opposite.

Does llms.txt do anything?

On the current evidence, almost nothing that sends you traffic reads it.

Google’s John Mueller wrote on Bluesky on 17 June 2025: “FWIW no AI system currently uses llms.txt”, and in June 2026 he called the idea “purely speculative”.

Google’s documentation tells you that you do not need to create “new machine readable files, AI text files, or markup”.

The operator of around 80,000 blogs reported from server logs that nothing requests the file, while ordinary pages are aggressively scraped.

A separate study by an AEO vendor that itself generates llms.txt logged server requests across 83 sites over twelve weeks to 19 July 2026. OpenAI’s crawler fetched llms.txt 7 times against 3,990 robots.txt fetches. Anthropic fetched it 9 times. Perplexity never fetched it at all. The one exception is Meta, whose crawler fetched llms.txt 193 times, more often than it fetched robots.txt. So “nothing reads it” is too strong. The accurate version is that the crawlers which decide whether you appear in an answer are not asking for it.

Two of the four positive data points in our set are an employee of a company selling documentation tooling and an unquantified brand claim. Not one of the four carries a number.

There is a live contradiction worth knowing about, because it is where the confusion comes from. Chrome’s Lighthouse now audits for llms.txt while Google Search says you do not need it. Google is contradicting Google across two products, and vendors quote whichever half suits them.

Our position: publishing the file costs almost nothing, so do it if you like, but anyone charging you for it or presenting it as a ranking factor is ahead of the evidence.

Which AI crawlers actually decide whether you get cited?

This is the most actionable section on this page, and only one of the fourteen pages we read names a single actual user agent.

OpenAI documents four separate crawlers, and they do different jobs:

CrawlerWhat OpenAI documents it for
OAI-SearchBot“used to surface websites in search results in ChatGPT’s search features”
GPTBot“used to crawl content that may be used in training our generative AI foundation models”
ChatGPT-UserUser initiated actions
OAI-AdsBotAd safety validation

The last two descriptions are our summary of OpenAI’s page, not quotations. The line that matters most is a quotation:

Sites that are opted out of OAI-SearchBot will not be shown in ChatGPT search answers, though can still appear as navigational links.

Read that carefully. If you blocked OpenAI because you did not want your content training a model, and you blocked the wrong bot, you removed yourself from ChatGPT’s search answers. Those are separate decisions and most coverage of “should I block GPTBot” conflates them. The wider version of this work is in our guide on how to improve brand visibility in AI search engines.

Anthropic likewise runs Claude-SearchBot separately from ClaudeBot. Perplexity asks you to allow PerplexityBot, and our Perplexity SEO guide covers the rest of that setup. If you want the ChatGPT specific version of this work rather than the general case, our guide to ranking in ChatGPT search goes through it step by step.

One dataset worth citing carefully, from Cloro, which sells AI citation tracking APIs. Across 1,058 heavily cited domains, those blocking PerplexityBot showed a median Perplexity citation rate of zero against 1.167 for those allowing it, where the rate is citations per Google organic appearance over a two day window. Cloro states the data cannot prove causation and that blocking domains may differ in other ways. No controlled block and unblock experiment exists anywhere that we could find.

Cyrus Shepard, who founded the SEO software company Zyppy, cross-referenced 54 published experiments, studies, explainers and patents and scored URL accessibility top of his list at 9.5, the highest score on his scale. He states plainly that these are correlations and not ranking factors. The reason to act on it anyway is not his score. It is OpenAI’s own sentence two paragraphs up.

How do you check ChatGPT traffic in GA4?

People search for this and none of the fourteen pages we read answers it. Here is the practical version, and then the reason it will disappoint you.

In GA4, look at Reports, then Acquisition, then Traffic acquisition, and add a filter on session source containing chatgpt.com, perplexity.ai, copilot.microsoft.com or gemini.google.com. Build it as an exploration if you want it saved.

What you will see is a fraction of the truth, and you should know why before you present it to anyone or fold it into your automated SEO reports.

Why can’t you measure this properly?

Because the referrer is stripped, and this is the single biggest hole in the subject.

When someone reads about you inside an assistant and then visits you, the visit frequently arrives with no referrer and lands in your analytics as direct traffic. When they read about you and do not visit at all, there is nothing to see.

None of the fourteen pages we read mentions this. Seven of them publish a measured result. None of the seven states an attribution model. Three disclose no method at all. One page’s headline claims a 994 percent rise in referral traffic from language models, on a search result where no page explains that referral traffic is the thing these systems do not reliably send.

The independent evidence on whether AI answers even produce clicks is not encouraging. Pew Research tracked 900 US adults across 68,879 unique Google searches in March 2025. Users who saw an AI summary clicked a traditional search result in 8 percent of visits, against 15 percent for those who did not, and just 1 percent clicked a link inside the summary itself. That study is about Google’s AI summaries specifically, not assistants in general, and it is the strongest independent quantitative source in our whole research set.

Cloudflare’s crawl to refer ratios, measured over 19 to 26 June 2025, put Anthropic at 70,900 crawls per referral. Cloudflare itself concedes that native app traffic carries no referrer and that this may overstate the ratios by an amount it cannot determine. The organization with the best data on this says it cannot quantify the gap.

So what can you honestly measure? Inclusion, not traffic. Agree a fixed set of prompts a buyer in your category would actually type, run them on a schedule, and record whether you are cited, mentioned, recommended, misrepresented or missing. That is repeatable and it is checkable. Our guide to tracking brand visibility in AI search covers how to run it, and citations versus mentions covers why the difference changes what you should count.

What is the difference between AI Overviews and AI Mode?

Google’s autocomplete suggests this question. None of the fourteen pages we read answers it, Google’s own guide included, which names both surfaces and never distinguishes them.

What can be said from the documentation and from observation:

AI Overviews appear above or within a conventional results page, alongside links, for a subset of queries. AI Mode is a separate conversational surface you enter deliberately, where the answer is the interface and links are secondary. If AI Overviews are the surface you care about, we cover how to optimize content for Google AI Overviews separately.

For your purposes the practical difference is exposure. An AI Overview competes with your own listing on the same page. AI Mode can replace the page entirely. Our Judaica client’s case study documents being listed first under Megastores in an AI Mode answer, which is a placement with no equivalent in classic results.

We are not going to publish a confident mechanical distinction between how the two select sources, because Google has not published one and we have not tested it at a scale that would support a claim.

Which claims about AI search have nothing behind them?

This section exists because our research kept running into figures that everyone repeats and nobody can trace, plus one that is easy to trace and has no evidence attached to it. Five worth knowing:

Claim in circulationWhat we found when we traced it
Microsoft confirmed schema markup helps their LLMsA paraphrase of a spoken conference remark, reported second hand. No Microsoft document.
Google confirmed AI Overviews send higher quality clicksTraceable to Liz Reid, VP and Head of Google Search, on 6 August 2025, but with no data, method, sample or period published, in a post that dismisses outside research as flawed. A position, not a finding.
80 percent of AI cited URLs do not rank in Google’s top 100Found only in a vendor’s forum comment
AI crawlers do not execute JavaScriptNone of OpenAI’s, Anthropic’s or Perplexity’s documentation addresses rendering either way
Around 70 percent click through dropOne unsourced comment. Roughly double the 34.5 percent that Ahrefs actually measured.

Two vendor datasets also sit on this same search result contradicting each other without comment. One reports AI Overviews on 25 percent of searches from a sample of 21.9 million queries. Another reports about 16 percent after a peak near 25 percent. Both rank. Neither mentions the other.

Two pages here prescribe conflicting ideal answer lengths, 40 to 60 words and one to two sentences. Google’s own AI Overview for a neighbouring query prescribes a third, 40 to 70 words. None of the three cites a test.

How do you use AI to do SEO work?

Use it for the parts of SEO that are language work, and keep it away from the parts that are data work. AI is good at drafting, restructuring, summarizing, clustering and first-pass analysis. It is bad at anything requiring a real number, because it will produce a confident one that is wrong.

That split is not a style preference. It has been tested. WordStream ran 50 SEO questions through five models and found Gemini reporting 90,000 monthly searches for a keyword whose actual volume is 1,500, an error of sixty times. An Ahrefs analyst puts it more bluntly, writing that AI keyword metrics are not metrics at all, they are guesses.

Use AI for thisDo not use AI for this
First drafts you will edit heavilyKeyword volume, difficulty or CPC figures
Clustering a keyword export into themesDeciding which keywords are worth targeting, without checking the data
Rewriting a section to answer a question directlyWriting anything you will publish unedited
Summarizing competitor pages you have already fetchedTelling you what competitors rank for
Drafting meta titles and descriptions at volumeAny statistic, date, price or citation in published copy
Turning a messy audit export into a prioritized listDeciding priority without you checking it
Writing the regex, the script or the spreadsheet formulaRunning the analysis and trusting the output unchecked

The rule underneath the table: AI can operate on data you have verified, and it cannot be the source of the data. Every failure we have seen in our own work and in the published accounts comes from crossing that line.

How do AI search optimization tools improve SERP rankings?

Nobody has shown that they do. The association between a tool’s content score and where a page ranks has been measured twice, and it is weak. The question that actually matters, whether using the tool causes your rankings to rise, has never been publicly tested by anyone, including the vendors who sell the tools.

That is a strong claim, so here is the evidence behind it.

StudyMethodFindingWhat it does not show
Ahrefs content score study20 keywords, five tools including Surfer, Frase, NeuronWriter and Clearscope, measured within a single results pageWeak correlations everywhereWhether raising a score changes anything
Surfer’s own study10,000 queries, one million results entries, measured across the top 1000.28 correlation between its Content Score and rankingSame. Also published by the vendor, with no limitations section

Those two look contradictory and they are not. Ahrefs measured within one results page, where every page is already competitive and the range is narrow. Surfer measured across the top 100, where position 3 and position 97 sit in the same dataset. A wider range produces correlation mechanically. Neither study states this, and it is the reason the numbers differ.

The closest thing to a causal test we found is a practitioner who ran one of these tools hands-on and then declined to accept a single automated change, precisely because he could not isolate the effect. In that test the tool’s Auto-Optimize raised an article’s score by 34 points in under a minute and invented five claims that appear nowhere in the source article.

There is one real controlled test of AI content in the wild, and it is worth the space because of what it found. SearchPilot ran the same AI content rewrite for the same travel client in two markets as a proper control versus variant test. The US arm gained 12.6%. The Australian arm lost 3.3%, with the confidence interval crossing zero. Same content, same client, same method, opposite outcomes. A separate SearchPilot test adding AI content to sparse listing pages returned 18% more organic sessions.

What to take from this: content scores are a proxy for looking like the pages that already rank. That is not worthless, because it catches genuine omissions. It is also not a ranking lever, and it is one of the plainest SEO automation limits: a tool that reports your score went from 42 to 76 has told you about the tool, not about Google.

If you are paying for an AI SEO tool or an AI SEO retainer, send us what you are being charged for. We will tell you which of the two meanings you are buying, and which parts of it have evidence behind them.

Get an honest read on your AI SEO spend

How do you tell a real AI SEO tool from a wrapper?

Ask where the data comes from. A real tool has its own index, its own crawl, or a licensed data source, and it will tell you which. A wrapper passes your prompt to someone else’s model and formats the reply. Both can be useful. They should not cost the same.

The listicles ranking for AI SEO tools name more than forty products between them, one page alone naming twenty-three. Not one of the 23 pages we read tells you how to tell them apart. Five questions that do:

  1. Where does your ranking or volume data come from? A real answer names an index or a licensed provider. A wrapper will say the model estimates it, which means the numbers are generated rather than measured.
  2. What happens if I ask for a keyword with no search volume? A tool with real data returns nothing or zero. A wrapper invents a plausible number.
  3. Can I see the same figure in a second tool? If the number exists nowhere else, it was generated.
  4. Does the output change when I run the same query twice? Real data is stable. Model output drifts.
  5. What does the tool refuse to answer? A well-built product has boundaries and says so. A wrapper answers everything, confidently, including things it cannot know.

What happens if you use AI to publish at scale?

Usually very little traffic, and sometimes a collapse. Google’s spam policy names this directly as scaled content abuse, and the published outcomes from people who have tried it at volume are poor enough to settle the question on economics alone, before you get to the policy risk.

Start with the economics. SE Ranking published 2,000 unedited AI articles across 20 new domains and tracked them for 16 months. The total was 1,381 clicks. That is 0.7 clicks per article across the whole experiment. This study gets cited constantly and we have not seen anyone divide the clicks by the articles.

Then the controlled comparison. Reboot Online invented a keyword that existed nowhere, built ten new domains, and had five write content with AI and five with humans. Human-written pages averaged position 4.4 against 6.6 for AI, and human won 21 of 25 head-to-head pairs.

Then the downside. Lily Ray tracked more than 220 sites publishing AI content and reports that 54% lost at least 30% of peak traffic, 39% lost at least 50%, and 22% lost at least 75%, typically on a twelve to eighteen month curve. Glenn Gabe’s August 2026 spam update case studies include sites losing more than 200,000 queries.

Does Google penalize AI content, then?

Not for being AI-written. For being scaled and unhelpful, which is a different thing, and the distinction is where most advice goes wrong.

Google’s spam policies page, updated 28 August 2026, names scaled content abuse as generating many pages primarily to manipulate rankings rather than help people, whether automated, human or a combination. The method is not the violation. The purpose and the volume are.

Set against that, Semrush found AI and human content roughly level in live top-ten results, around 57% against 58%. That looks like it contradicts the failure reports, and it does not. Semrush sampled pages already ranking, which means established domains with editing behind them. SE Ranking and Reboot both used brand-new domains with no authority. Every result in this section fits one explanation: edited AI content on a trusted domain performs fine, and unedited AI content on a new domain does not.

One caveat we will state because the studies mostly do not. Several of these figures depend on AI-detection tools to classify content, and detector accuracy is contested. Treat any precise percentage of AI content in a dataset as directional rather than exact.

Worth knowing. If traffic falls and Search Console shows no manual action, that is the normal pattern for algorithmic suppression, not evidence that nothing is wrong. The report you would naturally check is the one that stays silent.

Does machine learning in SEO mean anything?

It means Google uses machine learning. It does not mean you can optimize for it. Those two get collapsed into one idea constantly, and the second half is not a real activity.

Google has been unusually direct about this. On MUM, Google has stated it was never used for ranking, and described its live use as limited to things like COVID vaccine naming and related topics in video results. On RankBrain, Google’s John Mueller told SEOs to stop optimizing for it years ago.

So when a page or a proposal offers to optimize your content for RankBrain, BERT or MUM, it is describing something the named systems either do not do or that Google has explicitly said not to chase. Of the 23 pages we read, one names these systems at all.

What is left once you remove the myth is ordinary and useful. Google’s systems understand language rather than matching strings, which is an argument for writing plainly about a subject and naming the things in it, and an argument against keyword density. That is the whole practical content of machine learning in SEO. We cover the identity side of it in our guide to entity SEO.

What should you actually do first?

In this order, because each step depends on the one before.

  1. Check you are fetchable by the retrieval bots, not just the training ones. OAI-SearchBot, Claude-SearchBot, PerplexityBot and Googlebot are the agents tied to whether you can be used in an answer. GPTBot, ClaudeBot and Google-Extended are training and grounding controls, and blocking those is a separate decision. Check robots.txt line by line.
  2. Audit for nosnippet, data-nosnippet and max-snippet. These are Google’s own named controls and an accidental one suppresses you.
  3. Confirm your content is in the HTML, not only after JavaScript runs. Nobody documents whether AI crawlers render, so do not rely on it.
  4. Put the answer to each real question in one extractable passage near the top, with the claim, the number and the source together.
  5. Fix your classic SEO. AI Overviews are built on the index, so this is not optional.
  6. Agree a prompt set and baseline it before you change anything else. Without a baseline you cannot tell whether any of this worked.
  7. Only then consider schema and llms.txt, knowing the evidence for both is weak and Google says neither is required.

Notice what is not on that list. Nothing about volume of content, nothing about publishing frequency, nothing about acronyms.

Why choose Hustle Marketers or Ishant Sharma for AI search optimization

Ishant Sharma has worked in Google Ads, Microsoft Ads and SEO since 2013, and Hustle Marketers runs paid search, paid social and ecommerce SEO with the results published rather than described.

What that means against the specific problems on this page:

  • We publish placements you can open and check. A New York Judaica store was listed first under Megastores in a Google AI Mode answer for best judaica store in new york in September 2026, in a case study that also reports organic clicks up 31 percent and 10.59K free listing clicks, with the data source and measurement window named for every figure.
  • It is not one result. We also publish an SEO and AI visibility growth case study and an ecommerce store’s SEO and AEO growth, both with the reporting window named.
  • We tell you what we cannot attribute. Referrer stripping is real and we will not present direct traffic as proof of an AI citation.
  • We will not sell you llms.txt or schema as an AI ranking factor, because the evidence does not support it and Google says neither is required.
  • We baseline a prompt set before changing anything, so there is something to compare against.
  • We separate branded from non branded, because growth that is only people typing your name is not new demand.

If you are choosing a provider rather than doing this yourself, our page on what to look for in an AI SEO agency sets out the questions worth asking, and we separately checked what five AI SEO agency rankings actually show. For the ecommerce version of this work, see AI SEO for ecommerce and ecommerce SEO.

About the author

This page was researched and written by Ishant Sharma, founder of Hustle Marketers, who has more than twelve years of experience in Google Ads, Meta Ads, SEO and ecommerce PPC. Every platform document quoted here, Google’s two AI guidance pages and OpenAI’s crawler documentation, was read at source. Where a Google employee’s remark reached us through trade coverage, the publication and the date are named in the text. If something here is wrong, tell us and we will correct the page rather than leave it.

AI search optimization FAQs

Is AI search optimization different from SEO?

Partly. AI Overviews and AI Mode are built on Google’s index, so your classic SEO carries most of the weight there. Assistants retrieve with their own crawlers and can describe you without linking, so crawler access and what other sites say about you matter more. Most of the underlying work is shared.

Google says no, in these words: “Structured data isn’t required for generative AI search, and there’s no special schema.org markup you need to add.” Ahrefs tested 1,885 pages adding JSON-LD and measured no uplift on any platform. It may still help a brand the systems cannot identify, but nobody has tested that.

Should I publish llms.txt?

It costs little, so publish it if you want. But Google’s John Mueller said in June 2025 that no AI system uses it, Google’s documentation says you do not need new AI text files, and an operator of around 80,000 blogs reports that nothing requests it. In one vendor study of 83 sites, OpenAI’s crawler fetched it 7 times in twelve weeks. Treat anyone charging for it as ahead of the evidence. The same test is worth applying to a whole scope of work, which is why our buyer guide to choosing a generative engine optimization agency asks which engine each line item is supposed to move.

Will blocking GPTBot remove me from ChatGPT?

Not by itself. GPTBot is the training crawler. OAI-SearchBot is the one tied to ChatGPT’s search answers, and OpenAI states that sites opted out of it “will not be shown in ChatGPT search answers, though can still appear as navigational links”. Check which one you blocked.

How do I know if AI search is sending me traffic?

Largely you do not, and any provider who tells you otherwise is overstating. Assistants frequently strip the referrer, so visits arrive as direct. Measure inclusion against a fixed prompt set instead, and treat traffic as a secondary signal.

Does being cited in an AI answer actually bring visitors?

Less often than the category implies, at least on Google. Pew Research tracked 900 US adults across 68,879 unique Google searches in March 2025. Those who saw an AI summary clicked a traditional search result in 8 percent of visits against 15 percent for those who did not, and 1 percent clicked a link inside the summary.

How long does AI search optimization take?

We are not going to give you a number, because we have not run enough engagements at enough sizes to publish an honest one, and citations move when models update. Agree a re-test cadence in writing instead so you can see whether a result held.

Can anyone guarantee an AI citation?

No. Search Engine Roundtable reported in January 2026 that Google’s Danny Sullivan, speaking about tools claiming to write content better, said claims are simply claims and no one can guarantee anything. Nobody controls whether an assistant names you.

What is AI SEO?

The term covers two different jobs. One is optimizing your site so AI assistants such as ChatGPT, Google AI Overviews and Perplexity cite you, which is also called GEO, AEO or AI visibility. The other is using AI software to do ordinary SEO work faster, such as drafting content or clustering keywords. They cost differently, they are sold by different people and they solve different problems, so establish which one is on the table before you buy anything.

Do AI SEO tools actually improve rankings?

Nobody has shown that they do. The link between a tool’s content score and ranking position has been measured twice, by Ahrefs within a single results page and by Surfer across the top 100, and the results are weak and disagree with each other for a methodological reason neither states. The causal question, whether using the tool raises your rankings, has never been publicly tested by anyone, including the vendors. Content scores are a proxy for resembling pages that already rank, which is useful for catching omissions and is not a ranking lever.

Can AI-written content rank on Google?

Edited AI content on an established site can rank, and unedited AI content on a new site generally does not. Semrush found AI and human content roughly level in live top-ten results, but that sample was drawn from pages already ranking on trusted domains. On new domains the picture reverses: 2,000 unedited AI articles produced 1,381 clicks in 16 months, and a controlled test on an invented keyword had human-written pages beating AI-written ones 21 times out of 25. The variable that reconciles those results is domain authority plus human editing, not the writing method.

Will Google penalize me for using AI?

Not for using AI. Google’s spam policies, updated 28 August 2026, name scaled content abuse as producing many pages primarily to manipulate rankings rather than to help people, and explicitly say this applies whether the pages are made by automation, by humans or by both. The method is not the violation. Volume plus lack of purpose is. One practical note: algorithmic suppression does not produce a manual action, so an empty Manual Actions report is not evidence that nothing is wrong.

Can you optimize for RankBrain, BERT or MUM?

No. Google has stated MUM was never used for ranking, and Google’s John Mueller told SEOs to stop optimizing for RankBrain years ago. Anyone offering to optimize your content for a named Google machine learning system is selling something the system either does not do or that Google has said not to chase. What survives is ordinary and useful: write plainly about a subject and name the things in it, because these systems read language rather than match strings.

How do you use AI for SEO without getting into trouble?

Keep it on language work and off data work, and edit everything before it publishes. Use it to draft, restructure, cluster and summarize material you have already verified. Never take a search volume, difficulty score, statistic, date, price or citation from a model, because it will produce a confident wrong one: in one test a model reported 90,000 monthly searches for a keyword whose real volume was 1,500. And publish at a volume a human can actually review, because the failure cases all involve scale without editing.

We will look at your site and tell you which of the two AI SEO problems you actually have, and whether the work is worth buying. If we think your constraint is somewhere else, we will say so.

Ask us what your site actually needs

Sources and how we researched this

Competitor pages. Fourteen pages fetched and analyzed across six informational search terms for this topic, with 54 result slots classified. Each page was assessed against the same fixed set of ten questions plus its complete heading list, not a paragraph by paragraph read, so a tactic discussed only in body prose could have been missed and the counts here should be read as close estimates rather than exact. Where a count concerns what practitioners recommend, we give it out of the thirteen third-party pages and exclude Google’s own documentation, which is one of the fourteen. Of the six terms, the first ranked result was read for four. One could not be fetched, because the response returned only page metadata, and one was skipped as a near duplicate of a page already in the set. No coverage was inferred for either.

Question phrasings. People Also Ask was read from live US Google result pages in a signed in browser session, which can personalise results, on 25 September 2026. Only the initial unexpanded four question block was captured for each term, and two of the six terms showed no block at all. Autocomplete was collected the same way. On the head term, one of the four People Also Ask questions is “What is AI search optimization called?”, which is the naming question, asked back by Google itself.

Platform documentation. Google’s guide to optimizing for generative AI features, Google’s page on AI features and your website, and OpenAI’s crawler documentation were each read at source and quoted directly rather than through secondary coverage.

Independent and vendor research. Pew Research is the strongest independent quantitative source in our set. The only other non-vendor quantitative work we found is an unreviewed preprint by Sriram Selvam and Anneswa Ghosh, which found that moving a page up within a matched pair bought only 7.9 points of citation probability, and that 15 percent of citation decisions flipped on a rerun with everything else held constant. Every other quantitative figure here comes from a company selling something adjacent to this market, including Ahrefs, Cloudflare, Cloro, Zyppy and the llms.txt study, and we name them so you can weigh them.

What we could not reach, stated plainly because it changes what this page can claim. Reddit is blocked at the network level here, and it is where this subject is argued most actively, so that gap stands. LinkedIn and X are robots blocked. We obtained no video transcripts, only descriptions and chapter titles. Our citation source lists are incomplete because we did not expand every list.

What we did not do. We did not publish a timeline or a price, because we cannot support either honestly. We did not test whether schema helps an unknown entity, which is the one schema question that remains genuinely open.

Ishant

Ishant Sharma is the Founder and CEO of Hustle Marketers, a Google Partner digital marketing agency. With 12+ years of experience in Google Ads, Meta Ads, SEO, and e-commerce PPC, he has helped 2,500+ brands generate $780M+ in trackable sales. Upwork Top Rated Plus with 100% Job Success Score. Ishant Sharma is the digital marketing specialist, not the Indian cricketer of the same name.

I hope you enjoy reading this blog post. If you want my team to just do your marketing for you, click here.
Scroll to Top