Technical SEO Checklist 2027: 105 Checks for AI Search

Ishant

Ishant

Published on November 12, 2026

Technical SEO checklist showing crawlability, indexation, Core Web Vitals and schema from Hustle Marketers

Summarize this article with:

Technical SEO checklist showing crawlability, indexation, Core Web Vitals and schema from Hustle Marketers
A practical technical SEO checklist for helping search engines crawl, index and understand a website.

Most technical SEO checklists still audit for one reader. That reader is Googlebot. But your pages now get judged by a second group as well, and that group behaves nothing like a search crawler. For example, GPTBot, OAI-SearchBot, PerplexityBot and ClaudeBot arrive with less patience, weaker rendering and different goals. Some of them never arrive at all, yet still quote you. This checklist covers all 105 checks we run on client sites, split across 18 categories. Every item tells you why it matters and how to check it, so you can hand the list straight to a developer. For the strategy sitting on top of these fixes, see what Google documents about AI search and what it stays silent on.

The shift nobody has priced in yet is this: AI systems are not simply crawlers you allow or block, they are users who have to read your page, render it, and increasingly act on it.

Four layers of search visibility showing how a webpage moves from index to snippet, render and AI agent interaction
A page must pass through index, snippet, render and agent layers before it can be fully discovered, understood and used.

Updated for 2027: we rechecked this list against Google and Microsoft documentation changes published through October 2026. The count is still 105 checks in 18 categories. What moved is in the new section on what changed in technical SEO for 2027, including the new Search Console control and report for AI Overviews and AI Mode. This edition also corrects how Google treats 404 and 410 responses and what blocking ClaudeBot does, and it adds hreflang checks, platform notes and a symptom-to-check troubleshooting table.

Key Takeaways

  • 105 checks, 18 categories, 4 priority levels. Every item states why it matters and exactly how to check it, and the first 75 are infrastructure checks while the last 30 cover keyword, content and link diagnostics.
  • Fix these four first: the four-variant redirect test on a deep URL, the filetype: search for exposed documents, each AI user agent line in robots.txt, and your crawled-but-not-indexed ratio. Under an hour combined, so start there.
  • Ten things here appear in no other published technical SEO checklist we audited, including the log file method for finding rendering gaps, accessibility tree testing, agent commerce readiness, and why you get cited by bots that never visit you.
  • llms.txt is not robots.txt for AI. Three widely-shared checklists get this wrong, while Google now says Google Search ignores the file. Add it if you want, but do not count it as strategy.
  • Do not noindex thin pages. Noindex still leaves them crawled and still counted in sitewide quality. Instead, remove them or improve them.
  • Short on time? The 50-item version for small sites is near the end, and it still covers roughly 90% of the risk.

What Is a Technical SEO Checklist and Who Needs One

A technical SEO checklist is a repeatable list of infrastructure checks that confirm search engines and AI systems can reach, read, render and trust your website. It sits underneath content and links. When it breaks, nothing above it works.

You need one in four situations. First, before a migration or replatform. Second, when organic traffic drops without an obvious content cause. Third, when you inherit a site and have no history on it. Fourth, on a fixed cadence, because technical debt accumulates quietly as a site grows.

In general, run the full 105 checks quarterly on a large site. On a small brochure site, however, run the 50-item short version twice a year. Then use SEO automation tools to monitor the critical items monthly.

How This Checklist Is Prioritised

Every check carries one of four labels. So treat them as a work order, not a suggestion.

Technical SEO audit priority flow from crawling and indexing to rendering, Core Web Vitals and schema
Fix search-engine access first, then improve rendering, experience and structured data.
PriorityWhat it meansFix window
CriticalBlocks crawling, indexing or revenue right nowSame week
HighCosts rankings or crawl efficiency at scaleWithin 30 days
MediumCompounding drag, but cheap to fix in batchesWithin the quarter
LowHousekeeping or speculative upsideWhen capacity allows

Technical SEO vs On-Page SEO: Where Does Each Check Belong?

Technical SEO covers whether search engines and AI systems can reach, render, index and trust your pages, while on-page SEO covers what each page says and targets. In this checklist, checks 1 to 75 are technical infrastructure, and checks 76 to 105 are the keyword, content and link diagnostics that sit on top of it.

We keep all 105 in one list because an audit fails when two teams run the halves and never compare notes. For example, a page can pass every crawl check and still miss, simply because it targets the wrong intent. So the table below shows where each group sits and who usually fixes it.

LayerCategoriesChecksUsually fixed by
Access and crawlingSetup and access, crawlability, indexation, robots and directives, duplicate content and canonicals, site architecture1 to 39 (39 checks)SEO lead and developer
Rendering, speed and securityRendering and JavaScript, speed and Core Web Vitals, server and security, mobile40 to 50 (11 checks)Developer and hosting team
Templates and markupOn-page technical, structured data51 to 61 (11 checks)Developer and SEO lead
AI and agent layerLogs and monitoring, AI search, agent readability62 to 75 (14 checks)SEO lead and developer
Beyond technicalKeyword and intent, content quality, off-page76 to 105 (30 checks)Content and outreach teams

That split also explains the count: the title says 105 because every numbered row on this page is a check you can pass or fail, and 75 of them are strictly technical. If you only need the infrastructure audit, stop after check 75. However, when a page is indexed but still not winning, the last 30 usually tell you why. For a shorter view of what Google itself requires, see our page on technical SEO requirements.

What Changed in Technical SEO for 2027?

Three things changed how this checklist runs going into 2027. Google added a site-wide Search Console control and a performance report for AI Overviews and AI Mode, Google published guidance that rules out several popular AI tactics, and Microsoft added Copilot citation reporting to Bing Webmaster Tools. However, none of them adds a new crawl or index requirement. Instead, they change how you run checks you already have.

Here are the dated changes that matter most, as well as the checks on this list each one touches. After that, a second table lists the smaller documentation changes.

DateWhat changedChecks affectedWhat to do
February 10, 2026Microsoft launched the AI Performance report in Bing Webmaster Tools (public preview): Total Citations, Average Cited Pages, Grounding Queries and page-level citation activity across Copilot, Bing AI summaries and select partners2, 16, 71Claim the site in Bing Webmaster Tools and record a citation baseline before you change anything
May 7, 2026FAQ rich results stopped appearing in Google Search. Google then removed the documentation on June 15, 202656, 61So stop counting FAQPage as a deliverable, as the structured data section explains
May 15, 2026Google published its guide to optimizing for generative AI features. It also says you do not need AI text files, special markup or Markdown, and you do not need to break content into tiny pieces66, 72Keep llms.txt at Low priority, and also do not split pages into fragments for AI

What Changed for Spam, AI Controls and Reporting?

DateWhat changedChecks affectedWhat to do
May 15, 2026Google clarified that its spam policies also apply to generative AI responses in Search89, 95Treat content made only to steer AI answers as a spam risk rather than a shortcut
June 3, 2026 (all websites by August 31, 2026)Google announced a Search generative AI control in Search Console that includes or excludes a site from AI Overviews, AI Mode and generative AI features in Discover, plus a Generative AI performance report that shows impressions by page, country, date and device25, 67When you check AI access in robots.txt, confirm the control is set to include, then use the report to see which pages appear in AI answers
October 26, 2026Google updated its guidance on preventing user-generated spam, such as blocking indexing of link-shared content and monitoring abuse in Search Console8, 10Also add user-generated and link-shared URL patterns to your spam URL review
Current definitionGA4’s default channel group includes an AI Assistant channel for referrals from sources such as ChatGPT, Gemini, Deepseek, Copilot and Grok. However, it does not include Google’s AI Overviews or AI Mode3Confirm the AI Assistant channel shows up in your acquisition reports, but do not expect Google AI features there

The full list of Google’s dated changes is on the Search Central documentation updates page, which is worth a monthly look if you run audits for a living.

Which of These Changes Should You Add to Your Audit First?

The June 3 row is the one to add to your audit today. After all, a site-wide switch that removes you from AI Overviews and AI Mode is the kind of setting one person changes once and everyone forgets. Google says the control is not used as a ranking or inclusion signal for other parts of Search, and that changes generally take a few days to apply, so record its state every time you run check 25.

Similarly, the reporting rows matter for the same reason. Search Console’s Generative AI performance report shows impressions only, for AI Overviews and AI Mode combined, with no clicks or queries, and it may not appear at all on a site with too few AI impressions. So your AI visibility picture still has to be built from several sources. Our guide on how to improve brand visibility in AI search shows how to combine them, and tracking brand mentions in AI search covers the manual prompt method.

Which Other Google Documentation Changes Touch These Checks?

Smaller documentation changes matter too, because each one shifts how a specific check runs. Google’s Search Central changelog and blog list these (checked October 29, 2026).

DateWhat changedChecksWhat to do
December 15 to 18, 2025Google may skip rendering when a page has a noindex tag or a non-200 status22, 41, 59Set robots tags and canonicals in raw HTML rather than with JavaScript
March 31, 2026Googlebot reads only the first 2MB of an HTML file, and Google moved its crawler IP range files40, 48Keep key tags high in the HTML, then use the new IP files before you block bots
April 13, 2026Back button hijacking became an explicit spam violation, enforced from June 15, 202646 to 49Review ad and library scripts when you run the security pass
June 15, 2026Google said Google Search ignores llms.txt files72Keep llms.txt at Low priority, since only other services might use it
June 17, 2026Site move guidance now covers Change of Address for every www and non-www variant27, 29When you move a domain, file it for each verified variant
July 10, 2026Google may hold pages in a duplicate cluster for up to two weeks after a fix27, 30Wait two weeks before you judge a duplicate fix
October 21, 2026The control what you share page now lists the Search generative AI control too25, 67Review it together with robots meta, data-nosnippet and X-Robots-Tag
October 23, 2026URL guidance now explains why hyphens beat underscores55Use hyphens rather than underscores in new slugs

Setup and Access: The Three Checks That Come First

You cannot audit what you cannot measure. So before anything else, confirm you actually own the data.

#CheckWhy it mattersHow to checkPriority
1Set up and verify Google Search ConsoleBecause it is the only first-party record of how Google crawls, indexes and ranks the siteVerify the domain property in GSC rather than just the URL prefixCritical
2Create and submit an XML sitemapTells search engines which URLs you consider important enough to crawl, as long as it lists only canonical URLsYoast, RankMath or a generator, then submit in GSC and Bing Webmaster Tools and split any file over 50,000 URLs or 50MB uncompressedHigh
3Set up Google Analytics 4Shows engagement and behaviour signals that a crawl report cannot see, such as AI Assistant referralsGA4 property plus a working tag, then confirmation in DebugViewMedium

Also add Bing Webmaster Tools to that list. It costs ten minutes and it matters more than it used to, because ChatGPT leans on the Bing index for live answers. We cover that properly in our guide to ranking in ChatGPT search.

Crawlability: 12 Checks Before Anything Else Counts

Crawlability is the first prerequisite. After all, without a crawl there is no index, and without an index there is no ranking and no AI citation. So this section runs before every other section in a real audit.

Decision tree for checking whether Google can crawl and index a webpage
Check URL discovery, robots.txt, noindex directives and canonical tags before requesting indexing.
#CheckWhy it mattersHow to checkPriority
4Score overall crawlability before anything elseIf Google cannot crawl efficiently, then the rest of the audit is theoreticalSemrush Site Audit crawlability score, then Screaming Frog crawl completion rate as a cross-checkCritical
5301 redirect retired posts when they hold backlinksBecause deleting a page without a redirect throws away link equity you already paid forCheck backlinks first, then 301 to the closest replacementHigh
6Run a full site crawl in order to catch domain-level issuesA crawler checks 100+ predefined issues that accumulate as you add pages, so it catches what manual checks missScreaming Frog, Ahrefs Site Audit or Semrush Site AuditHigh
7Schedule an on-demand crawl instead of relying on search operatorsGoogle does not recrawl low value pages often, so operator results go staleWeekly scheduled crawl in your audit tool, then a diff against last weekHigh
8Serve 410 for spam or fabricated parameter URLsAny 4xx gets junk URLs dropped once Google recrawls them, and 410 also records that the removal is deliberatehtaccess or server config rule matched on the injected pattern, then a test on sample URLsHigh

How Do You Catch Crawl Waste and Broken Pages?

#CheckWhy it mattersHow to checkPriority
9Track crawl requests per day, then improve themCrawl capacity decides how quickly new and updated pages get seen, so a falling line is an early warningGSC Crawl Stats report, then server logs for the detailHigh
10Watch for parameter bloat as well as unexpected URL count growthA sudden jump in discovered URLs usually means an attack or a faceted nav bug, so investigate it the same weekCompare GSC indexed count trend with crawl totals, then repeat month over monthHigh
11Count 3xx redirect chains and 404 pages from the crawlChains waste crawl budget and are invisible unless you crawl for themSite Audit redirect report, then the broken page reportMedium
12Fix broken landing pages, 404s and 403sA high broken response rate damages perceived site quality because users and crawlers hit dead endsScreaming Frog status code report, then a cross-check with logsMedium
13Fix internal and external broken linksBroken links waste equity and also frustrate users mid-journeySite Audit report for pages linking to broken pages, then a fix listMedium
14Use custom crawl queries in order to find template-level issuesLets you export every affected URL and hand the developer one fix instead of fiftyScreaming Frog custom extraction or Ahrefs Data ExplorerMedium
15Do not rely on XML sitemaps for LLM discoveryLLMs build no index of their own, so they have no reason to read your sitemapKeep sitemaps for search engines, while treating AI discovery separatelyLow

How Far Crawl Budget Can Actually Move

Crawl budget gets treated as a fixed constraint. In fact, it is not. For example, in one case reported by Jairo Guerrero of AirOps, a client site went from roughly 8,000 crawl requests per day to about 1.2 million. As a result, a full-site crawl fell from an estimated thirteen years to around a month.

That is a single practitioner account rather than a published study, so treat the exact figures as directional. The mechanism behind it, however, is not in dispute. Crawl capacity responds to server speed, internal link structure, response codes and the ratio of useful pages to junk. All four are things you control, so the ceiling moves when you fix them.

Then track the number in the Search Console Crawl Stats report month over month. A flat line while you publish is a warning. A falling line while your URL count grows is the shape that usually precedes a ranking drop. Still, most sites under 10,000 pages never hit a crawl budget limit, as the FAQ below explains.

Why 410 Beats 404 for Injected URLs

This one deserves more than a table row, because almost every other checklist recommends robots.txt or noindex for parameter junk. That advice fails when the URLs were never yours.

SEO guide to HTTP status codes 200, 301, 404 and 410
Use the correct HTTP status code and avoid redirect chains that waste crawl efficiency.

A negative SEO attack or a compromised plugin can generate hundreds of thousands of fabricated URLs overnight. In one case we reviewed, a site went from roughly 10,000 known pages to over 500,000 discovered URLs, and rankings fell within days. However, blocking those URLs in robots.txt makes it worse. Google can no longer crawl them, so it cannot see that they are gone, and they linger in the index, which matches Google’s crawl budget guide (checked October 29, 2026): blocked URLs stay in the crawl queue much longer.

What Should Junk URLs Return Instead of a robots.txt Block?

Serve 410 Gone instead, because a 404 says the page might come back while a 410 says it will not. However, Google treats both the same way, and its HTTP status code guide (checked October 29, 2026) says every 4xx code except 429 gets the URL removed from the index. So 410 beats 404 on clarity, in your logs and for the next developer, not on speed. Match the injected pattern at server level rather than page by page, because you cannot hand-edit half a million URLs.

Blocking junk URLs in robots.txt is the most common wrong answer in a spam cleanup. You are not hiding the problem from Google, you are hiding the solution.

Indexation: 8 Checks That Decide If You Exist

Crawling gets you read. Indexing then gets you eligible. And because AI systems retrieve from search indexes first, an unindexed page cannot be cited even by a chatbot that has never visited your server.

XML sitemap decision tree for canonical, indexable and valuable URLs
Include only canonical, successful and indexable URLs that should appear in search results.
#CheckWhy it mattersHow to checkPriority
16Verify money pages are indexed in Google and BingAI assistants retrieve from search indexes, and several also lean on BingGSC Pages report, Bing Webmaster Tools URL Inspection, site: operatorCritical
17Look for indexed pagination and sort parameter URLsFaceted URLs waste crawl budget and also create near-duplicates at scale, which is why our ecommerce SEO checklist makes this its own stepsite:domain.com inurl:page and site:domain.com inurl:sortHigh
18Measure the crawled-but-not-indexed ratioA high ratio means Google is reading pages and then choosing to reject themGSC Page indexing report, divide indexed by total knownHigh
19Find insecure pages still sitting in the indexReveals http URLs that never redirect and still compete with your https setsite:domain.com -inurl:httpsHigh
20Treat site quality as a bulk judgement, not a page judgementA strong page on a weak site inherits the site score, and vice versaCompare indexed URL count against total URL countHigh
21Check WordPress footprints that should not be indexedTag, author and paginated archives get indexed by default and also duplicate contentsite:domain.com inurl:tag OR inurl:authorMedium
22Do not noindex thin content, remove it or improve itA noindexed page is still crawled and still counted in sitewide qualityView source for noindex, then cross-check against traffic dataMedium
23Find leftover default WordPress install pagesIndexed sample pages signal an unfinished, low care site, so delete themsite:domain.com inurl:hello-world OR inurl:sample-pageLow

How to Read Your Crawled-But-Not-Indexed Ratio

Most guides tell you to open the GSC Page indexing report. Very few tell you what the number means, or why automated reports never match Search Console when you cross-check it. So here is the version we use with clients.

To begin with, take indexed pages and divide by total known pages. Then band the result.

Indexation rateWhat it usually meansWhat to do next
Above 80%Healthy, because Google broadly agrees with your URL setMonitor quarterly, no action
50% to 80%Bloat is forming, usually parameters or thin archivesFind the pattern, then remove or consolidate it
Below 50%Google is actively rejecting most of what it crawlsTreat as a quality problem rather than a crawl problem, then prune first

The reason the low band is a quality problem rather than a crawl problem is worth spelling out. For instance, Guerrero describes a client where roughly 80 percent of crawled pages carried no value at all, and the effect was not confined to those pages. As a result, the junk dragged down the whole-site assessment, including the pages that were fine.

The instinct at that point is to noindex the weak pages. Resist it. Noindex removes the page from results but Google still crawls it and still weighs it when judging the site. Instead, deleting or genuinely improving the page is what changes the sitewide picture. We go deeper on this in our programmatic SEO guide, where the problem shows up fastest.

Robots and Directives: 3 Checks With Outsized Consequences

In fact, one wrong character in robots.txt can switch off an entire site. This is the shortest section in the checklist, yet also the most dangerous.

#CheckWhy it mattersHow to checkPriority
24Check for Allow versus Disallow typos in robots.txtA single wrong directive silently blocks every bot, and nothing alerts you when it happensRead the file manually, then check the GSC robots.txt reportCritical
25Confirm robots.txt does not block AI user agentsBlocking the agent sets your AI crawl budget to zero even if you rank in GoogleOpen domain.com/robots.txt, then check each named agent line and your firewall rulesCritical
26Confirm robots.txt exists and returns plain textControls where bots go, for example cart, search and feed URLsVisit domain.com/robots.txt, then confirm a 200 plain text responseHigh

The AI User Agents Worth Naming Individually

Blocking AI crawlers is a real business decision, not a default. But most sites block them by accident, through an inherited rule or a security plugin. So check each agent separately, because they do different jobs.

User agentOperatorWhat it doesConsequence of blocking
GPTBotOpenAICollects content used for model trainingYou lose training presence but not live citations
OAI-SearchBotOpenAIIndexes content for live ChatGPT search answersYou lose live ChatGPT citation eligibility, though navigational links can still appear
ChatGPT-UserOpenAIFetches a page when a user action requires itOpenAI says robots.txt rules may not apply, so a block may not hold
PerplexityBotPerplexityIndexes for Perplexity answersYou lose Perplexity visibility
ClaudeBotAnthropicCollects content that may be used for model trainingFuture content is left out of training, not out of Claude search
Google-ExtendedGoogleControls Gemini and AI training use, separate from GooglebotYou keep Google Search, but you lose Gemini grounding

Note the split in that last row. Google-Extended is not Googlebot. Blocking it does not affect your Google Search rankings, which is exactly why so many teams get the decision wrong in both directions.

Which AI Crawlers Can You Actually Block With robots.txt?

You can block the training and search crawlers with robots.txt, but the fetchers that act on a user’s request may not obey it. OpenAI says robots.txt rules may not apply to ChatGPT-User, and Perplexity says Perplexity-User generally ignores them, because a person asked for the page (both checked October 29, 2026).

Anthropic works differently. Its help center (April 2026) lists three agents, and it says blocking Claude-User stops Claude from fetching your pages for user questions. So check every agent you care about, not just the famous ones.

OperatorTraining crawlerSearch crawlerUser-triggered fetcher
OpenAIGPTBotOAI-SearchBotChatGPT-User, where robots.txt may not apply
AnthropicClaudeBotClaude-SearchBotClaude-User, which you can block
PerplexityNone listed, since PerplexityBot is not used for model trainingPerplexityBotPerplexity-User, which generally ignores robots.txt
GoogleGoogle-Extended token for Gemini training and groundingGooglebot, which also feeds AI Overviews and AI ModeUser-triggered fetchers, which generally ignore robots.txt

Both OpenAI and Perplexity say a robots.txt change can take about 24 hours to reach their systems. In addition, robots.txt is only half the picture. If your firewall blocks an operator’s published IP ranges, the bot fails even when robots.txt allows it, so review check 48 alongside check 25.

For Google, blocking Google-Extended does not remove you from AI Overviews or AI Mode. Instead, Google’s AI features guide (checked October 29, 2026) names robots.txt rules for Googlebot and snippet controls as the way to manage those features, alongside the Search generative AI control in Search Console.

A Straight Answer on llms.txt

Several checklists now recommend llms.txt as if it were robots.txt for AI. That framing is wrong, and it is worth saying plainly.

llms.txt is a proposed plain-text file listing your key pages for language models. No major AI provider has committed to reading it. Google’s guide to optimizing for generative AI features also says, in a note added on June 15, 2026, that Google Search ignores these files, so they neither help nor hurt your visibility there. More importantly, it does not match how these systems actually work, because they retrieve through search indexes rather than browsing a directory of your choosing.

So add it if you like. After all, it takes ten minutes and costs nothing. Just do not count it as an AI visibility strategy, and do not let it displace the checks in this list that genuinely move the needle.

Duplicate Content and Canonicals: 5 Checks

Duplicate content rarely arrives as a copy-paste problem in the first place. It arrives as an architecture problem, and the fix is usually a redirect rule rather than a rewrite.

Canonical URL decision tree for duplicate pages, preferred versions and redirects
Choose between a self-referencing canonical, preferred canonical, 301 redirect or removal based on page duplication and replacement.
#CheckWhy it mattersHow to checkPriority
27Pick one canonical version: http vs https, www vs non-wwwFour live variants split signals and also leave insecure duplicates indexedLoad all four variants, then confirm each 301s to the same targetCritical
28Apply the chosen redirect site-wide, not only on the home pageA home-page-only redirect still leaves every inner page duplicatedTest a deep product or blog URL across all four variants rather than the rootHigh
29Check whether an old domain still has indexed pagesRedirecting only the home page leaves the old domain competing with you until every URL redirectssite:olddomain.comHigh
30Hunt keyword cannibalisation by sorting URLs and title tagsTwo pages on one intent split clicks, so both underperformScreaming Frog crawl filtered by title, then compare GSC query overlapHigh
31Delete useless tag archives on small sitesTag pages add crawl depth and duplication but no ranking upsideSite Audit crawled pages sorted by crawl depthMedium

The Four-Variant Test Almost Nobody Runs Properly

Here is the check itself, and it takes two minutes. First of all, pick a deep URL, not your home page. For example, a product page or a blog post is ideal. Then request all four of these and record the response.

  • http://domain.com/deep-page/
  • https://domain.com/deep-page/
  • http://www.domain.com/deep-page/
  • https://www.domain.com/deep-page/

Three of them should return a single 301 to the fourth. If any of them returns a 200, then you have a live duplicate. Similarly, if any of them returns a chain of two or more redirects, you are leaking crawl budget on every request.

We test the deep URL rather than the home page because home-page-only rules are the single most common miss. A server rule that redirects the root correctly can still pass inner paths through untouched.

How Do You Check Hreflang and International SEO Setup?

Hreflang checks only apply when you publish the same page in more than one language or for more than one country, which is why they sit outside the 105 numbered checks. If that is your site, add the five checks below to the duplicate content pass, because hreflang is how Google learns which version to show in each market.

Hreflang checkWhat Google’s guidance says (checked October 29, 2026)How to check
Every version lists itself and all othersEach language version must list itself as well as all other language versionsCrawl one URL per language, then compare the hreflang sets
Links point both waysIf two pages do not both point to each other, Google ignores the tagsUse your crawler’s hreflang report to find missing return links
Fully qualified URLsAlternate URLs must include the protocol, such as https://example.com/foo, not /fooSearch the hreflang values for relative paths before launch
One method, used consistentlyHTML tags, HTTP headers and sitemaps are equivalent, so using all three gives no benefitPick one method, then remove stray duplicates
A fallback for unmatched usersUse x-default for language selectors or for home pages that redirect by languageConfirm one x-default URL per set of pages, then test a selector

Also note that Google treats localized pages as duplicates only when the main content is left untranslated. So a translated template with untranslated body copy is a duplicate content problem, not an hreflang problem.

Site Architecture and Internal Linking: 8 Checks

Internal links are how discovery actually happens, and they are the cheapest ranking lever most sites still ignore.

To see which links on a page are followed, including a page-level meta robots nofollow, run it through our free dofollow link checker.

SEO site architecture diagram showing important pages within three clicks and an orphan page without internal links
Use a shallow website structure and internal links to keep important pages easy to discover.
#CheckWhy it mattersHow to checkPriority
32Add internal links from relevant existing pages to each new postPasses equity and also speeds up discovery of the new URLsite:yourdomain.com + keyword, then add contextual linksHigh
33Avoid nesting a keyword-optimised page as a folder for child pagesStructures such as /city-attorneys/wrongful-death/ raise cannibalisation riskReview folder depth in Screaming FrogHigh
34Investigate pages blocked from crawling and confirm the reasonSome blocks are intentional, while others silently hide pages you want rankedSite Audit blocked list, then view source for noindexHigh
35Keep every important page within three clicks of the home pageDeep pages read as low priority, so crawlers may never reach themSite Audit crawl depth reportHigh
36Automate internal linking and pruning with agents or purpose-built toolsBecause it scales two tasks that directly affect crawl efficiency and relevanceInternal linking tool or a scripted candidate-scoring passMedium
37Build internal links to new and important pages deliberatelySearch engines discover pages through links before sitemapsInternal link report in your crawlerMedium
38Find and fix orphan pagesNo internal links means poor discovery and also no user path to money pagesCrawl output cross-referenced against sitemap and logsMedium
39Link out to relevant high authority external resourcesCorrelation work suggests pages that cite strong sources tend to rank betterManual review during content productionMedium

Rendering and JavaScript: The Section Most Checklists Skip

Googlebot renders JavaScript. However, most AI crawlers do not, or they give up far sooner. So a page that looks complete in your browser can reach an AI system as an empty shell. That is why this layer gets two checks of its own, numbers 40 and 41.

#CheckWhy it mattersHow to checkPriority
40Keep HTML lean and avoid bloated markupHeavy DOMs waste the limited patience and budget of AI crawlers, and Googlebot also stops reading an HTML file after its first 2MB (Google, March 2026)Lighthouse DOM size and page weight metricsHigh
41Move client-side rendering to server-side renderingSimpler AI crawlers miss anything that needs JavaScript to appear, even though Googlebot renders itCompare view-source HTML with the rendered DOM, then fix any gapsHigh

The Two-Minute Raw HTML Test

Open the page, then use view-source rather than inspect element. View-source shows what the server sent. Inspect, by comparison, shows what your browser built afterwards.

Then search that raw HTML for your H1, your first paragraph, your price and your primary call to action. If any of them is missing from view-source but visible on screen, JavaScript is producing it. Googlebot will probably still see it. An AI crawler, however, often will not.

The Log File Check That Finds Rendering Gaps

This is the check we have not seen in any competing checklist, and it is the one that finds problems the other 104 checks miss.

When a bot fetches a page, it requests the HTML document. If it then also renders, it requests the JavaScript and CSS assets that page depends on. So you can compare the two counts directly in your server logs.

To begin with, filter your logs by user agent. Count HTML document requests. Then count requests for the JS bundles those pages load. If a bot fetched a page template 10,000 times but pulled the associated JavaScript only a few hundred times, that bot is not rendering your pages. As a result, whatever your JavaScript produces is invisible to it.

So that single ratio tells you whether server-side rendering is a nice-to-have or an emergency. And it is measurable today, because you already have the logs.

What you see in logsWhat it meansAction
HTML fetches high while JS fetches stay near zeroSo the bot reads raw HTML onlyMove critical content to server-rendered HTML
HTML and JS fetches roughly proportionalThe bot rendersNo rendering action needed
No fetches at all for a named agentBlocked, or never discoveredCheck robots.txt, then internal links and index status

Speed and Core Web Vitals: 4 Checks

Speed stopped being purely a ranking conversation. It is now also a citation conversation, because a crawler on a budget abandons slow responses.

Core Web Vitals passing targets for LCP, INP and CLS
A website should aim for LCP within 2.5 seconds, INP within 200 milliseconds and CLS within 0.1.
#CheckWhy it mattersHow to checkPriority
42Find and fix slow-loading pagesPage experience affects conversion rate as well as rankingsPageSpeed Insights, GTmetrix, then real user data in GSCHigh
43Improve server response time specificallyTime to first byte governs whether a crawler waits or moves onServer response time in PageSpeed Insights as well as hosting logsHigh
44Review the Core Web Vitals section of your site auditGoogle has confirmed Core Web Vitals as a ranking input, so a failing URL group is worth fixingGSC Core Web Vitals report by URL groupMedium
45Use CWV as the 28-day baseline and site speed as the instant checkCWV is a rolling field-data window, whereas lab speed is immediateGSC for CWV, while PageSpeed Insights gives the instant readMedium

Does Site Speed Affect AI Citations?

There is emerging evidence that this now affects citations as well as rankings. A study of more than 2,000 sites, cited in the AirOps session, reported that faster sites were referenced roughly 20 to 30 percent more often in AI answers, with one slide putting the gap at 1.47 times. We have not reviewed the underlying methodology, so treat those numbers as a direction of travel rather than a benchmark.

The mechanism is intuitive enough. In short, a crawler working through a large candidate set on a fixed budget abandons slow responses. Googlebot usually comes back later. In contrast, an AI crawler often does not.

Google’s published Core Web Vitals thresholds are LCP under 2.5 seconds, INP under 200 milliseconds and CLS under 0.1. Measure against field data where you have it, because lab scores flatter you. web.dev adds (checked October 29, 2026) that a page passes only when it meets all three targets at the 75th percentile of page loads, split by mobile and desktop.

How Many Sites Pass Core Web Vitals Today?

Fewer than half of mobile sites pass. The HTTP Archive Web Almanac 2025, published in January 2026, found that 48% of mobile websites and 56% of desktop websites had good Core Web Vitals in its July 2025 data, up from 44% and 55% in 2024.

So a pass is still an edge on mobile. Also, INP replaced First Input Delay as a Core Web Vital on March 12, 2024, according to web.dev, so any audit template that still grades FID is out of date. In that case, swap the FID column for INP before you run checks 44 and 45. Our Core Web Vitals and INP guide covers the fixes for WordPress and Shopify themes.

Server and Security: 4 Checks That Are Really Business Risk

This section is short, and every item in it is Critical. That is not padding. Each one can take a site off the map in a week, so treat them as emergencies.

#CheckWhy it mattersHow to checkPriority
46Investigate strange outbound links on indexed pagesUnrelated outbound links usually mean the site has been compromised, so act the same daysite: search plus manual review, then a malware scanCritical
47Monitor server uptime and 5xx error rateDuring downtime crawlers get nothing and you lose citations for that windowUptime monitor as well as 5xx rate in server logsCritical
48Put bot protection in front of the serverSpoofed crawler floods can crash the origin and kill visibility, so verify real Googlebot against Google’s published IP ranges before you block anythingCloudflare or an equivalent WAF with bot challenge rules, while allowing verified crawlersCritical
49Search indexed file types for exposed private documentsIndexed PDFs and spreadsheets routinely leak salaries, addresses and client data, so treat any hit as a breachsite:domain.com filetype:pdf, then repeat for xls, doc, csvCritical

Run the Exposed File Search Before Your Client Does

Check 49 is the one that changes the temperature of a client meeting. So run these four searches on any site you take over.

  • site:domain.com filetype:pdf
  • site:domain.com filetype:xls OR filetype:xlsx
  • site:domain.com filetype:doc OR filetype:docx
  • site:domain.com filetype:csv

Most of the time you find old brochures. Occasionally, though, you find an internal payroll sheet or a client contact export sitting in an uploads folder. Fix it by removing the file and serving 410, then requesting removal in GSC. Do not simply noindex it, because the file itself remains publicly downloadable.

Mobile: 1 Check, Still Non-Negotiable

#CheckWhy it mattersHow to checkPriority
50Confirm pages are genuinely mobile friendlyGoogle indexes the mobile version, so mobile is the version that countsURL Inspection in GSC plus manual device testing, since Google retired the Mobile Usability report in December 2023High

On-Page Technical: 5 Checks at Template Level

These are on-page items, but you audit them technically. In other words, you are looking for template errors that affect hundreds of pages at once, not for one weak title.

#CheckWhy it mattersHow to checkPriority
51Write title tags and meta descriptions that earn the clickExact-match keywords do not correlate with position, but clicks still bring trafficSERP preview tool, then GSC CTR by queryHigh
52Run an on-page checker per target page and keywordSurfaces concrete gaps such as missing variations and weak readabilitySemrush On Page SEO Checker in manual modeMedium
53Skim your site: results for title tag problemsWrong brand or domain in titles hurts CTR and also looks brokensite:domain.com, then read the SERP titlesMedium
54Use intitle: to count pages carrying a wrong or outdated titleFinds template-level title errors at scale instead of one at a timesite:domain.com intitle:"old brand name"Medium
55Use short, descriptive URLsShorter URLs correlate with better rankings and are also easier to citeSet the slug to the primary keyword, drop stop words and use hyphens rather than underscores (Google, October 2026)Medium

Structured Data: 6 Checks After Google Removed FAQ Rich Results

Structured data is the one area where a 2027 checklist has to be more careful than a 2023 one. Google removed FAQ rich results from Search on 7 May 2026. HowTo went the same way in 2023. Plenty of checklists still tell you to add both. Six checks below, then two things worth saying plainly.

Schema markup validation workflow showing how to test structured data before deployment
Validate structured data against the rendered page, not the source file, before it ships.
#CheckWhy it mattersHow to checkPriority
56Map every template to a schema type that still earns a Search appearanceMarkup built for a retired feature costs dev time but returns nothingCompare each template against Google’s structured data gallery, one template at a timeCritical
57Make structured data match the visible text on the pageGoogle states this outright, so a mismatch risks a manual action for spammy markupRich Results Test, then read the rendered page beside the outputCritical
58Validate with the Rich Results Test and the Schema.org validatorThe first tells you what Google can use, whereas the second shows what is syntactically broken. They answer different questionsRun both on one representative URL per templateHigh
59Confirm your JSON-LD survives rendering, not just view-sourceSchema injected by a tag manager or by JavaScript is invisible when the page is judged on raw HTMLCompare view-source against the rendered DOM, the same test as check 41High
60Keep one Organization block with sameAs, consistent sitewideConflicting entity blocks across templates weaken the entity you are trying to establishCrawl for duplicate Organization nodes on the same URLMedium
61Watch the Enhancements reports in Search Console for silent dropsValid items fall off after a template release while nobody files a bugGSC, Enhancements, per type, month over monthMedium

The Schema Types That Stopped Paying in 2026

FAQPage and HowTo are two of the most commonly recommended types in technical SEO checklists. However, neither produces a result in Google Search any more.

TypeCurrent statusWhat to do
FAQPageRich results stopped appearing on 7 May 2026. The appearance filter, rich result report and Rich Results Test support were dropped in June 2026, and Search Console API support ended in August 2026Leave existing markup in place if removing it is costly, because it causes no harm. Also stop counting it as a deliverable and stop reporting FAQ impressions that no longer exist
HowToRemoved from mobile in August 2023 and desktop in September 2023. No Search appearance sinceRemove it from new templates. Also, do not add it to a 2027 build
SpeakableStill in the gallery, still beta, still aimed at news publishers in a limited set of localesSkip it unless you are a news publisher inside that set

The wider point is that markup follows Google’s feature list, not schema.org’s vocabulary. Schema.org publishes hundreds of types. In contrast, Google renders a couple of dozen. So build for the second list.

In addition, the same clean-up hit other types. Google’s documentation updates page (checked October 29, 2026) shows course info, estimated salary, learning video, special announcement and vehicle listing documentation removed on September 9, 2025, and practice problem documentation removed on January 6, 2026. Google also clarified in November 2025 that Dataset markup is used only by Dataset Search, not Google Search.

A Straight Answer on Schema and AI Search

You will read everywhere that schema is how you get cited by AI. Google’s own documentation on AI features says the opposite in one line: there is no special schema.org structured data you need to add to appear in AI Overviews or AI Mode.

What Google does ask for is that the page is indexed, eligible to be shown with a snippet, and that its structured data matches the visible text. That last part is the useful instruction. Schema does not buy you a citation. However, inaccurate schema can cost you one.

The studies reporting large citation lifts from schema are correlational. In other words, sites that mark up properly tend to be sites that do everything else properly too. Mark up for the Search features you are genuinely eligible for, keep it accurate, and treat any AI benefit as a side effect rather than the business case. Our restaurant schema markup guide shows what that looks like for one business type, with Google’s support status for each restaurant feature.

Logs and Monitoring: 4 Checks Nobody Else Teaches

Search Console reports only on Google. So your server logs are currently the only reliable source of truth on AI crawler behaviour. As a result, this section is the highest-leverage part of a modern audit.

#CheckWhy it mattersHow to checkPriority
62Analyse logs for absence of signalBecause important pages with zero bot hits effectively do not existScreaming Frog Log Analyser, OnCrawl or JetOctopusHigh
63Identify which AI bots hit your server and how oftenGSC reports Google only, so logs are your sole AI crawler datasetFilter logs by user agent string per bot, then verify the IPs against each operator’s published rangesHigh
64Compare bot HTML fetches against JavaScript file loadsA large difference shows that this specific bot does not render, so it misses JavaScript contentCross-reference HTML and JS asset entries per user agent, then compare the ratioMedium
65Prefer a platform that gives you raw server log accessSome hosted platforms block log access entirely, so you cannot see AI crawling at allAsk your host or platform for raw access logs before you commitMedium

What to Do When Your Platform Will Not Give You Logs

Hosted ecommerce platforms are the usual problem here. Shopify, for example, does not expose raw server logs, so you cannot confirm whether an AI crawler ever reached a product page.

Even so, you still have options. Three workarounds get you most of the way.

  1. Put the site behind a CDN such as Cloudflare and read the CDN logs instead. The request still passes through it.
  2. Use GSC Crawl Stats as a proxy for Googlebot behaviour, then treat it as a floor rather than a full picture.
  3. Finally, add lightweight server-side logging through an app or middleware layer where the platform allows it.

If none of those is possible, then factor it into your platform decision. Losing log visibility is a real cost, and it is worth naming at the point of choosing a stack rather than two years later.

AI Search, GEO and AEO: 7 Checks

This is where a 2027 checklist earns its title. In fact, these items are not content advice, they are infrastructure decisions about how retrieval works.

#CheckWhy it mattersHow to checkPriority
66Cover topics and entities, not only exact keywordsRetrieval works on semantic similarity, then reranks candidatesEntity and subtopic coverage audit, then a comparison with the SERPHigh
67Expect the impressions-up, clicks-down patternAI answers raise impressions while cutting clicks on informational queriesGSC Performance, then compare impressions and clicks trend linesHigh
68Optimise beyond Google: YouTube, Reddit, Amazon, TikTok, LinkedInDemand is fragmenting, and AI systems also cite those platforms heavilyMap where your audience searches, then build for each surfaceHigh
69Optimise the title and snippet that the search API returnsThat short extract is frequently all the model ever sees of your page, so put the answer in itSERP preview plus GSC, then check what text actually gets pulledHigh
70Plan for organic click decline while still harvesting itClick share is compressing on informational intent, so diversify in parallelTrack click share trend by intent type rather than sitewideHigh
71Accept that citations can happen without a crawlWhen the index snippet answers the prompt, no AI bot ever visits your serverCompare AI citation tracking with your server log hits, then note the gapMedium
72Implement llms.txt but keep expectations lowCheap to add but unproven in practice, and Google Search ignores it (Google, June 2026)Add /llms.txt in root, then measure nothing from itLow

How Retrieval Picks Candidates Before It Picks an Answer

Check 66 tells you to cover topics and entities rather than exact keywords. The reason sits in how retrieval works, and it runs in two stages. Our guide to generative engine optimization covers what to do with that on the content side.

First the system finds candidates by meaning rather than by string match. In effect, your query becomes a numerical representation, and passages that sit close to it in meaning get pulled in. That is why a page can be retrieved for a phrasing it never literally contains.

Then it reranks that pool before choosing what to quote. Guerrero puts the size of that shortlist in the region of 150 results. We have not found published documentation confirming that specific figure, so treat it as a practitioner estimate. The two-stage shape, retrieve broadly then rerank, is well established regardless of the exact number.

The practical instruction is the same either way. Name the entities. Then list the tools, standards, competitors, regulations and adjacent concepts a knowledgeable person would expect. Breadth of coverage gets you into the candidate pool. In contrast, repeating one phrase does not.

Why You Can Get Cited by a Bot That Never Visited You

This is the most misunderstood mechanic in AI search, and it changes what you should optimise.

When someone asks an AI assistant a question, the system does not usually crawl the open web in that moment. Instead, it queries a search index, gets back a set of results, and each result arrives as a title plus a short text extract. If that extract already answers the prompt, then the model can write its answer and cite you without ever requesting your page.

Two consequences follow. First, your server logs will under-report AI visibility, because many citations leave no trace there. Second, the snippet is the artefact being judged, not your full page.

So the practical move is to make the first 200 characters of the relevant section carry the answer. Front-load the fact. Put the number, the definition or the direct response near the top of the passage rather than after three paragraphs of setup.

The scale of the click compression is worth naming, even though the numbers move around by study. Eric Siu cites work putting the organic traffic reduction from AI Overviews at more than 37 percent on affected queries. Published figures in this area vary widely, so use it to size the risk rather than to forecast your own result. Segmenting your own Search Console data by intent gives a far more useful answer than any industry average.

We got this wrong on our own site for most of 2025. We optimised full pages for depth and buried every direct answer under context. Impressions climbed, citations did not.

How Much Do AI Overviews Cut Clicks to Classic Results?

By a lot on informational queries, although the exact share varies by study. Ahrefs found that an AI Overview correlated with a 58% lower average click-through rate for the top-ranking page in December 2025 data, up from 34.5% in its April 2025 study. Pew Research Center found that users who saw an AI summary clicked a traditional result in 8% of visits, against 15% without one.

Questions trigger these summaries most. In the same Pew data from March 2025, 60% of searches that began with who, what, when or why produced an AI summary, compared with 18% of all searches. As a result, question-led informational pages feel the cut first, which is why checks 67 and 70 ask you to segment clicks by intent.

The audience is large, too. On its Q2 2025 earnings call, as reported by TechCrunch, Google said AI Overviews had 2 billion monthly users and AI Mode more than 100 million in the US and India. However, Google’s AI features guide (checked October 29, 2026) says clicks from results pages with AI Overviews are higher quality, so track engaged sessions and conversions next to clicks before you judge a drop.

Agent Readability: 3 Checks for the Web That Clicks Back

AI agents are starting to complete tasks rather than just summarise pages. That means your site now needs to be operable by software, not only readable by it. No other technical SEO checklist we audited covers this at all.

#CheckWhy it mattersHow to checkPriority
73Test key conversion flows in the accessibility treeBecause it shows exactly what an agent understands as a button, link or plain textChrome DevTools, Elements panel, Accessibility paneHigh
74Use real semantic tags for interactive elementsA styled div has no role and no accessible name, so agents cannot act on itCompare the accessibility tree with the visual UI, then fix any unnamed controlHigh
75Track agent commerce protocols from Google, Shopify, PayPal and StripeAgent-led purchasing is being built now, so it needs machine-readable checkoutFollow the published protocol documentation for your platform, such as the Universal Commerce Protocol that Google namesMedium

How to Read Your Own Accessibility Tree

The check takes about five minutes and needs no tools beyond Chrome.

  1. Open the page and press F12 to launch DevTools.
  2. Go to the Elements panel, then open the Accessibility pane on the right.
  3. Then enable the full-page accessibility tree view.
  4. Find your primary action, for example Add to Cart or Request a Quote.
  5. Finally, read its role and its accessible name.

You want to see a role of button or link with a clear name. What you often see instead is a generic node with no name at all. That happens when the element is built like this:

<div class="btn" onclick="submitOrder()">Add to Cart</div>

Visually that is a button. Structurally, however, it is a rectangle. An agent reading the accessibility tree sees no interactive control, so it cannot complete the task. The fix is genuinely this simple:

<button type="submit">Add to Cart</button>

Same appearance with the right CSS, completely different machine meaning. Run this on your cart, your quote form and your booking flow first, because those are the pages where failure costs money.

What Does Google Say About AI Agents Using Your Site?

Google now gives the same advice as this section. Its guide to optimizing for generative AI features (checked October 29, 2026) says browser agents may analyze screenshots, inspect the DOM structure and interpret the accessibility tree when they complete tasks such as booking or comparing products.

The guide also names the Universal Commerce Protocol as one of the emerging protocols that will let Search agents do more, which is the kind of standard check 75 asks you to track. In addition, web.dev’s guide to agent-friendly websites (April 2026) lists screenshots, raw HTML and the accessibility tree as the three main ways agents read a page, and it calls WebMCP a proposed standard that is still in active development.

Keyword and Intent: 13 Checks

Technical work gets you eligible. Intent work, in turn, decides whether eligibility turns into traffic. These checks belong in a technical audit because they are diagnostic, not creative.

#CheckWhy it mattersHow to checkPriority
76Assess search intent before writing anythingWhen the SERP shows blog posts, a product page will not rank thereSearch the keyword, then study page type and format of the top tenCritical
77Pull keywords ranking in positions 2 to 15Small gains here move traffic fastest, so start the campaign hereOrganic Research positions report, then a custom range of 2 to 15Critical
78Check the organic keyword trend before diagnosing anythingBecause a sharp decline points to a penalty or link problem that outranks all elseSemrush or Ahrefs organic overview, then a 24-month viewHigh
79Collect related queries from autocomplete and People Also AskThese reveal the questions the page must answer in order to satisfy intentSERP features as well as the questions report in your keyword toolHigh
80Filter keyword lists by city or region for local businessesPrioritises high transactional local terms rather than national noiseKeyword filter on the location modifier, such as a city nameHigh
81Find intent mismatches on pages that already rank badlyA commercial page cannot rank for informational intent, however good it isCompare your page type with the SERP result types, then fix the mismatchHigh

Which Checks Cover Targeting, Difficulty and Position Bands?

#CheckWhy it mattersHow to checkPriority
82Find subtopics from what the top pages already rank forA winning page typically ranks for hundreds of related terms, so its profile maps the topicKeyword tool, then filter competitor keywords to positions 1 to 5High
83Only create a new page when the intent is genuinely differentSame intent belongs on one page, whereas different intent needs its ownCompare the two candidate SERPs by hand before you buildHigh
84Pick one primary keyword target per pageEvery page needs a single main target so the topic is unambiguousKeyword mapping sheet, so each URL has one primaryHigh
85Target long tail queries of four or more wordsLower volume but far higher conversion, and AI queries run much longerFilter your keyword set by word count, then screen for intentHigh
86Assess difficulty using competitor metrics rather than just a KD scoreKD is a rough estimate, so never use it aloneBacklink and referring domain counts of the top ten pagesMedium
87Audit positions 16 to 50 as existing opportunitiesFixing the strongest pages usually lifts this band as a knock-on effectOrganic Research positions 16 to 50Medium
88Treat positions 51 to 100 as clustering opportunitiesRanking that low usually means no focused page exists for the intent, so build or merge oneOrganic Research positions 51 to 100Medium

Why the Page Above You Has Already Done Your Research

Check 82 is the cheapest research method on this list, and the scale of it surprises people. Ahrefs analysis of roughly 3 million searches found that the page ranking first typically also ranks for close to 1,000 other keywords.

So the page beating you has already mapped the topic. Then pull its full keyword profile, filter to positions 1 to 5, and read the result as a coverage map. You are not copying the page. Instead, you are reading the evidence of what the topic actually contains.

The same body of work also sits behind check 55. A study of around 2 million keywords found shorter URLs correlate with better positions, which is why the slug should carry the primary keyword and nothing else. Similarly, check 39 comes from correlation work suggesting pages that link out to high authority resources tend to rank better, not worse.

The Position Band Framework

Most guides stop at low-hanging fruit and mean positions 2 to 10. We widen the first band and add two more, because SERP features and AI answers have pushed the click cliff further up the page.

Position bandWhat it usually meansFirst action
2 to 15Google already trusts the page, but it is not the best answer yetImprove on-page depth, add internal links, then refresh the data
16 to 50Relevance is recognised but authority or coverage is thinExpand coverage, fix intent match, then add supporting internal links
51 to 100No focused page exists for this intent, so you are ranking by accidentBuild a dedicated page, or fold the term into the right cluster

Content Quality: 10 Checks

Quality is a technical concern because it is measured in bulk. So a hundred weak pages will drag down the twenty good ones sitting beside them.

#CheckWhy it mattersHow to checkPriority
89Keep content, links and E-E-A-T as the core building blocksFirst-hand experience and named authorship matter more now that AI content floods the webAuthor bios, sourced data and real case studies, as well as named reviewersHigh
90Make sure the content solves the searcher’s actual problemGeneric advice fails, whereas practical and measurable advice winsCompare your page with the PAA questions on the SERP, then fill the gapsHigh
91Run a content audit against GA4 and GSC dataSorting pages into rewrite, remove, update and review makes the site leaner and also easier to crawlContent audit tool with GA4 and GSC connectedHigh
92Apply a business lens before deleting any pageSome low-traffic pages still serve sales, legal or proof functions a tool cannot seeManual review of every flagged page with the commercial ownerMedium
93Check engagement and traffic together before keeping a pageLow dwell time as well as no traffic marks a losing page that needs a decisionContent audit table view, then sort by both metricsMedium

How Do You Check Readability and Page Polish?

#CheckWhy it mattersHow to checkPriority
94Check reading comprehension levelComplex writing loses readers regardless of how correct it isHemingway Editor or Yoast readability analysisMedium
95Define writer guidelines before generating anything with AIStyle-matched output reaches usable quality, whereas generic output does notHave the model extract patterns from your best existing posts before it writesMedium
96Improve visual readability with short sentences and useful imagesA wall of text drives readers away, while charts add meaning stock photos cannotManual review, then remove filler imageryMedium
97Search for placeholder text still live on the siteIndexed placeholder text signals an unfinished, low quality site, so remove it before launchsite:domain.com "dolor sit amet"Medium
98Write an introduction that confirms the reader is in the right placeA weak intro loses the reader before the useful part arrivesResonate, build credibility, then promise the specific outcomeMedium

Two Content Quality Numbers Worth Holding On To

Check 94 exists because reading level is a distribution problem, not a matter of taste. Around half the adult population in the United States reads below an eighth grade level. A page written at graduate reading level is therefore not more authoritative. Instead, it is less accessible to half of its audience.

Check 95 is the one most teams skip when they start generating with AI. Eric Siu’s framing is that output reaches roughly 95 percent of usable quality when the model has learned your house style first, against generic output when it has not.

The method is straightforward. Have the model read ten of your strongest existing posts and extract the patterns: typical length, sentence rhythm, how you use tables, how you cite data, where links sit, how you open and close. After that, save it as a standing instruction. Then generate.

Off-Page: 7 Checks

Links are still the clearest external trust signal, and they now feed AI citation patterns as well as rankings.

#CheckWhy it mattersHow to checkPriority
99Audit the backlink profile for toxic and unnatural linksPublic network patterns and spammy web 2.0 links still cause real problemsSemrush Backlink Audit or Ahrefs referring domains reviewMedium
100Check that anchor text is mostly branded and genericAggressive exact match anchors are still the classic over-optimisation signalBacklink Audit anchors reportMedium
101Prospect links from sites already linking to top-ranking pagesBecause they have already shown they will link to content on this topicTop ten pages, then their backlinks reportMedium
102Treat citations broadly: links, reviews and unlinked mentionsAI systems weigh brand mentions, not only hyperlinksBacklink tools as well as a brand mention and local citation auditMedium
103Expand prospecting with a content database filtered by referring domainsFinds pages with genuine link pull beyond the current top tenContent Explorer style search, then a filter of at least 20 referring domainsLow
104Only disavow for genuinely nasty links such as negative SEODisavow is rarely needed but very easy to misuseDisavow tool, treated as a last resortLow
105Build the prospect list before you publish, not afterProspecting first means you have people to promote to as soon as you publishOutreach list assembled during the research phaseLow

Full Comparison: What Every Major Technical SEO Checklist Covers

Before writing this, we audited fourteen published technical SEO checklists, including the ones from Semrush, Ahrefs, Backlinko, DebugBear, Sitebulb, seoClarity, Yotpo and Page One Power. We logged what each one covers and, more usefully, what none of them do.

Here is the whole comparison. Everything below reflects those pages as published in August 2026, and every one of them is a genuinely useful resource. The point is not that they are weak. Rather, the point is that the field has a shared gap.

ChecklistApprox wordsItemsAI crawler robots.txtLog file analysisHTML vs JS fetch gapAccessibility treeAgent commerceCitation without crawl
Hustle Marketers (this page)~14,500105Yes, per agentYes, with methodYesYesYesYes
DebugBear~6,80074Yes, full tableNoNoNoNoNo
Yotpo~6,800NarrativeYesNamed, not taughtNoNoNoNo
Semrush~3,700~21YesNoNoNoMentioned, no protocolNo
NoGood~4,800~60PartialNoNoNoNoNo
Wellows~5,300~50NoGooglebot onlyNoNoNoNo
Page One Power~3,000~60Yes, training vs retrievalYesGestures at itNoNoNo
Ahrefs~4,20082Yes, one lineNoNoNoNoNo
Backlinko~4,80035 to 40YesNamed, not taughtNoNoNoNo
GWContent~3,80047YesNoNoNoNoNo
cognitiveSEO~9,50031NoNoNoNoNoNo
Shortlist.io~3,00050+NoGooglebot onlyNoNoNoNo
DashThis~2,90011NoNoNoNoNoNo
seoClarity~5,00016NoNoNoNoNoNo
Sitebulb~850Sheet onlyNoNoNoNoNoNo

The Ten Things No Other Checklist Explains

Four of the columns above are empty across all fourteen competitors. Five more items are covered so thinly elsewhere that they may as well be. Here is the full list, with an honest note on who else touches each one.

#What we coverWho else covers itFind it in
1Comparing bot HTML fetches against JavaScript loads to find rendering gapsNobody. One competitor gestures at raw HTML differences without a methodLogs section
2Reading your accessibility tree in Chrome DevTools to see what an agent understandsZero of fourteenAgent readability
3Agent commerce protocols from Google, Shopify, PayPal and StripeZero of fourteen. One names the concept without naming a protocolAgent readability
4Why you get cited by AI systems that never request your pageZero of fourteen. All assume a crawl happensAI search section
5Serving 410 for injected and spam URLs, not just dead productsThree mention 410, all for discontinued products onlyCrawlability
6The crawled-but-not-indexed ratio as a banded quality metricThe GSC status is common, but the ratio with thresholds appears nowhereIndexation
7Site quality judged in bulk, built into audit steps rather than mentionedTwo glancing mentions, no action attachedIndexation
8llms.txt with its actual limitations stated plainlyTwo are honest about it. Three recommend it uncritically and one describes it incorrectlyRobots and directives
9The four-variant redirect test run on a deep URL rather than the home pageCanonical choice is universal. Testing inner URLs across all four variants is notDuplicate and canonical
10That FAQPage and HowTo markup no longer earn a Search appearance, with the datesZero of fourteen. One still recommends adding both types in a 2026 checklistStructured data section

What Do the Ten Gaps Have in Common?

The pattern underneath all ten is the same. Every other checklist treats an AI system as a crawler you either permit or block. None of them treats it as a user that has to read the page, render it, and increasingly act on it.

The 50-Item Short Version for Small Sites

Running all 105 checks on a twelve-page site is a poor use of a morning. So here is the reduced set. Run these, and skip the rest until the site grows past a few hundred URLs.

First of all, keep every Critical item: 1, 4, 16, 24, 25, 27, 46, 47, 48, 49, 56, 57, 76 and 77. Then add these High items: 2, 5, 6, 8, 9, 10, 17, 18, 19, 20, 26, 28, 29, 30, 32, 33, 34, 35, 40, 41, 42, 43, 50, 51, 62, 63, 66, 67, 68, 69, 70, 73, 74, 78, 79 and 89.

That is 50 checks, and it covers roughly 90% of the risk on a small site.

How Do You Run a Technical SEO Audit Step by Step?

Run it in nine passes, in the order the data depends on itself: access first, then quick tests, the crawl, the index, rendering and logs, performance and security, templates, the AI and agent layer, and finally intent, content and links. The numbering on this page is a category order, so use these passes as the work order.

Which Checks Belong in Each Audit Pass?

Every one of the 105 checks sits in exactly one pass, so no check is run twice.

PassWhat you doChecksWhat you should have at the end
1Confirm access: Search Console domain property, sitemaps, GA4, and Bing Webmaster Tools alongside them1 to 3Verified data sources you can trust before the crawl
2Run the four quick tests from the key takeaways: deep URL redirects, exposed files, AI user agents, crawled but not indexed ratio18, 25, 28, 49A same-day list of the most expensive problems
3Crawl the site, read robots.txt, then map the internal link structure4 to 15, 24, 26, 32 to 39Status codes, redirect chains, broken links, crawl depth and orphan pages
4Reconcile the crawl with the index and the canonical set16, 17, 19 to 23, 27, 29 to 31Your indexation band and the duplicate patterns behind it
5Test rendering, then read the server logs40, 41, 59, 62 to 65Which bots read raw HTML only, and which pages they never fetch
6Check speed, server health, security and mobile42 to 48, 50Critical security findings as well as the slowest templates
7Audit templates: on-page elements and structured data51 to 58, 60, 61One fix per template rather than one per page
8Audit the AI search and agent layer66 to 75Snippet quality, AI visibility baseline and agent-readable controls
9Check intent, content quality and links76 to 105Pages to fix, merge, remove or promote

How Should You Prioritize What the Audit Finds?

Then sort every finding by its priority label, not by pass. Across the list, the labels split like this:

PriorityNumber of checksWhich onesFix window
Critical141, 4, 16, 24, 25, 27, 46, 47, 48, 49, 56, 57, 76, 77Same week
High47Spread across 16 of the 18 categoriesWithin 30 days
Medium38Largest groups: Content Quality (7), then Crawlability, Site Architecture, On-Page Technical and Off-Page (4 each)Within the quarter
Low615, 23, 72, 103, 104, 105When capacity allows

Two patterns are worth knowing before you start. All four Server and Security checks are Critical, so pass 6 can produce emergencies even on a site that looks healthy. And none of the seven Off-Page checks is above Medium, so link work never jumps the queue ahead of access, indexing and rendering. If you want this run for you, the ranges and what drives them are in our breakdown of what an SEO audit costs.

What Are the Most Common Technical SEO Issues and How Do You Fix Them?

Most technical SEO problems show up as one of a few symptoms, from pages missing in Google to content AI tools never quote. So start from the symptom you see, then run only the checks that diagnose it.

SymptomLikely causeChecksFirst fix
Pages missing after a launch or migrationA leftover noindex, a robots.txt block or a root-only redirect22, 24, 26 to 28Remove the block, then test a deep URL
Crawled, currently not indexed keeps risingThin, duplicate or parameter pages17, 18, 20, 22Remove or improve pages rather than noindex them
Discovered, currently not indexed keeps risingWeak internal links or a slow server9, 35, 37, 38, 43Link from strong pages, then speed up the server
Sudden jump in discovered URLsInjected spam or a faceted navigation bug8, 10, 17Return 410 or 404 at server level instead of blocking
Ranks in Google but never cited by AI toolsAn AI agent blocked in robots.txt or by the firewall25, 48, 63Allow the search agents, then confirm hits in logs
Visible on screen but missing from AI answersContent that only JavaScript produces41, 59, 64Run the view-source test, then server-render it
Poor URLs in the Core Web Vitals reportSlow server, heavy scripts or layout shifts42 to 45Fix by URL group, then wait 28 days
Rich result impressions dropA template release or a retired feature such as FAQ56, 57, 61Compare each template with Google’s gallery
Old domain or http pages still rankingRedirects only at the root, or a missed variant19, 27, 29Redirect every URL, then file Change of Address
Impressions up while clicks fallAI answers absorbing informational clicks67, 70Segment by intent, then check the Generative AI report

How Do You Confirm a Fix Worked?

Rerun the same checks after the fix ships. For example, a robots.txt or noindex fix shows up in the URL Inspection tool on the next crawl, while a Core Web Vitals fix needs the full 28-day field window before the report changes. If the symptom stays after that, move to the next likely cause in the row.

Which Tools Do You Need for Each Technical SEO Check?

Google Search Console is the most used tool on this list: it appears in the how-to of 17 of the 105 checks. A site crawler comes next at 14, then keyword and backlink tools and Google search operators at 12 each. And 9 of the 14 Critical checks need nothing more than Search Console, the Rich Results Test, search operators or a plain browser.

We counted every tool named in the How to check column of the tables above. A check can name more than one tool, so the counts overlap.

ToolWho provides itChecks that use itCount
Google Search ConsoleGoogle, free1, 2, 9, 10, 16, 18, 24, 30, 42, 44, 45, 50, 51, 61, 67, 69, 9117
A site crawler or site audit toolThird party4, 6, 7, 11, 12, 13, 14, 30, 31, 33, 34, 35, 37, 3814
Keyword and backlink research toolsThird party77, 78, 79, 80, 82, 86, 87, 88, 99, 100, 102, 10312
Google search operators (site:, inurl:, filetype:, intitle:)Google, free16, 17, 19, 21, 23, 29, 32, 46, 49, 53, 54, 9712
Server or CDN logsYour host or CDN9, 12, 38, 43, 47, 62, 63, 65, 719
Chrome DevTools and view-sourceBuilt into Chrome, free22, 34, 41, 59, 73, 746
PageSpeed Insights or LighthouseGoogle, free40, 42, 43, 454
Bing Webmaster ToolsMicrosoft, free2, 162
Google Analytics 4Google, free3, 912
Rich Results Test and Schema.org validatorGoogle and Schema.org, free57, 582

Which Critical Checks Need Paid Tools?

The five Critical checks that need more than free tools are 4 (a crawler), 46 (a malware scan), 47 (an uptime monitor), 48 (a firewall with bot rules) and 77 (a keyword tool positions report). In contrast, everything else on the Critical list can be done today with what Google gives you for free. To validate markup quickly while you work through pass 7, use our free schema checker.

Bing Webmaster Tools only appears twice, but do not read that as low importance. After all, it is the only free first-party source of Copilot citation data, which is why it sits in pass 1 next to Search Console.

Which Checks Change on Shopify, WooCommerce and Magento?

The 105 checks stay the same on every platform, but the weak spots move. Shopify limits what you can see, while WooCommerce and Magento give you more control and more ways to create duplicate URLs. So run the full list, then spend extra time on the checks below for your platform.

PlatformChecks that need extra careWhyDeep dive
Shopify10, 17, 26, 62 to 65Duplicate product URLs, soft 404s and the default robots.txt, plus no raw server logsShopify SEO issues
WooCommerce2, 17, 21, 26Filter URLs that multiply, archives without canonicals and also an indexable cart and account areaWooCommerce technical SEO
Magento10, 17, 27Layered navigation creates faceted URL bloat unless filters are controlledMagento layered navigation SEO
Any large store9, 35, 41Category architecture, variant URLs and pagination, as well as headless JavaScript storefrontsecommerce technical SEO

How Often to Run a Technical SEO Audit

Cadence depends on site size and on how much changes between audits. If you are budgeting for one rather than running it yourself, our breakdown of what an SEO audit costs covers the ranges and what drives them.

Site sizeFull 105-point auditCritical items onlyLog analysis
Under 500 URLsTwice a yearQuarterlyTwice a year
500 to 10,000 URLsQuarterlyMonthlyQuarterly
Over 10,000 URLsQuarterlyMonthlyMonthly
After any migrationImmediately, then at 30 daysWeekly for a monthWeekly for a month

Where the Data in This Checklist Comes From

Most technical SEO checklists cite nothing. In fact, of the fourteen we audited, four of the highest-authority pages carry no external statistics at all. So it is worth being explicit about where the numbers on this page came from.

The field observations quoted throughout are drawn from five practitioner sessions, each linked below. Where a figure is a single practitioner’s reported client result rather than a published study, we have said so in the text rather than dressing it up as research.

SourceWhat it contributed to this checklist
AirOps and Jairo Guerrero, Tech SEO Checklist for AI Search VisibilityCrawl budget scaling, the 410 spam fix, log file analysis for AI bots, rendering gaps, accessibility tree, agent commerce, citation without crawl
Ahrefs, A Basic Yet Powerful Technical SEO Audit for BeginnersSearch operator recipes, the four-variant redirect test, exposed file discovery, WordPress footprints, title tag auditing at scale
Ahrefs, SEO Checklist: How to Get More Organic TrafficIntent assessment, subtopic discovery, URL length, readability, internal and outbound linking, link prospecting
Semrush and Nathan Gotch, Comprehensive SEO Audit GuidePosition band analysis, cannibalisation auditing, intent mismatch, content audit decisions, backlink and anchor auditing
Eric Siu, Complete Guide to AI SEO in 2026Impressions up clicks down, multi-platform search, long tail strategy, AI automation of internal linking, citations as a broad category

Google’s own documentation is the source for the Core Web Vitals thresholds, how Google treats 404 and 410 responses and the sitemap size limits, while the AI crawler rules come from OpenAI, Anthropic and Perplexity documentation.

Frequently Asked Questions

What is the difference between a technical SEO checklist and a technical SEO audit?

The checklist is the list of things to check, the audit is the act of checking them on a specific site. A checklist is reusable and generic. An audit produces findings, priorities and owners for one property at one point in time. Use this checklist as the input to your audit template.

How long does a technical SEO audit take?

Between four hours and three weeks, depending almost entirely on site size and log access. A small brochure site running the 50-item version takes a focused morning. A large ecommerce site with faceted navigation, multiple templates and log file analysis realistically takes two to three weeks including the write-up.

Do I need to fix every item on the checklist?

No, and trying to is the fastest way to fix nothing. Clear every Critical item first, because those block crawling, indexing or revenue outright. Then work through High items over the following month. Medium and Low items are batch work you schedule, not emergencies.

Should I block AI crawlers like GPTBot in robots.txt?

Only if you have a specific commercial reason, and even then, block selectively. GPTBot collects training data while OAI-SearchBot powers live answers. Blocking the second one removes you from ChatGPT citations entirely. Most businesses that want AI visibility should allow the retrieval agents and decide separately about the training ones.

Is llms.txt worth adding to my site?

It is cheap to add and currently unproven, so treat it as optional housekeeping rather than strategy. No major AI provider has committed to reading it, and it does not reflect how retrieval actually works. Add it if you want, then spend your real effort on indexation, snippets and rendering.

How do I know if AI systems can actually read my page?

Use view-source rather than inspect element, then check whether your key content is present in the raw HTML. If your headline, price or main answer only appears after JavaScript runs, most AI crawlers will miss it. Follow that with a server log check comparing HTML fetches against JavaScript loads for each named AI agent.

Why do my server logs show no AI crawler visits even though I get cited?

Because many citations never require a visit. AI systems query a search index first and receive a title plus a short snippet for each result. When that snippet answers the prompt, the model writes its answer without fetching your page. Your logs will therefore always under-report AI visibility.

Can an AI agent complete a purchase on my website?

Only if your interactive elements are real semantic HTML. Open Chrome DevTools, view the accessibility tree, and check whether your primary action has a button or link role with a clear accessible name. A styled div reads as an inert rectangle to an agent, so the flow fails silently.

What is the most commonly missed check on this list?

Confirming that the http to https and www redirect applies site-wide rather than only on the home page. We find this on a meaningful share of audits. The root redirects perfectly, every inner URL stays reachable on all four variants, and the whole site quietly runs as duplicates.

Does schema markup help you get cited in AI search?

Not directly. Google’s own documentation on AI features states there is no special schema.org structured data you need to add to appear in AI Overviews or AI Mode. What it does ask is that the page is indexed, eligible to be shown with a snippet, and that its structured data matches the visible text. Accurate schema will not win you a citation on its own, but inaccurate schema can cost you one.

Should I remove FAQ schema now that Google has dropped FAQ rich results?

No need to rip it out. Google stopped showing FAQ rich results on 7 May 2026 and confirmed the markup causes no problems if it stays. Just stop treating it as a deliverable, remove it from new templates, and stop reporting FAQ impressions that no longer exist.

Can you opt your site out of AI Overviews and AI Mode?

Yes. Search Console has a Search generative AI control, under Settings, that excludes your site’s links and content from AI Overviews, AI Mode and generative AI features in Discover. Google says it is not used as a ranking or inclusion signal for other parts of Search, and changes generally take a few days. Page-level controls such as nosnippet, data-nosnippet, max-snippet and noindex also limit what Google shows.

No. Google’s guide to optimizing for generative AI features, added in May 2026, says there is no requirement to break content into tiny pieces for AI to understand it, and that there is no ideal page length. Write sections that answer their heading directly, because that helps readers and snippets, but do not split pages into fragments for AI.

What should a technical SEO checklist include?

A technical SEO checklist should cover setup and access, crawlability, indexation, robots.txt, canonicals, site architecture, rendering, speed, security, mobile, template-level on-page elements, structured data and server logs. In 2027 it also needs AI crawler access, AI search visibility and agent readability. This one adds keyword, content and link diagnostics, for 105 checks in total.

Is a 410 better than a 404 for SEO?

Not for Google rankings. Google’s HTTP status code guide (checked October 29, 2026) treats every 4xx code except 429 the same way, and Google’s John Mueller said in 2024 that the difference is too small to matter for SEO. Still, use 410 when a removal is deliberate, because it reads more clearly in logs, and never block junk URLs in robots.txt instead.

Do you need hreflang checks in a technical SEO audit?

Only if your site has versions of the same page for different languages or countries. In that case, check that every version lists itself and all the others, that each pair links both ways, that URLs are fully qualified and that an x-default page exists for unmatched visitors. Otherwise, single-language sites can skip hreflang entirely.

Does blocking Google-Extended remove you from AI Overviews?

No. Google says Google-Extended controls whether your content is used to train Gemini models and to ground answers in the Gemini apps and Vertex AI, and that it has no effect on Google Search. Instead, AI Overviews and AI Mode follow your Googlebot rules, your snippet controls and the Search generative AI control in Search Console.

Does crawl budget matter for a small website?

Rarely. Google’s crawl budget guide (checked October 29, 2026) is written for sites with more than a million pages that change weekly, sites with more than 10,000 pages that change daily, or sites with many URLs stuck in Discovered, currently not indexed. For smaller sites, Google says an up-to-date sitemap and regular Page indexing checks are enough.

Where to Start This Week

Pick three things. Run the four-variant redirect test on a deep URL. Also run the filetype searches for exposed documents. Then open your robots.txt and check each AI user agent line individually. Those three take under an hour combined and they surface the most expensive problems in this list.

After that, work the Critical items in order, then move to High. Do not start at item one and grind forward, because the numbering is a category order, not a priority order.

If you want a second pair of eyes on the results, we run this exact 105-point process for clients. Grab a Free ROI Audit and we will send you the findings with owners and priorities attached.

Ishant

Ishant Sharma is the Founder and CEO of Hustle Marketers, a Google Partner digital marketing agency. Working in Google Ads, Meta Ads, SEO and ecommerce PPC since 2013, he has helped 2,500+ brands generate $780M+ in trackable sales. Upwork Top Rated Plus with 100% Job Success Score. Ishant Sharma is the digital marketing specialist, not the Indian cricketer of the same name.

I hope you enjoy reading this blog post. If you want my team to just do your marketing for you, click here.
Scroll to Top