Chapter in one paragraph
When an AI system answers a buyer's question, it does not read your whole page and admire it. It hunts for the specific passage that answers the question and lifts it. Extraction is the layer that makes real answers liftable: lead with the answer, write self-contained passages, let the right crawlers in, put the content in the HTML, and use schema only where it genuinely clarifies. No special file or markup gets you cited. Clarity and access do. This chapter gives you the six-check diagnostic and the moves, in priority order.
The outcome is Extraction. The mechanism is what most people loosely call AI discoverability. This chapter is about a narrow, specific question: when the machine has decided your business deserves to be in the answer, can it actually pull your answer out cleanly and cite you for it?
Authority, the last layer, was about whether you deserve to be the answer. This layer assumes you do. The substance is real, the coverage is complete, the depth is there. None of that guarantees you get quoted. A genuinely expert page can be passed over because its answer is buried under four paragraphs of warm-up, because the crawler was blocked, because the content only appears after JavaScript runs, or because the one good sentence cannot survive being lifted out of the page. Extraction fixes the gap between deserving the citation and getting it.
The spine of this book is Understand, then Trust, then Recommend. Foundation made you crawlable. Trust gave the machine corroboration. Authority gave it substance worth returning to. Extraction makes that substance liftable. It is the least glamorous layer and the one with the most snake oil sold around it, so I am going to be blunt about what works and what is being oversold.
One thing before the method, because it governs the entire chapter.
What This Layer Solves
Start with how the machine actually reads. When a buyer asks an AI system a question, the system fans that question out into many sub-questions and assembles a single answer from sources across all of them.[1] You met this in the last chapter. The part that matters for Extraction: for each sub-question, the system is not reading your whole page and admiring it. It is hunting for the specific passage that answers that sub-question, and lifting it.
Google has been able to do this at the passage level for years. In 2020 it announced a ranking change that, in its own words, let it "better understand the relevancy of specific passages" so it could "find that needle-in-the-haystack information you're looking for."[2] AI answer engines do the same thing, harder. They read your page, find the chunk that answers the question in front of them, and quote or paraphrase that chunk.
That changes what "good content" has to do. A page can be comprehensive and still lose if its answers are buried. Picture two sources answering "how much does a fractional CFO cost." One opens the section with: "A fractional CFO typically costs between X and Y per month, depending on scope and company stage." The other opens with three hundred words about the history of financial leadership before it gets to a number. The machine lands on both and finds the answer instantly in the first and not the second. The first gets quoted.
The second wrote a better essay and lost the citation.
So Extraction is the discipline of writing and structuring so that the answer to each question a buyer asks sits right where the machine can find it, in a form it can lift without dragging in the surrounding clutter.
Now the part you will not hear from most people selling AI visibility. Extraction is not a set of secret files you add to trick the machine into citing you. Google's own documentation on AI features is explicit: "There are no additional requirements to appear in AI Overviews or AI Mode, nor other special optimizations necessary," and "You don't need to create new machine readable files, AI text files, or markup to appear in these features. There's also no special schema.org structured data that you need to add."[3] Read that twice, because an entire cottage industry is selling the opposite.
So if there is no magic file, what is the work? Clarity. You write the answer first. You make each passage self-contained. You make sure the crawlers can reach the page and the content is actually in the HTML. You use structured data where it genuinely removes ambiguity, knowing it is an aid and not an entry ticket. That is the whole layer: unglamorous, durable, and impossible to fake, which is exactly why it survives in a world where gaming does not.
And it feeds the same loop every layer feeds. A clean, extractable answer becomes a citation, the citation puts your business in front of a buyer at the moment of decision, and that buyer arrives warm because the machine vouched for you before the click.
The Extraction Diagnostic: Six Checks
Run this against your most important commercial pages before you change anything. Six checks. The output is your Extraction readiness.
Check 1. Does each section lead with its answer?
Take your top commercial page and read the first sentence under each heading. Is the direct answer there, or is it three paragraphs down after the setup? Fail:sections that open with context, history, or marketing and bury the actual answer in the middle. The machine reads top-down and lifts the first clean answer it finds. If yours is buried, it lifts someone else's.
Check 2. Are your passages self-contained?
Pick any paragraph that answers a real buyer question. Now imagine it quoted on its own, with no page around it. Does it still make sense? Fail:paragraphs that lean on "as mentioned above," "this," "that approach," or a pronoun whose subject is two sections back. A passage that only makes sense in place cannot be lifted out, and lifting it out is exactly what the machine does.
Check 3. Can the AI crawlers actually reach the page?
Open your robots.txt. Look for blanket disallows and for named AI user agents. Fail: a robots.txt that blocks the crawlers you actually want reading you, or a default-deny posture you set once and forgot. You cannot be cited by a system whose crawler you turned away at the door. How to decide who to let in is Move 3; the diagnostic step is just to know what your robots.txt currently says, because most owners do not.
Check 4. Is the content in the HTML, or only after JavaScript?
View the page source, or fetch it the way a simple crawler would, with JavaScript off. Is the real content there, or is it a near-empty shell that fills in only when scripts run? Fail: answers that exist only in client-rendered JavaScript. Some crawlers render, many do not, and the ones that do not see an empty page where your answer should be. This is a Foundation issue that bites hardest at the Extraction layer.
Check 5. Do your headings match the questions buyers ask?
Look at your H2s and H3s. Are they the literal questions a buyer types, or are they clever phrases? Fail:a section that answers "how much does it cost" but is titled "Investment Philosophy." The machine matches sub-questions to content. Headings that are the question are the easiest possible match.
Check 6. Where you use structured data, does it clarify rather than decorate?
If you use schema, check that it describes what is genuinely on the page (your organization, the author, the article, the product) and that it does not contradict the visible content. Fail: schema bolted on as a supposed ranking trick, schema that disagrees with the page, or FAQ schema added in the belief it still earns rich results for ordinary sites. It does not, and Move 5 explains why. Pass: honest entity markup that helps a machine disambiguate who you are.
Score
Six of six is an Extraction pass. One point per check passed.
Four to five is a partial worth fixing before you write anything new.
Three or under means your real answers are effectively hidden from the systems that would otherwise quote them.
The Core Moves
Six moves, in priority order. The order is deliberate: the first two are free, durable, and account for most of the result. The later ones matter less than the people selling them claim.
Move 1. Lead with the answer
This is the highest-ROI move in the layer and it costs nothing but discipline. For every question your page addresses, put the direct answer in the first sentence or two of its section. Then explain, qualify, and expand underneath.
This is the inverted pyramid, the structure journalists have used for a century and that usability research confirms readers prefer: "The most important information (or what might even be considered the conclusion) is presented first."[4] It serves the human who is scanning and the machine that is extracting, because they want the same thing. Answer first, support second.
Concretely, rewrite section openers. "Strategic financial leadership is a journey that..." becomes "A fractional CFO costs between X and Y per month." The journey can come after. Do this across a page and you have done the single most effective thing in this chapter.
Move 2. Write atomic, self-contained passages
One question per section. One idea per passage. And each passage able to stand alone if it is lifted out, because it will be.
Atomic means a reader, or a machine, can drop into that section cold and get a complete answer without needing the three sections above it. Kill the dependencies. Replace "as we discussed, this approach saves money" with "Outsourcing bookkeeping saves a small business roughly X per year." Name the subject in the sentence. Avoid pronouns that point off-screen, and avoid "above," "below," and "the previous section." Those words are fine for a human reading in order and fatal for a passage being quoted on its own.
This is not dumbing down. A page built from clear, self-contained, answer-first passages is a better page for a human too. The machine is just a brutally literal reader, and writing for it well means writing clearly, which you should have been doing anyway.
Move 3. Let the right crawlers in, on purpose
You cannot be cited by a system you have blocked. AI companies publish named crawlers, and you control them in robots.txt. The point is to make a deliberate choice, not a default one.
The current state of play, as of 2026:
OpenAI
GPTBot (model training), OAI-SearchBot (surfacing your site in ChatGPT's search features), and ChatGPT-User (live retrieval when a user asks ChatGPT something that makes it visit your page).[5]
Anthropic
ClaudeBot (training), Claude-SearchBot (search indexing), and Claude-User (live retrieval for a user's request). Older tokens you may see in stale tutorials, anthropic-ai and Claude-Web, are deprecated.[5]
Perplexity
PerplexityBot (search surfacing, respects robots.txt) and Perplexity-User (user-triggered retrieval, which is generally not governed by robots.txt because a person asked for it).[5]
Google-Extended, which controls whether your content trains Gemini and grounds its answers. Google states plainly that Google-Extended "does not impact a site's inclusion in Google Search nor is it used as a ranking signal."[5]
The move is to decide, not to default. If you want to appear in AI search and answers, allow the search and retrieval crawlers. If you have a specific reason to opt out of model training, you can block the training crawlers while still allowing the retrieval and search ones. The mistake is a blanket disallow set once in a panic, or a robots.txt nobody has read in two years, quietly excluding you from the surfaces you are trying to win.
Founder translation
Ask whoever manages your site whether anything in robots.txt or a security plugin is blocking AI crawlers; one line set in a privacy reflex can erase you from ChatGPT and Perplexity answers.
Consultant move
Read the live robots.txt, confirm the search and retrieval agents (OAI-SearchBot, Claude-SearchBot, PerplexityBot) are allowed, and fetch a key page with JavaScript off to verify the answer is in the HTML.
Move 4. Put the content in the HTML
If your answer only exists after JavaScript runs, you are betting your citation on whether each crawler renders. Many do not. Make the content present in the server-rendered HTML so a fetch of the page returns the actual answer, not an empty shell.
For most sites on a modern framework this means server-side rendering or static generation for content pages, not client-only rendering. Test it the simple way: fetch the page with JavaScript disabled and confirm your answers are in the response. If they are not, no amount of answer-first writing helps, because the reader you are writing for never receives the words.
Move 5. Use structured data as a clarity aid, not a magic input
Here is where honesty separates this book from the pitch decks. Structured data helps machines understand a page. Google describes it as giving "explicit clues about the meaning of a page," and it makes you eligible for certain enhanced displays.[6] It is genuinely useful for disambiguation: Organization schema that says who you are, Person schema that ties an author to an identity, Article schema that marks up a piece of writing. Use those where they remove real ambiguity about your entity. That is sound work.
What structured data is not: a requirement for AI citation, or a ranking trick. Google's AI-features documentation says directly that there is "no special schema.org structured data that you need to add" to appear in AI Overviews or AI Mode.[3]And the specific schema most people reach for first, FAQPage, no longer does what they think. In August 2023 Google limited FAQ rich results to "well-known, authoritative government and health websites," and stated that for all other sites "this rich result will no longer be shown regularly."[7] If you are not a government or health authority, adding FAQ schema to chase rich results is chasing a result that was retired. Mark up your entities honestly and skip the schema theater.
Move 6. llms.txt, the honest version
You will be told you need an llms.txt file. You almost certainly do not, and here is why, so a vendor cannot sell it to you as the answer.
llms.txt is a real proposal. Jeremy Howard of Answer.AI introduced it in September 2024 as a markdown index of your important content, a file "designed for reading by language models, not just humans."[8]It is a reasonable idea and it costs little to publish. But as of 2026, the major search and AI crawlers do not use it. Google's John Mueller said as much publicly, comparing it to a site simply asserting "I have the best website ever" and listing its own pages.[8] Google has indicated it will not crawl and use llms.txt for ranking, and no major AI lab has publicly confirmed using it for discovery.[8]
So the honest position: if you maintain developer documentation or content aimed at AI coding assistants, an llms.txt file can help that specific audience, and publishing one is cheap. But do not let anyone tell you it is what gets your business cited by ChatGPT or Google. The thing that gets you cited is everything in Moves 1 through 4. Spend your effort there.
The Walkthrough: One Page, One Focused Day
A real extraction pass on a single high-value page. The business: a B2B advisory firm, the same kind of firm from the last chapter. The page: its cornerstone guide on a core service, genuinely comprehensive, written by someone who knows the subject cold. Despite the depth, it was almost never quoted by AI systems when the firm tested the buyer questions it should have owned. The Authority was real. The Extraction was broken. Initial Extraction diagnostic: two of six.
Step 1. Map the page to its sub-questions (an hour)
List the buyer sub-questions the page should answer. For this guide, about twelve. Then find, for each one, the exact place on the page where it is answered. Two patterns showed up immediately. Some sub-questions were answered, but the answer was buried mid-section under setup. A few were not answered cleanly anywhere, only implied across paragraphs.
Step 2. Rewrite every section to lead with its answer (half a day)
For each of the twelve sub-questions, find its section and move the direct answer to the first sentence. The history, the nuance, the caveats stay, but underneath the answer, not in front of it. This is the highest-impact half-day in the whole layer. The page said the same true things as before. It just said the conclusion first now.
Step 3. Make the passages atomic (a few hours)
Go through and cut the dependencies. Every "as noted above" gets the noun written back in. Every pronoun pointing off-screen gets its subject restored. Each section is read cold, on its own, and edited until it stands alone. The test is simple: copy any section, paste it into a blank document, and check that it is a complete answer with no missing context.
Step 4. Fix the plumbing (an hour)
Check robots.txt. The firm had, at some point, added a blanket disallow for one AI crawler in a vague privacy reflex, with no decision behind it. Resolve it deliberately: allow the search and retrieval crawlers it wants citations from, keep any training opt-out it actually means. Then confirm the page renders server-side, the content in the HTML with scripts off. Here it was fine, but on many sites this is where the citation was leaking the whole time.
Step 5. Honest schema (an hour)
Add or correct Organization and Person markup so the machine can tie the page to the firm and the author as entities. Remove the FAQ schema someone had added expecting rich results that no longer come. Validate that the structured data matches the visible page and contradicts nothing.
Step 6. Re-test the fan-out (ongoing)
Re-run the same twelve buyer sub-questions against the AI systems on a cadence and watch whether the firm starts appearing in the answers it restructured for. This is the measurement loop, and it is the only honest scoreboard for this layer.
The whole pass is roughly a focused day for one important page. Nothing about the firm's expertise changed. The answers were as correct after as before; they simply became liftable.
Common Mistakes
Mistake 1. Burying the answer
The most common and most costly. Pages that open every section with setup, context, or a marketing wind-up, and hide the answer in the middle. The machine reads the top of the section, does not find the answer, and lifts a competitor who led with it.
Mistake 2. Believing a file or a tag is what gets you cited
The overclaim trap, and it is everywhere right now. Teams add llms.txt, bolt on schema, and believe they have "done AI visibility," while their answers stay buried and their crawlers stay blocked. Google says plainly that no special files or schema are required.[3] The work is clarity and access, not artifacts. A clean, answer-first, crawlable plain-HTML page with no special files at all beats a site with perfect schema and buried, blocked, JavaScript-only answers.
Mistake 3. Blocking the crawlers by accident
A blanket robots.txt disallow, a default-deny set in a privacy panic, a security plugin that quietly excludes bots. Then the team wonders why the business never appears in AI answers. You cannot be quoted by a crawler you turned away. Read your robots.txt and make the choice on purpose.
Mistake 4. JavaScript-only content
The answer exists, but only after scripts run, and the crawler that did not render sees an empty page. This one hides well because the page looks perfect in a browser. Test with JavaScript off; if your answers vanish, so do your citations.
Mistake 5. FAQ schema cargo-culting
Adding FAQPage markup to ordinary business pages expecting the rich results that Google retired for non-authoritative sites in 2023.[7] At best it does nothing. At worst it is FAQ markup stuffed with marketing copy that contradicts the page. Mark up entities honestly. Do not perform schema you do not need.
Mistake 6. Context-dependent passages
Writing that reads fine top to bottom but falls apart the moment a single section is lifted out. "This is why we recommend it." Why is what? Recommend what? A human reading in order knows. A machine quoting the passage alone does not, and either drops you or quotes you saying something that makes no sense without the page. Write every passage to survive on its own.
Case Study: Real Authority, Unliftable
The client: a professional-services firm with a flagship guide that was, by any honest read, the best resource on its subject in its niche. Written by the founder, drawing on years of real practice. The expertise was not in question. But when the firm tested the buyer questions it should have owned, AI systems quoted thinner competitors and skipped the firm entirely.
The founder's framing was the one I hear constantly at this layer. "We have the best content and we are invisible in AI." That is the signature of an Extraction failure. Real authority, unliftable.
What the diagnostic found
Extraction score: failing. Answers buried under long section intros. Passages laced with "as above" and dangling pronouns. A robots.txt that, on inspection, was turning away crawlers the firm wanted citations from. Headings written as clever phrases, not buyer questions. FAQ schema present and doing nothing.
The intervention
No new content. The page already had the answers. We rewrote every section to lead with its answer, made each passage atomic and self-contained, rewrote headings as the literal buyer questions, resolved the robots.txt to allow the crawlers the firm wanted, confirmed the answers were in the server-rendered HTML, and replaced the decorative schema with honest Organization and Person markup. A focused day or two on one page.
The lesson is the cleanest in the book. We added no expertise and we removed none. We made the expertise that was already there easy for a machine to lift and quote.
That is the boundary this layer keeps. The firm owned the answer. I made the answer extractable. I cannot make a hollow page citable, and I would not try, because the machines are now good enough that the hollow page fails anyway. Extraction is the honest layer: it rewards real answers, clearly written, openly served, and nothing else.
Make the answer real. Then make it impossible to miss.
Frequently Asked Questions
What is the Extraction layer of AI visibility?
Extraction is the layer that makes real answers easy for AI systems to lift and quote. When a buyer asks an AI system a question, it fans the question out into sub-questions and hunts for the specific passage that answers each one. The unit of citation is the passage, not the page. Extraction closes the gap between deserving the citation and getting it: answer-first sections, self-contained passages, crawler access, content in the server-rendered HTML, and honest entity markup.
Do you need llms.txt or special schema to appear in AI answers?
No. Google's documentation on AI features states there are no additional requirements to appear in AI Overviews or AI Mode: no new machine-readable files, AI text files, or special schema.org structured data. llms.txt is a real proposal, but as of 2026 the major search and AI crawlers do not use it. The work that earns citations is clarity and access, not artifacts.
How do I check whether AI systems can extract my content?
Run the six checks above: does each section lead with its answer, are passages self-contained, can the AI crawlers reach the page in robots.txt, is the content in the server-rendered HTML, do headings match the literal questions buyers ask, and does any structured data clarify rather than decorate. Six of six is an Extraction pass. Three or under means your real answers are effectively hidden.
Which AI crawlers should I allow in robots.txt?
Decide on purpose rather than by default. If you want to appear in AI search and answers, allow the search and retrieval crawlers: OAI-SearchBot and ChatGPT-User (OpenAI), Claude-SearchBot and Claude-User (Anthropic), and PerplexityBot and Perplexity-User (Perplexity). You can block the training crawlers while still allowing retrieval and search. Google states that Google-Extended does not impact a site's inclusion in Google Search and is not used as a ranking signal.
Sources
- Google Search Central, AI Features and Your Website, last updated December 10, 2025; and Google, AI in Search: Going beyond information to intelligence, The Keyword, May 20, 2025. Both describe the query fan-out technique: AI Overviews and AI Mode issue multiple related searches across subtopics to develop a response.
- Prabhakar Raghavan, How AI is powering a more helpful Google, The Keyword (Search On 2020), October 15, 2020. The passage-ranking announcement quoted in this chapter.
- Google Search Central, AI Features and Your Website, last updated December 10, 2025. Verbatim: "There are no additional requirements to appear in AI Overviews or AI Mode, nor other special optimizations necessary."
- Amy Schade, Inverted Pyramid: Writing for Comprehension, Nielsen Norman Group, February 11, 2018.
- AI crawler documentation from each operator, accessed June 2026: OpenAI, Bots documentation; Anthropic, crawler support article; Perplexity, Perplexity Crawlers documentation; Google, Google common crawlers (Google-Extended).
- Google Search Central, Intro to How Structured Data Markup Works, accessed June 2026.
- Google Search Central Blog, Changes to HowTo and FAQ Rich Results, August 8, 2023.
- Jeremy Howard, The /llms.txt file, Answer.AI, September 3, 2024 (spec at llmstxt.org). Google's stance: Search Engine Journal, coverage of John Mueller's comments, June 15, 2026, and Search Engine Land, Google says llms.txt will not be used, July 24, 2025.
Get the Full Book First
A new chapter publishes here every week. Part 3, the operating manual, is not being serialized. The list gets the complete book free the day it launches.
Next: Chapter 6 · Layer 5 · publishes August 8
Recommendation
The layer that exists only because answer engines do: turning satisfied customers into corroborated proof.
Run Your Visibility Report
The six-check extraction diagnostic done for you, plus what to fix first. No commitment.
Run Your Visibility Report