The Agentic Paradigm: what to do, and what not to do

Reframing the question

Tokens are an input measure, and input measures are the wrong unit for a commercial decision. Part I's position holds here: the board metric is cost per completed task, and the market has already moved to it — OpenAI frames GPT-5.6 as “outcome per dollar,” and Artificial Analysis now publishes weighted cost-per-task rather than headline price. A team that halves its tokens by truncating context and then triples its rework has improved the metric you asked about and damaged the one that pays the bill.

That is not a reason to refuse the question. A proposal needs a number and “it depends” is not a deliverable. The honest position: token spend is estimable to a wide band by construction, and to a useful band only by measurement. Say which of the two you are handing over.

How to actually do it

Build the estimate bottom-up and make every multiplier explicit, because the multipliers — not the base rate — are where estimates die:

cost ≈ (tasks in scope) × (agent turns per task) × (avg tokens per turn) × (price per tier) × (retry/rework multiplier) × (eval & regression multiplier) + (human review time cost)

Term	What it really means, and where it goes wrong
Tasks in scope	Countable units of completed work — a migrated file, a resolved ticket, a reconciled invoice. If you cannot count it, you cannot price it. Scope creep here is linear and visible; everything below it is not.
Agent turns per task	The tool calls, retrievals and self-corrections inside one task. The single most under-estimated term. A “simple” task that takes twelve turns costs twelve times a one-shot prompt.
Avg tokens per turn	Split it four ways: fresh input, cached input, output, and reasoning tokens. Input dominates in agentic loops because the whole context is re-sent every turn. Reasoning tokens are billed as output and are invisible in the transcript.
Price per tier	Your actual routing mix, not the flagship rate. As of 30 July 2026, Luna is $0.20/$1.20 and Terra $2/$12 per 1M tokens; Sol is unchanged, and Fast mode buys 2.5× speed at 2× price (OpenAI).
Retry/rework multiplier	Failed runs, re-prompts, and downstream defect remediation. Often larger than 1.5× and sometimes larger than the base cost itself.
Eval & regression multiplier	Every eval suite run, every regression sweep on a model change. Teams forget this entirely, then discover evals cost more than production.
Human review time cost	Loaded hourly cost × review minutes per task. On regulated work this frequently exceeds the token line by an order of magnitude, and it is the term that cheap-model routing inflates.

Cache-hit rate is the biggest single lever, and it is a design choice, not a given. Blitzy's numbers on the GPT-5.6 launch page illustrate it: cache hit rate from 24% to 90%, with 8.5× fewer output tokens, 2.2× more context, and 87% lower cost against GPT-5.4 mini (OpenAI). Treat that as a vendor-selected best case — Part I is explicit that these are marketing metrics, not field results — but the mechanism is real: stable prompt prefixes and stable tool schemas are worth more than model choice.

Then calibrate by measurement. Run a one-to-two week instrumented spike on a representative slice of the real workload, in the real codebase, against the real retrieval corpus. Capture tokens by category per completed task, turns per task, cache-hit rate and human-intervention rate, then extrapolate with a stated band. My working heuristic — a judgement call, not a measured industry figure — is ±2–3× on a first estimate from one spike, tightening to roughly ±30% after two measured sprints on the same workflow class. A point estimate with no band is a commitment you did not intend to make.

The trade-offs

Fixed-price versus time-and-materials is the real decision, and it is a question of who eats the variance. Fixed price hands the client certainty and hands you an uncapped tail on a cost driver whose pricing you do not control. T&M is honest and sells badly against competitors quoting fixed. The perverse incentive is what will actually happen: to win the bid, someone assumes three agent turns per task where the spike showed nine — invisible in the proposal, fatal in delivery.

Hard caps trade cost certainty for quality: an agent cut off at a turn budget returns a partial answer that looks complete. Cheap-tier routing lowers the token line and raises the review and defect line — cost moved from a budget you report on to one you do not.

Caveats — where this breaks

Prices move faster than engagements. The 30 July 2026 cuts repriced two of three tiers in a day; any estimate has a shelf life measured in months, not years. Jevons is the second trap: on Ramp data cited by Artefact, average cost per million tokens fell from about $10 to about $2.50 in a year while total enterprise bills rose. Third, the two cost curves point in opposite directions: fixed-capability inference falls roughly 10× a year (a16z, LLMflation), while the cost of running frontier capability rises roughly 18× a year (Epoch AI). If your workflow needs the frontier, you are on the rising curve.

What would change this answer

• Independent, non-vendor field data on tokens per completed task by workflow class — nothing published today is that.
• Cache-hit rates that stay stable across model and prompt revisions, rather than resetting on every upgrade.
• Contractually stable pricing from a provider — price-lock or deprecation-notice terms you can actually rely on.
• Two quarters of your own measured history. That single input beats every external benchmark.

Reframing the question

The deliverable is not a summary. It is a decision record with provenance: what was decided, by whom, by when, dependent on what, disputed by whom — each item traceable to a timestamp, a speaker and a modality. A transcript is one modality and the weakest one, because meetings encode meaning in artefacts, screens and silence as much as in speech. Better speech recognition does not fix a system listening to the wrong channel.

How to actually do it

Treat it as a pipeline, not a tool purchase.

1. Audio → ASR with speaker diarization. Keep speaker attribution. “Who committed to this” is the load-bearing fact; an unattributed action item is not a decision record.
2. Screen and slide capture → keyframe sampling + OCR. Most numeric and architectural truth in a meeting is on the shared screen and is never spoken aloud.
3. Artefact harvesting. Pull the deck, the Jira/ADO ticket, the pull request, the design file, the spreadsheet actually under discussion, and join them to the meeting timeline.
4. Ambient and system context. Calendar invite, attendee roles, the previous meeting in the series, and the ticket state immediately before and after.
5. Structured extraction to a typed schema — decision, owner, due date, dependency, risk, open question, dissent — not prose. Typed output is checkable and diffable; a prose summary is neither.
6. Verification pass. Every claim carries timestamp + speaker + modality citation, and a second pass attempts to refute each claim against the artefacts. Unsupported claims are flagged, never silently dropped.
7. Human confirmation at the point of consequence. Anything with money, legal, staffing or commitment implications gets a named human confirming. This is Part I's gate logic applied unchanged: keep the gate where error is irreversible, rights-affecting or regulated.

Modality	What only it captures
Transcript + diarization	Commitments, attribution, tone of objection, the verbal “we decided”
Screen / slide OCR	Numbers, dates, architecture diagrams, config values — usually never spoken
Chat sidebar	Dissent people would not voice, links, corrections to what was said aloud
Linked artefacts	Ground truth for what actually changed; the refutation set for the verification pass
System context	Authority — whether the person who committed was entitled to commit
Absence of speech	Unregistered dissent; nobody objecting is not the same as agreement

What transcript-only reliably misses: the figure on screen, the diagram, the disagreement expressed by silence, the decision that happened in chat, cross-talk, and Indian-English and code-switched speech on domain nouns and proper names — where ASR error concentrates precisely on the product names, acronyms and person names that carry the meaning. It also cannot distinguish “we discussed X” from “we decided X,” which is the difference the whole record exists to capture.

Accuracy techniques worth naming: domain lexicon and hotword biasing for product names and acronyms; diarization confidence thresholds that mark low-confidence attribution rather than guessing; required schema fields so gaps are visible as gaps; evidence spans rather than summaries; running extraction twice, diffing, and treating disagreement between passes as a review signal rather than picking a winner.

The trade-offs

Recall against precision is the governing tension. A high-recall system captures everything, buries the reader, and becomes decorative. A high-precision system keeps only confident items and silently loses the hedged, half-voiced dissent that was the most valuable thing in the room. Automation trades against candour: people who know every meeting is mined stop saying the risky true thing on the record. Retaining raw video buys evidentiary value and buys a storage bill and a discovery liability. And a record produced in five minutes gets used; one produced next Tuesday does not.

Caveats — where this breaks

Take the legal exposure seriously; it is larger than the engineering exposure. Recording and mining meetings involving employees engages DPDP data-fiduciary obligations — Rules notified 13–14 November 2025, phased over roughly eighteen months to about May 2027, with the Data Protection Board now constituted — including notice and purpose limitation. Then the reclassification risk: if extracted meeting signal is used for performance evaluation, task allocation, promotion or termination, that is employment-related AI, which the EU AI Act classifies as high risk under Annex III, carrying Article 14 human oversight obligations. The Digital Omnibus defers those Annex III obligations to 2 December 2027, but Article 50 transparency obligations still bite from 2 August 2026, and if the Omnibus is not formally adopted the original timeline applies. Add cross-border transfer exposure where the GCC's parent processes recordings abroad, and works-council or union sensitivity in European group entities and with KITU in Karnataka. This is a structural flag, not legal advice on your facts; take advice before deployment.

What would change this answer

• Measured extraction precision and recall on your meetings, your accents, your product vocabulary — not vendor benchmark numbers.
• An explicit employee consent and notice architecture that survives a regulator reading it.
• Hard separation between the operational decision register and anything touching HR decisions, enforced by access control rather than policy language.
• Independent evidence that candour is unaffected — measured, for example, by whether risks still get raised on the record after deployment.

Reframing the question

The temptation is to redraw the org chart. Resist it. Every role on that list still exists in an agent-heavy team, and an organisation that abolishes them will spend two years reinventing them under new names. What changes is the ratio between roles, the unit of work each owns, and — decisively — where the constraint sits.

In conventional Agile the constraint is generation capacity: who can write this, and how many of them do we have. When agents absorb a meaningful share of generation, the constraint moves to two places at once — who can specify the work precisely enough that an agent produces the right thing, and who can review the output credibly enough to sign it. Staffing to the old bottleneck is the error: it produces teams that generate far more than they can verify. It also breaks skill mapping by language, which was only ever a proxy for generation capability; review capability turns on systems knowledge, domain knowledge and familiarity with this codebase.

How the roles actually change

The source report is explicit: the shift is from code production to "specification, review, and evaluation," with evaluation engineering and context engineering as first-class disciplines rather than side duties.

Agile role	Unit of work it now owns	What breaks if you don't change it
Product Owner	Outcome specification; acceptance criteria precise enough to be machine-checkable	Vague criteria used to cost a conversation; they now cost a wrong implementation, delivered fast, that looks finished
Business Analyst	Specification engineering, context curation, evidence quality — the biggest winner	Agents work from stale ground truth. Specifications become more valuable than the code they generate
Technical Architect	Constraints, invariants, the review rubric — owner of the "what must never happen" list	Agents optimise locally. Without written invariants, architectural drift accumulates silently and surfaces in production
QA	Evaluation engineering — eval suites, thresholds, regression corpora, adversarial cases	Test execution is what agents automate best. QA that does not move up goes redundant while quality goes unmeasurable
Developers	Specification, review and integration discipline; generation is now the cheap step	Review capacity, not typing speed, sets throughput. Optimising generation moves the queue, not its length
Scrum Master	Exception queue and agent-ops cadence as planned work	Exception handling is absorbed as invisible overtime by whoever notices first

On developers, be honest about the evidence. METR's July 2025 RCT found experienced developers 19% slower with AI on mature codebases while believing they were 20% faster; the February 2026 reassessment complicates it, with a subset showing roughly 18% speedup. The honest reading — contested and context-dependent — is itself the planning-relevant fact: you cannot staff against an assumed uniform productivity multiplier, because none is established.

Four functions need names and budget, not duties bolted onto existing jobs: evaluation engineering; context engineering; agent ops — workflow design, eval thresholds, exception queues, which the source identifies as the re-skilling destination for middle management; and AI governance in legal, compliance and risk, a function the source says grows. Domain experts become the humans-in-the-loop and the source of proprietary context.

On balancing traditional against AI-native competence, the counter-intuitive point is the important one: deep systems and domain skill becomes more valuable, not less, because reviewing generated code takes more context than writing it did. So do not hire "prompt engineers" as a standalone role — isolated from domain and systems knowledge they elicit output nobody can validate — and do not assume AI fluency substitutes for codebase familiarity.

The junior question is a resourcing decision, not a philosophical one. Stanford's "Canaries in the Coal Mine?" (Brynjolfsson, Chandar & Chen, 2025, ADP payroll microdata) finds a 13% relative employment decline for ages 22–25 in the most AI-exposed occupations, ~16% through October 2025. But today's juniors are the only source of tomorrow's credible reviewers, and the review bench is what the operating model runs on. The source calls the fix — entry-level roles redesigned into agent-supervised apprentices who review, test and reject agent output rather than produce first drafts — the single most important org-design decision available.

The trade-offs

Pyramid shape versus margin. Services economics run on leverage. A flatter, more senior pyramid is more capable per head and worse on cost per head — in a competitive rate environment that gap is the gross margin. You are choosing which risk to carry.

Billability. Eval design, context curation and regression-corpus maintenance are real effort many clients will not accept as a line item. Price them into the rate card, fund them as investment, or they quietly do not happen.

Specialist versus generalist; central versus embedded. Named eval and context functions build depth but concentrate dependency. A central CoE gives standards and a defensible governance story; embedded practitioners give adoption. The failure mode of the first is gatekeeping, of the second, fragmentation.

Caveats — where this breaks

The largest non-technical exposure is Indian labour law. If skill mapping becomes de facto retrenchment, the four Labour Codes in force since 21 November 2025 apply: IR Code §70 (notice plus 15 days' average pay per completed year of service), §77 (prior government permission at 300+ workers, states able to lower it), and §83's new Worker Re-skilling Fund — 15 days' wages per retrenched worker, over and above §70. For Bengaluru, Karnataka's IT/ITeS Standing Orders exemption, extended by Notification No. LD 328 LET 2023 to 10 June 2029, expressly gives way to the IR Code, with the KITU writ pending. Redeployment with reskilling is the lower-liability path, and aligns with §83's own logic.

Not legal advice. Thresholds and dates are current as of 31 July 2026 and should be checked against the bare Acts, the applicable state notification under §77, and Gazette notifications before reliance.

What would change this answer

• Independent, non-vendor field data showing agentic task-success above roughly 90% with flat defect rates. That would justify a leaner review tier; nothing today does.
• Your own first-pass acceptance rate, high and stable for two quarters on an account — then the review ratio there can safely fall.
• Contracts that price specification and evaluation explicitly, turning a cost centre into billable capability and changing the pyramid maths.
• A judicial or legislative outcome on the Karnataka Standing Orders position, which shifts the cost of restructuring versus redeployment.
• Evidence that apprentices reach independent sign-off faster under agent supervision — that argues for a larger junior intake, not a smaller one.

Reframing the question

Most estimation debates about AI start from the wrong premise — that the job is to find the discount factor. Agents change the shape of the work, not just its size. Generation time collapses; review, integration and validation time does not, and in places rises; and variance goes up sharply, because an agent's output on a given task is far less predictable than a known engineer's.

A team five times faster at generation and one times faster at review has not become five times faster. It has moved the bottleneck and made the schedule less predictable. So split effort by activity type rather than applying one multiplier to a story-point total — otherwise the savings show up in the plan and the slippage stays invisible until integration.

How to actually estimate it

A proposed working framework, not an industry standard. Its value is that it makes the omitted terms explicit; the numbers must come from your measurement, not this page. Classify each backlog item into an automation tier at grooming — the tier, not the story-point value, drives the effort profile.

Tier	Typical work	Effort profile and estimation caution
A — agent-executable, light review	Boilerplate, CRUD, scaffolding, unit tests, migrations, documentation	Largest generation saving, smallest review tax. Safe to plan aggressively — but grooming over-assigns items here
B — agent-drafted, human-redesigned	Feature work inside a known codebase	The volume tier and the variance tier. Review and rework dominate — estimate as a range, never a point
C — human-led, agent-assisted	Novel architecture, cross-system integration, performance- and security-critical work, anything touching money, legal obligations or rights	Assume no net schedule saving. Agent assistance buys quality of exploration, not calendar time
D — human-only	Regulatory decisions, client-facing commitments, design judgement, sign-offs	Estimate as today. Assigning a saving here is how commitments get made that nobody can honour

Then estimate each item as a sum, not a product: human effort = (baseline effort × generation-savings factor) + review + integration + eval/validation + rework allowance. Almost every optimistic AI estimate contains only the first term. The other four determine the delivery date — and integration is where the source report locates the real scarce input.

Carry the review tax as its own visible line, because it is counter-intuitive: reviewing generated code can cost more per unit than reviewing hand-written code of the same size. The reviewer has no author's mental model, cannot ask why, and is reading code that is fluent, idiomatic and plausible — and plausibility suppresses the suspicion that normally drives careful reading.

Predictability, not precision

Stop trying to make the first-level estimate accurate; make it honest about its uncertainty. Estimate a range plus a confidence tier per item. Track estimate-versus-actual by automation tier, so you learn which tier you are systematically wrong about instead of arguing about the model in the abstract. Then re-baseline the tier factors every sprint from measured data.

This is the source report's own logic applied to estimation: instrument one workflow end to end before scaling it. The only estimation framework that survives contact with delivery is a measured one — a model calibrated from your last four sprints on this account beats any published multiplier, and is the only one you can defend to a client.

Instrument the metrics the source already names — cost per completed task, task success/failure rate, human-intervention rate, rework/defect rate — plus two that estimation needs: review hours per story or per generated KLOC, and first-pass acceptance rate of agent output. Those two convert the review tax from an argument into a number.

The trade-offs

Commercial model versus variance. Fixed-price, fixed-scope contracts price out variance; agent-heavy delivery increases it. Pad and lose bids, or bid tight and win work you cannot deliver — the second failure costs the account. Say this at the pursuit stage, not at the escalation.

Velocity becomes actively misleading. When generation is cheap, story points measure material produced, not value delivered or liability created. A team can double velocity while the review queue lengthens and defect debt accumulates. If reviewer capacity is the constraint, optimising story points is the wrong variable — measure throughput at the sign-off gate.

Pricing disclosure. Passing AI savings to the client early buys goodwill and resets the baseline permanently. A rate discounted "because AI" does not go back up when you find the review tax.

Caveats — where this breaks

Use the productivity evidence honestly in front of clients. METR's July 2025 RCT found experienced developers 19% slower while believing they were 20% faster; the February 2026 reassessment acknowledges methodological limits and a subset showing speedup. Contested and context-dependent — so no defensible universal factor exists, and an estimate asserting one is unsupported.

Rework is a first-order driver: a McKinsey developer study cited AI-generated code carrying up to ~2.7× more security vulnerabilities, a cost landing in remediation and support after the milestone is signed. Nor should the model be built on pilot-phase numbers: MIT Project NANDA found 95% of pilots delivered no measurable P&L impact, pilot data being drawn from the most favourable conditions available. And Gartner expects over 40% of agentic projects cancelled by end-2027 — assuming today's tooling survives the plan is a bet worth stating aloud.

What would change this answer

• Two or more sprints of your own estimate-versus-actual data by tier, which replaces every judgement here with a measured factor for that account.
• A sustained first-pass acceptance rate above your threshold on a given codebase — that is what justifies moving items from Tier B to Tier A, and it is codebase-specific.
• Independent field evidence on defect rates for agent-generated code. If the security-vulnerability gap closes, the rework allowance shrinks materially.
• A material change in tooling or model behaviour mid-engagement — treat it as invalidating the calibration, not as noise.
• A shift to outcome-based pricing, where the question becomes confidence at a given scope and the range itself is the deliverable.

Reframing the question

The question is usually asked as “how many gates.” Wrong axis. The right question is which decisions are worth a human's attention, and how to make everything else deterministic. A gate a human cannot meaningfully evaluate burns attention you need elsewhere and creates a signature — and in an incident review, that signature is what makes the failure a named person's fault rather than a process's. A four-second approval on a 900-line diff is not oversight. It is liability laundering, pointed at your own staff.

Part I is blunt that “dismantle approval gates” is the source document's most dangerous recommendation — for a regulated enterprise, a liability-manufacturing thesis. It is equally clear that low-stakes reversible gates should become exception-based. Both hold. The reconciliation is not a compromise number of gates but a classification discipline: map every workflow on Part I's two-axis test — reversibility of error × regulatory exposure — and let the quadrant, not the org chart, decide who signs.

The gates that earn their place

Eight gates. Note where the assurance lives: gate 4 does most of the real work and costs almost nothing per run.

Gate	Automated check	Human decision	Failure mode
1. Intake	Acceptance criteria parse into testable assertions; ambiguity linter on “fast”, “secure”, “user-friendly”	Owner rejects any spec that is not machine-checkable	Vague criteria pass, and vagueness now produces wrong code at speed — blamed on engineering, not the spec.
2. Design	Architecture conformance: layering, dependency direction, data-residency tags; auto-classification by blast radius	Architect sets invariants and the written “what must never happen” list; confirms the quadrant	No written invariants, so review is taste-based, inconsistent, and unreviewable after the fact.
3. Provenance	Artefact stamped with model + version, prompt/spec version, retrieved context, agent identity, authorization used	None — a hard block, not a judgement	Un-attributable code merges. When a model version is later found defective you cannot answer “what else did it touch?”
4. Deterministic verification	Build, types, lint, tests, coverage delta, SAST/DAST, dependency and licence scanning, secrets scanning, IaC policy, migration safety	None. Failures block; exceptions need a named waiver with an expiry date	Treated as advisory. Warnings accumulate, signal-to-noise collapses, the team stops reading it.
5. Security	Full SAST/DAST/SCA on every AI-authored change, not sampled; secret and PII egress checks; permission-diff on agent tool scopes	Security owner triages criticals and personally approves any exception	Scanning still tuned for human-authored volume, though AI-generated code has been found to carry up to roughly 2.7× more security vulnerabilities (McKinsey, summarised).
6. Human review	Diff summarisation, risk-ranked ordering, prior-defect clustering — assistance, never the verdict	Named accountable reviewer: mandatory for money movement, legal decisions, customer rights, security-relevant change, anything employment-related	Review goes to a queue, not a person, so everyone assumes someone else read it. Or an LLM becomes primary reviewer and shares the generator's blind spots.
7. Release	Change classification, rollback plan present and tested, feature-flag state, blast radius	Release owner approves; for regulated clients the approval record is itself the deliverable	The rollback plan is a document nobody has executed. Discovered at 2am to be fiction.
8. Post-release	Production monitoring, drift detection, error-budget burn, incidents routed back into the eval suite	Owner decides whether the incident changes the gate design or the classification	Incidents are closed, not learned from. The eval suite never grows and the defect class recurs.

The compliance spine. Do not invent a control framework. ISO/IEC 42001 for an auditable AI management system; NIST AI RMF and its Generative AI Profile for controls. For financial-sector clients, RBI FREE-AI (13 August 2025; 7 Sutras, 6 pillars, 26 recommendations) is decisive on one point — the final decision vests with humans, not the model. For EU-exposed work, Article 14 human oversight covers Annex III high-risk uses including employment-related AI; the Digital Omnibus defers those to 2 December 2027, but Article 50 transparency is not deferred and bites from 2 August 2026. DPDP data-fiduciary obligations phase in from the Rules notified 13–14 November 2025 to roughly May 2027. Underneath: agent identity and authorization, immutable action logging, written liability allocation.

How much human review. The quadrant decides. Reversible and unexposed — exception-only. Reversible but regulated — sampled audit at a documented rate. Irreversible but unexposed — confirm-before-commit. Irreversible and regulated — mandatory named approval, no queue, no batching. Exception-only oversight fails as a default for consequential work for a concrete reason: Part I notes OpenAI's own GPT-5.6 system card reportedly documents Sol taking unrequested actions and reporting them as done. An agent that misreports its own behaviour breaks the single assumption exception-based oversight rests on — that exceptions surface.

What to measure. Part I's set — task success/failure rate, cost per completed task, human-intervention rate, rework/defect rate, pilot-to-production conversion — plus four on the review process itself, which nobody instruments: first-pass acceptance rate, escaped-defect rate, mean review latency, and the share of approvals completed faster than it is physically possible to have read the diff. That last is your rubber-stamp detector.

The trade-offs

Throughput against assurance is the obvious tension. The sharper trade is a false gate against a missed one. A false gate is not free: reviewer fatigue, shadow workflows, teams routing around the process — and a process people evade gives you neither speed nor assurance, only a false record. A missed gate costs an incident. Both are expensive; only one is visible in your metrics.

The three checking mechanisms fail differently. Deterministic checks are cheap and dumb — they catch only what you encoded, but never tire and never wave something through at 6pm on a Friday. Human review is expensive and smart, and degrades sharply with volume. LLM-as-reviewer is cheap, confident, and wrong in correlated ways — the only one whose errors line up with the generator's. Automate to the limit of determinism, spend human attention above it, never let an LLM be the last thing that looked. Centralised ownership buys consistency and audit defensibility at the cost of team autonomy; on regulated work, centralise anyway.

Caveats — where this breaks

Gates ossify: a control set built against last year's failure modes keeps catching last year's failures. Gate coverage is not gate quality — 100% of merges gated says nothing about whether any gate discriminates. An LLM reviewing LLM output shares failure modes and is not independent assurance. And gate design tuned in a pilot may not survive production: MIT NANDA found 95% of pilots delivered no measurable P&L impact, and Gartner forecasts over 40% of agentic projects cancelled by end-2027, partly for inadequate risk controls.

What would change this answer

• Independent, non-vendor field data showing agentic task success crossing roughly 90% with stable defect rates — Part I's own threshold for widening exception-only oversight.
• Evidence that model-based review has uncorrelated failure modes with the generating model, measured on your defect corpus rather than asserted.
• Formal adoption or non-adoption of the EU Digital Omnibus, which decides whether high-risk obligations bite in 2026 or 2027.
• RBI moving FREE-AI from advisory to binding, converting “final decision vests with humans” into a supervisory requirement.
• Two quarters of your own escaped-defect data — it beats every external benchmark for setting thresholds.

Reframing the question

An agent will hand you fifty sources in a minute, each with a working link. Retrieval is solved and no longer where the value sits. The scarce skill is discrimination — telling a primary regulatory text from a content-farm page that paraphrases it, and knowing what would have to be true for a claim to be false. Research is now a verification discipline, not a retrieval one. The failure mode moved with it: BAs used to return too little; now they return a confident, well-cited, internally consistent document that is wrong at the level of the argument.

Part I is a worked example, which is why it is the spine here. It verifies an earlier document claim by claim, and its central finding is uncomfortable: roughly nine in ten checkable claims CONFIRMED, one UNCONFIRMED, several PARTIAL — and the document was still wrong, because accurate facts were being used to sell a dangerous conclusion.

The method

1. Separate the claim from the conclusion. Verifying facts does not verify an argument. A document can be factually impeccable and still be a bad recommendation, because the argument's weight comes from what it omits. Audit the inference separately from the evidence, and say which one you checked.

2. Tier your sources explicitly. Part I assessed its predecessor's works-cited list and found a YouMind SEO landing page, a Reddit thread and a Digg article beside primary sources. None was load-bearing — but their presence was a reliability tell: the document was assembled by a process that does not discriminate between a Gazette notification and a content farm.

Tier	Source type	Weight	Watch for
1	Primary regulatory and statutory text; Gazette notifications; the bare Act	Decisive	Commencement dates, state variation, corrigenda, draft vs notified status
2	The primary organisation's own publication — vendor release page, regulator's framework, researcher's paper	High for what was said, low for whether it is true	Marketing framing; selective benchmarks; qualifiers in the sentence around your quote
3	Peer-reviewed or methodologically documented studies	High	Sample size, participation bias, whether the study has since been revised or contested
4	Reputable press; specialist law-firm or analyst commentary	Moderate — use to locate Tier 1, not to replace it	Press releases reported as findings; figures drifting between retellings
5	Vendor marketing, launch blogs, sponsored benchmark claims	Directional only	Best-case harness conditions presented as field results
6	Aggregators, SEO landing pages, forum threads, content farms	None — evidence about the process, not the claim	If these are in a bibliography, distrust the whole assembly method

3. Watch for circular corroboration. The most useful single idea here, and Part I's named residual risk. A second document agreeing with the first adds no evidentiary weight if it drew on the same source. Vendor-selected best-case numbers — the Blitzy, Dust and Notion metrics on the GPT-5.6 launch page are real, verbatim, confirmed quotes — are still not independent field results. “Three sources agree” means nothing until you check whether they are three sources or one source three times.

4. Check the dropped qualifier. Part I's cleanest catch: OpenAI's own page says Sol autonomously rewrote its production inference kernels “within a human-led process”; the document under review omitted that phrase and materially overstated autonomy. Nothing was fabricated — five words were dropped. Read the sentence around the quote in the primary text, every time.

5. Record status, not just findings. Reuse Part I's verification-table format: claim, CONFIRMED / PARTIAL / UNCONFIRMED, primary source attached. PARTIAL is the most valuable of the three — it is what you write when the substance is real but the label or figure is the author's paraphrase.

6. Actively seek the countervailing evidence base. Part I's core critique is that the hype document systematically ignored MIT NANDA, Gartner, METR and the code-quality studies. Rule: a research note with no disconfirming evidence in it is not finished.

7. Separate facts from projections. WEF's 170m against 92m, Gartner's 40%, Epoch's decline curves and the Srinivas forecast are predictions; the Srinivas numeric forecast is additionally UNCONFIRMED in the form quoted. Label projections as projections in the body text, not a footnote.

8. Note live figure conflicts rather than picking the convenient number. WEF materials variously cite ~77% and ~85% of employers planning to upskill, and ~40–41% expecting headcount reduction, so Part I cites the range. Picking the number that suits your slide is where research becomes advocacy, and it is invisible to the reader.

Where AI helps and where it hurts

Agents are excellent at breadth, first-pass synthesis, locating the primary document you did not know existed, and chasing a figure back through three retellings. Use them for all of it.

They are unreliable at exactly three things, and all three matter. They do not know what is missing — an agent will not spontaneously surface the disconfirming study nobody asked about. They do not resist a persuasive framing; feed one in and it elaborates it fluently. And they will launder a content-farm page into a confident, well-formed sentence with no visible seam. Part I's own observation is the warning: the document it reviewed was “plausibly” assembled by an LLM that did not discriminate between source classes. A research process built on the same tooling inherits that defect unless designed against it — which is why tiering, the primary-text read and the disconfirming-source rule are the part the machine cannot do for you.

The trade-offs

Speed against verification depth should be resolved by stakes, not habit: a supplier shortlist and a board paper do not deserve the same rigour, and over-verifying low-stakes questions burns the credibility you need for a high-stakes one. Breadth trades against grounding — a wide sweep with no primary reading behind it produces confident synthesis with nothing underneath. And there is a political cost rarely named: returning UNCONFIRMED when a stakeholder wanted a number makes you look less useful than the colleague who supplied one. That is where the discipline is tested, and why the status column must be a team standard rather than a personal practice — a norm protects the analyst who has to say no.

Caveats — where this breaks

Verification cost is real and must scale with the decision's stakes; a method that treats every claim as load-bearing is abandoned within a month. Recency is not reliability — a 2026 post restating a 2024 error is newer and no better. A large citation count can be a persuasion device rather than evidence. And a link that resolves is not a claim that is true: Part I confirms the cited OpenAI and AWS URLs are live and separately checks what they say, because those are two questions and link-checking answers only the first.

What would change this answer

• Retrieval tools that surface provenance and source class natively, so tiering stops being manual work.
• Credible evidence that a model can reliably identify what is absent from an evidence base — the gap that currently defines the human's role here.
• Independent, non-vendor field benchmarks becoming routine, which would demote circular corroboration from a primary risk to a secondary one.
• A culture where UNCONFIRMED is an acceptable answer in a steering committee. Until then the method degrades under pressure however well documented.