Most enterprise AI programs stall for a plain reason: nobody agreed in advance what success looks like, who owns it, or what evidence would stop it. This guide covers readiness, pilot design, scaling, governance, and an ROI model your finance team can audit, updated for agents, current EU AI Act dates, and the cost behavior of 2026.
By Tom Morgan, Founder & Editor of AInvasion and the editor accountable for this article, as described in our methodology. Published 1 October 2026. Last reviewed 1 October 2026; next review due by 1 January 2027.
A pilot ends in one of three pre-agreed outcomes:
| Outcome | Meaning |
|---|---|
| Go | The primary metric beat the measured baseline by the agreed margin, and no guardrail was breached. |
| Adjust | The signal is promising but the evidence is thin, or one guardrail slipped. Fix it and re-run. |
| Stop | No meaningful gain, a stop condition fired, or the risk outweighs the value. |
If your pilot charter cannot name all three outcomes before launch, it describes a rollout, not a pilot.
In this guide
- Key takeaways
- What the evidence says
- What changed in 2026
- Pilots are experiments, not demos
- Readiness questions
- Choose the autonomy level
- Choosing the first use case
- Designing a pilot that can fail
- Evaluating AI output
- The 90-day pilot plan
- From pilot to production
- Governance and risk
- A defensible ROI model
- ROI model in a spreadsheet
- Common failure modes
- Build, buy, or partner
- Pilot charter template
- FAQ
- Sources and method
Key takeaways
- A pilot is an experiment. Write the hypothesis, baseline, guardrails, and go, adjust, or stop rule before any build starts, and make sure “stop” is a real possibility.
- Measure with system data, not impressions. In the best-known randomized trial of AI coding tools, experienced developers took 19% longer with AI while believing they had been 20% faster.
- Count every cost and haircut every benefit. Saved minutes only become money when staffing, throughput, or spend changes. In the worked example below, one pilot produces a loss, a near break-even, or a 39% return depending on a single assumption.
- Start at the lowest autonomy level that can prove the value. Retrieval and drafting come before acting with approval, which comes before acting alone.
- Date-check your compliance plan. EU high-risk obligations for Annex III systems now apply from 2 December 2027, but transparency duties start earlier and classification work does not wait.
What the evidence says about enterprise AI results
The large surveys agree on one pattern: individual use is common, and enterprise-level results are rare. McKinsey’s 2026 State of AI survey, published in August 2026 with 1,719 respondents, found that 80% said AI had improved their own productivity, while 37% attributed any EBIT impact to AI. The 37% is essentially flat on the prior year, and the share of “high performers” (organizations attributing 5% or more of EBIT to AI and reporting significant value) stayed near 6%. [1]
| Measure (McKinsey State of AI 2026) | Share of respondents | |
|---|---|---|
| Say AI improved their own productivity | 80% | ████████████████ |
| Say AI is scaling across the enterprise | 44% | █████████ |
| Attribute any EBIT impact to AI | 37% | ███████ |
| AI high performers (5% or more of EBIT, significant value) | about 6% | █ |
Self-reported; 1,719 respondents. Figures as reported by secondary coverage of the survey, so check the report itself before quoting. See [1].
Two other widely cited findings sharpen the picture, and both come with limits worth knowing.
- Cancellation risk for agents. Gartner predicted in June 2025 that over 40% of agentic AI projects will be canceled by the end of 2027, citing escalating costs, unclear business value, and inadequate risk controls. This is a forecast, and part of its basis was a poll of webinar attendees, so read it as a warning about discipline, not a measured failure rate. Gartner also estimated that only about 130 of the thousands of vendors marketing “agentic AI” were real, a practice it calls agent washing. [2]
- The perception gap. In a 2025 randomized controlled trial, METR had 16 experienced open-source developers complete 246 real tasks with and without AI tools. Before the study, the developers forecast a 24% speedup, and afterwards they estimated 20%. Measured completion time was 19% longer with AI. [3] The study is small, used early-2025 tools, and studied expert developers in familiar codebases, so it does not show that AI slows everyone down. It does show that feeling faster and being faster are different measurements.
Our reading is that these results share one cause. People judge AI by how the work feels, organizations judge it by licenses deployed, and almost nobody measures a baseline first. The rest of this guide is a way to fix that.
What changed in 2026
If you last read an enterprise AI playbook in 2023 or 2024, these are the shifts that change how you should plan.
- Agents moved from demo to deployment, mostly at large companies. McKinsey reports 40% of respondents at organizations above $1 billion in revenue are scaling agents in at least one function, up from 27% a year earlier. At smaller organizations, the figure stayed flat at 22%. [1] Agents need different controls from copilots; see autonomy levels.
- EU high-risk dates moved. The Digital Omnibus pushed Annex III obligations to 2 December 2027 and Annex I to 2 August 2028, while other dates did not move. Details are in governance and risk. [6]
- Outputs are non-deterministic and models change under you. The same prompt can produce different answers, and vendors update or retire models. That makes repeated-run evaluation and regression testing a requirement. See evaluating AI output.
- Cost is per task, not per seat. Usage-based pricing and multi-step agents mean the cost of one completed task can vary widely. Model cost per completed task, and cap it. See the ROI model.
- Shadow AI is the default state. Staff already use unsanctioned tools, which is why a sanctioned option and a clear policy are part of readiness.
Pilots are experiments, not demos
A demo shows that something can work. A pilot tests whether it does work, under real conditions, with real users, against a baseline you measured beforehand. That difference changes how you run everything.
| Demo | Pilot | |
|---|---|---|
| Goal | Persuade | Learn |
| Data | Curated | Messy and production-like |
| Users | Champions | Representative staff, including skeptics |
| Success criteria | Impressive output | Pre-agreed metrics against a measured baseline |
| Failure | Avoided | Allowed and documented |
| Output | Enthusiasm | A go, adjust, or stop decision |
A pilot that cannot produce a “stop” is a pre-approved rollout with extra steps. The practical test is simple: ask the sponsor, before launch, what result would make them cancel. If they cannot answer, the pilot has no decision rule yet.
From our own testing
We ran a small two-arm comparison ourselves: six B2B SaaS accounts on Instagram and LinkedIn over four months in late 2025, comparing scheduled posting with AI-written captions against posting only when there was a contextual reason. The second approach showed 2.3 times higher average engagement. We published it with the caveats attached, because six accounts, a December 2025 Instagram algorithm update, and our own unaudited measurement make it a directional observation, not proof of cause. It was a content test, not an enterprise pilot, but the discipline is the one we are asking of yours: say what you measured, say what else could explain it, and never round a hypothesis up to a finding. Read the full write-up.
Readiness: five questions before you spend anything
Before choosing a vendor or a model, answer these honestly.
- Is there a named business owner? Not a sponsor who attends the kickoff, but a P&L or operations leader whose targets move if this works.
- Is the problem measurable today? If you cannot state current cycle time, error rate, or cost per case, you cannot prove improvement later.
- Is the data accessible and permitted? Know where it lives, who owns it, what condition it is in, and whether policy, contract, or regulation limits its use with external models.
- Does a workflow exist for the output to land in? An AI draft nobody reviews, or a prediction nobody acts on, creates no value.
- Who handles exceptions? Every system produces wrong outputs. Decide in advance who catches them and how.
If two or more answers are “no,” fix those first. A readiness gap costs far less than a failed pilot. If staff is already using unapproved tools for this task, treat that as useful data: it shows demand, and it shows where your sanctioned option needs to be at least as convenient.
Choose the autonomy level before you choose the tool
The biggest design decision in a 2026 pilot is how much the system is allowed to do. We use a five-step ladder, which is our own working framework rather than an industry standard. Pilot at the lowest rung that can test your hypothesis, and promote a use case up the ladder only after it has produced evidence at the current rung.
| Level | What the system does | Minimum controls |
|---|---|---|
| 1. Retrieve | Answers questions from approved sources, read-only | Source citations, access control, answer-quality sampling |
| 2. Draft | Produces a draft a person edits and sends | Human review before anything leaves the building, override-rate tracking |
| 3. Act with approval | Proposes an action and waits for a person to approve each one | Least-privilege access, action logs, approval records |
| 4. Act within limits | Executes low-risk actions inside hard limits, with sampled review | Spending and scope caps, anomaly alerts, a tested kill switch |
| 5. Act autonomously | Handles whole cases and escalates exceptions only | All of the above, plus independent audit and a rollback path |
Most first pilots belong at level 2 or 3. Jumping to level 4 or 5 because a vendor demo looked smooth is the most common way to turn a recoverable error into an incident.
Choosing the first use case
Strong first use cases are frequent, bounded, measurable, reviewable, and low-regret if wrong. Score each candidate from 1 (weak) to 5 (strong) on the seven criteria below, then apply the gates.
| Criterion | What a high score looks like |
|---|---|
| Volume | The task repeats often enough for savings to compound |
| Measurability | A clear baseline and a metric finance recognizes |
| Data readiness | Data is accessible, reasonably clean, and cleared for use |
| Risk tolerance | Errors are cheap, visible, and reversible |
| Human-in-the-loop fit | A reviewer can check output quickly |
| Stakeholder pull | The team doing the work wants it |
| Integration effort | Fits existing tools without a re-platforming project |
Add gates, not just a total. A high total can hide a fatal weakness. Our rule is that a score of 1 on data readiness, risk tolerance, or measurability disqualifies a candidate regardless of its total. Here is how that plays out on three hypothetical candidates (illustrative scores).
| Candidate | Vol. | Meas. | Data | Risk | HITL | Pull | Integ. | Total | Gate |
|---|---|---|---|---|---|---|---|---|---|
| Support reply drafting with agent review | 5 | 5 | 4 | 4 | 5 | 4 | 3 | 30 | Pass |
| Invoice field extraction | 4 | 5 | 3 | 4 | 4 | 3 | 4 | 27 | Pass |
| Autonomous refund approval | 4 | 4 | 3 | 1 | 1 | 3 | 2 | 18 | Fails risk gate |
Typical starting points
- Internal knowledge retrieval over policies, contracts, runbooks, and support history
- Customer-support drafting and triage with human approval before sending
- Document processing: extraction, classification, and summarization of invoices, claims, applications, and filings
- Software engineering assistance: code review support, test generation, migration work (measure it, given the perception gap above)
- Sales and marketing operations: research, drafting, CRM hygiene
- Compliance and legal review support: first-pass clause comparison, with attorney sign-off
Poor first choices
- Fully autonomous decisions with legal, safety, or major financial consequences
- Use cases with no baseline and no owner
- “Transform the whole department” mandates
- Anything that depends on data you have not yet been cleared to use
Designing a pilot that can actually fail
Write these seven items down before any build begins. Together they are the pilot’s pre-registration: once results arrive, nobody gets to move the goalposts.
- Hypothesis. One sentence: “Using [tool] on [task] will reduce [metric] from [baseline] to [target] without raising [risk metric].”
- Baseline. Measure the current process for at least two to four weeks, using ticket data, system timestamps, or audit logs. Estimates from memory are not a baseline.
- Primary metric. One number that decides the outcome, such as handling time per case.
- Guardrail metrics. What must not get worse: quality score, error rate, escalation rate, customer satisfaction, compliance exceptions.
- Control or comparison. A matched group working the old way, or a before-and-after comparison that accounts for seasonality.
- Stop conditions. Thresholds that pause the pilot, such as a severity-one error, a data exposure, or quality below an agreed floor.
- Decision rule. What result means go, adjust, or stop, and who decides.
A decision rule you can adapt
Vague rules like “if it looks good we’ll scale” are where pilots go to die. A usable rule names a threshold on an interval, not on a single average. The thresholds below are examples; set yours with the business owner and finance before launch.
| Outcome | Condition |
|---|---|
| Go | The lower bound of the 80% interval for handling-time reduction is at least 15%, the quality score is no more than 2 points below baseline, there are no severity-one incidents, and the override rate sits inside the agreed band. |
| Adjust | The point estimate meets the 15% target but the interval’s lower bound does not, or exactly one guardrail is breached and has an identified fix. |
| Stop | The point estimate is below 5%, any stop condition fired, or the cost per completed task exceeds the value created at the high-realization case. |
Measure with logs, not with surveys
Self-reported time savings are the weakest evidence in this field, and the METR result above is the reason. Pull timestamps from the ticketing, workflow, or version-control system. Use surveys for adoption and trust, not for the savings figure that goes into the ROI model.
Sample sizes and honesty
Small pilots are noisy. Ten users over two weeks can show a dramatic effect that disappears at scale. Three effects routinely inflate early results:
- Novelty. Usage and effort are higher in the first weeks. Run long enough to see the settled pattern.
- Selection. Volunteers are not your average employee. Recruit a representative group, skeptics included.
- Observation. People work differently when they know they are measured. Where possible, compare with a control group under the same observation.
State the limits of your evidence plainly, report ranges, and do not extrapolate a best-case team to the whole company.
Evaluating AI output when answers are not deterministic
Traditional software tests assume the same input gives the same output. Generative systems do not, and the model underneath may change without your team touching anything. An evaluation set is how you keep control.
- Build the set early. Collect 100 to 300 representative real tasks with agreed-correct answers or quality rubrics, including hard and edge cases. This range is a practitioner heuristic rather than a statistical rule; at 100 tasks, each one moves the pass rate by one percentage point, which is a useful sense of resolution.
- Run each item more than once. Because outputs vary, report the pass rate across repeated runs, and track consistency, not just best-case performance.
- Calibrate automated graders. If a model grades another model’s output, check its grades against a human-labeled sample before trusting it, and re-check when either model changes.
- Re-run on every change. Model version, prompt, retrieval source, tool definition, or configuration. Treat it like a regression suite.
- Keep it portable. A vendor-neutral evaluation set is also your best defense against lock-in, because it lets you compare alternatives on your own tasks.
- Refresh it. Add real production failures to the set. A set that never changes slowly stops reflecting the work.
This single habit prevents more production surprises than any other, and it is cheap compared with the cost of discovering a regression through customers.
The 90-day pilot plan
A well-scoped pilot fits in roughly a quarter. Phases overlap on purpose: recruiting users and drafting guardrails run alongside the baseline, and build work starts before the baseline closes. Adjust to your environment.
gantt
title 13-week pilot timeline (gates are decisions, not milestones)
dateFormat X
axisFormat Week %s
section Phases
Frame (gate: charter signed) :0, 2
Baseline (gate: accepted by finance and owner) :1, 3
Build and configure (gate: pre-launch eval passed) :3, 3
Run (gate: minimum volume reached) :6, 4
Decide (gate: go, adjust, or stop recorded) :10, 2| Phase | Weeks | Key activities | Exit gate |
|---|---|---|---|
| Frame | 1-2 | Select use case, name owner, draft hypothesis, start data and security approvals | Signed pilot charter |
| Baseline | 2-4 | Measure the current process, define metrics and guardrails, recruit pilot users | Baseline report accepted by finance and the business owner |
| Build and configure | 4-7 | Integrate with the workflow, build the evaluation set, red-team obvious failure cases, train users | Pre-launch evaluation passed |
| Run | 7-11 | Live use with monitoring, weekly review, an issue log, and user feedback | Minimum run time and volume reached |
| Decide | 11-13 | Analyze results against baseline, review cost and risk, write the recommendation | Decision recorded |
Roles you need
- Business owner: accountable for the outcome.
- Program lead: runs the plan day to day.
- Technical lead: integration, evaluation, monitoring.
- Security and privacy partner: reviews data flows early, not at the end.
- Risk, legal, or compliance partner: sets boundaries for the use case.
- Finance partner: co-owns the baseline and the ROI model.
- Frontline champions and skeptics: real users who give honest feedback.
From pilot to production
A successful pilot proves the idea. Production needs a different set of capabilities, and teams that skip them tend to discover the gap through an incident.
Production readiness checklist
- Reliability: latency, uptime, and fallback behavior when the model or vendor is unavailable.
- Quality monitoring: sampled human review, automated evaluation, and drift detection against your evaluation set.
- Security: access control, logging, data retention rules, and testing for prompt injection and data leakage.
- Cost control: usage caps, per-use-case budgets, and alerts on spikes.
- Change management: training, updated procedures, and clear guidance on when to trust, check, or override output.
- Support model: who answers when it breaks or misbehaves.
- Exit plan: how you would swap models or vendors without rebuilding the workflow.
Extra controls when the system can act
Once a system can call tools, send messages, or change records, add these on top of the checklist.
- Least privilege. Give the agent the narrowest credentials that work, with read and write access separated.
- Treat retrieved content as untrusted. Web pages, emails, and documents can contain instructions aimed at the model. Prompt injection is the first item in the OWASP Top 10 for LLM Applications for this reason. [9]
- Log every action with enough detail to reconstruct what happened and why.
- Cap blast radius. Spending limits, rate limits, and scope limits, plus a kill switch someone has actually tested.
- Require approval for irreversible steps until the evidence supports loosening it.
Scale in waves
Expand to adjacent teams with similar work before moving to different functions. Each wave should re-validate the guardrail metrics, because new users, data, and edge cases change results. McKinsey’s high performers are distinguished largely by redesigning workflows rather than layering AI over existing ones, so budget for process redesign in every wave. [1]
Retire what does not earn its place
Set a review cadence. Tools that fail to show measurable value after a defined period should be reduced or switched off. That discipline keeps the portfolio credible with finance.
Governance and risk
Good governance makes good projects faster by removing ambiguity. It should not slow everything to a crawl.
A lightweight operating model
- Intake and triage: one route for proposing use cases, with a risk tier assigned up front.
- Risk tiering: low (internal drafting), medium (customer-facing with review), high (decisions affecting people, safety, finances, or rights).
- Tier-based controls: higher tiers need stronger testing, documentation, human oversight, and approval.
- Model and vendor inventory: know what is deployed, where, and on what data.
- Incident process: a defined path for harmful, biased, inaccurate, or leaked outputs.
Frameworks worth aligning to
- NIST AI Risk Management Framework (AI RMF 1.0). Voluntary guidance organized around four functions: Govern, Map, Measure, and Manage. NIST’s Generative AI Profile (NIST AI 600-1, July 2024) adds risks specific to generative systems. NIST states that AI RMF 1.0 is being revised, and in April 2026 it released a concept note for a critical-infrastructure profile, so check the current page before citing a version. [7]
- ISO/IEC 42001:2023. A certifiable AI management system standard, useful when customers or regulators ask for an external attestation. [8]
- OWASP Top 10 for LLM Applications. A practical security checklist for builders. [9]
- Sector rules. Financial-services model risk guidance, health-data privacy law, and employment-decision regulations all still apply to AI-assisted decisions.
The EU AI Act timeline as of October 2026
The Digital Omnibus, reported as Regulation (EU) 2026/1744 and in force since late July 2026, postponed the high-risk obligations because harmonised standards and national authorities were not ready. It did not remove the requirements. [6]
| Date | What applies |
|---|---|
| Already in effect | Prohibited practices (from 2 February 2025) and obligations for providers of general-purpose AI models (from 2 August 2025), under the Act’s original schedule |
| 2 August 2026 | Most Article 50 transparency obligations apply, such as telling people they are interacting with an AI system |
| 2 December 2026 | Marking of synthetic content under Article 50(2) after a shortened grace period. Commentary also reports new prohibited-practice provisions added by the Omnibus from this date |
| 2 December 2027 | Stand-alone Annex III high-risk systems, including employment and worker management, credit scoring, education, biometrics, and critical infrastructure |
| 2 August 2028 | High-risk AI embedded in regulated products under Annex I, such as medical devices and machinery |
Per legal commentary on the adopted Omnibus; verify against the official text.
Our advice on the delay
Plan against December 2027, but do not pause classification and governance work. Whether a system is high-risk is decided under Article 6, and commentary says the Commission published draft classification guidelines on 19 May 2026. One legal analysis notes that the Omnibus itself proves deadlines can slip, yet harmonised standards were still unfinished at that point. Betting on a second delay is a risk decision, so make it explicitly and with counsel.
This page is not legal advice. Commentators differ on some Article 50 details, so confirm which paragraphs apply to your use case with qualified counsel and the text on EUR-Lex.
Questions your risk partner will ask
- What data goes to the model, and where is it processed and stored?
- Is any of it used for vendor training, and can that be excluded by contract?
- Who can see prompts and outputs?
- What happens when the output is wrong, biased, or harmful?
- Can we explain and reproduce a given decision?
- What is the human reviewer’s actual authority to override?
Building a defensible ROI model
ROI is where programs lose credibility. The fix is to use finance’s definitions, count full costs, and separate hard savings from soft benefits.
ROI (%) = (total benefits - total costs) / total costs x 100
Payback period = total costs / net monthly benefitCount the full cost
| Category | Examples |
|---|---|
| Licenses and usage | Seats, API or token consumption, compute |
| Implementation | Integration, data preparation, evaluation set, security review |
| People | Program team time, user training, extra oversight, ongoing support |
| Governance | Legal, compliance, audit, red-teaming |
| Change management | Process redesign, communications, the productivity dip during adoption |
| Ongoing operations | Monitoring, evaluation upkeep, vendor management, incident handling |
Teams commonly count licenses and forget everything else. Finance will not. Watch for double counting in the other direction too: if your measured handling time already includes the reviewer reading the AI draft, do not add that review time again as a cost. Add only the extra oversight that sits outside the measured task, such as quality sampling and monitoring.
Classify benefits honestly
| Type | Definition | Treatment |
|---|---|---|
| Hard savings | Reduced spend that shows up in the budget: contractor hours cut, vendor fees avoided, hiring avoided | Count fully |
| Capacity release | Time freed that staff redeploy to valuable work | Count only if you can show where the time goes and what it produces |
| Revenue impact | Higher conversion, faster sales cycles, better retention | Count with attribution evidence and a conservative haircut |
| Risk reduction | Fewer errors, incidents, or compliance failures | Estimate expected value (probability x cost) and flag it as an estimate |
| Strategic or soft | Employee satisfaction, brand, optionality | Report separately; do not fold into ROI |
A fully worked example
This is a constructed scenario to show the mechanics, not a benchmark. A support team handles 20,000 tickets per quarter. A pilot with human review measured handling time falling from 12 to 9 minutes. The loaded cost of an agent hour is assumed at $45, and 3% of AI-assisted tickets needed 15 extra minutes of rework. Quarterly costs include one-time costs spread over six quarters.
| Line | Calculation | Value |
|---|---|---|
| Time saved | (12 − 9) min x 20,000 tickets | 1,000 hours |
| Less rework from errors | 20,000 x 3% x 15 min | 150 hours |
| Net hours saved | 1,000 − 150 | 850 hours |
| Gross capacity value | 850 x $45 | $38,250 |
| Quarterly costs | Licenses and usage $9,000; implementation (amortized) $6,000; quality sampling and monitoring $3,500; governance $1,500; training and change (amortized) $2,000 | $22,000 |
Now apply the realization factor, the share of saved time that converts into real value because staffing, throughput, or spend actually changes.
| Case | Realization | Benefit | Net value | ROI |
|---|---|---|---|---|
| Low | 30% | $11,475 | −$10,525 | −48% |
| Expected | 50% | $19,125 | −$2,875 | −13% |
| High | 80% | $30,600 | +$8,600 | +39% |
What this example teaches
The pilot “worked”: handling time fell by 25%. Yet at 50% realization it loses money, and it breaks even only when about 58% of the saved time turns into real value (22,000 / 38,250). If adoption comes in 30% lower than planned, even the high case turns slightly negative (−$580). A 25% speedup is not a business case. The business case is the plan for what happens to the freed hours.
A planning check before the pilot: break-even error rate
Suppose a person needs t minutes to check an AI draft, and a rejected draft means doing the whole task by hand in T minutes. If drafts are rejected a fraction e of the time, the expected time per task is t + e x T. Savings exist only when that is below T, which gives a break-even rejection rate of:
e < 1 - t / TIf checking a draft takes 9 minutes of a 12-minute task, drafts must be accepted more than three times in four before you save anything. Run this arithmetic before you build; it shows quickly when review effort, not model quality, is the constraint.
Cost behavior in the era of usage pricing and agents
Unit prices for models change often, but the bigger modeling risk is volume. An agent that makes many model calls per task, retries on failure, or reads long documents can make cost per completed task several times higher than a simple chat call. Model cost per completed task, not per seat or per token. Include a usage-growth scenario, set per-use-case budgets and alerts, and re-price at ten times pilot volume before you commit to a contract.
Rules that keep your model credible
- Use the baseline you measured, never a vendor’s claimed average.
- Report ranges (low, expected, high) rather than one number.
- Apply a realization haircut to capacity gains.
- Avoid double-counting review time, and include oversight that sits outside the measured task.
- Model usage growth, because consumption pricing can outrun savings.
- Show sensitivity: what happens to ROI if adoption is 30% lower or quality drops?
- Have finance sign off on assumptions before results arrive, not after.
- Track after launch. Pilot ROI is a forecast; production ROI is a measurement.
Leading and lagging indicators
| Leading (early signals) | Lagging (outcomes) |
|---|---|
| Weekly active users and usage depth | Cost per transaction |
| Output acceptance and override rates | End-to-end cycle time |
| Time to first draft (from logs) | Error and rework rates |
| User confidence scores | Revenue, retention, customer satisfaction |
ROI model in a spreadsheet: ranges and sensitivity
Paste this layout into a spreadsheet (inputs in column B). The default values reproduce the worked example above; replace them with your measured baseline and finance-approved costs.
| Row | Input or formula | Default |
|---|---|---|
| B1 | Tasks per quarter | 20000 |
| B2 | Baseline minutes per task | 12 |
| B3 | Minutes per task with AI | 9 |
| B4 | Loaded cost per hour ($) | 45 |
| B5 | Share of tasks using AI (0-1) | 1 |
| B6 | Rework rate on AI tasks (0-1) | 0.03 |
| B7 | Extra minutes per reworked task | 15 |
| B8 | Realization, low / expected / high | 0.3 / 0.5 / 0.8 |
| B9 | Quarterly costs, total (amortized) | 22000 |
| B10 | Net hours saved | =B1*B5*(B2-B3)/60 - B1*B5*B6*B7/60 |
| B11 | Gross capacity value | =B10*B4 |
| B12 | Net value at realization r | =B11*r - B9 |
| B13 | ROI at realization r | =B12/B9 |
| B14 | Break-even realization | =B9/B11 |
Sensitivity (costs held constant, which is conservative for usage-based pricing):
| Adoption | Low (30%) | Expected (50%) | High (80%) |
|---|---|---|---|
| 100% of tasks | −$10,525 | −$2,875 | +$8,600 |
| 70% of tasks | −$13,968 | −$8,612 | −$580 |
Outputs are only as good as the inputs. The model does not know your seasonality, your staffing constraints, or whether the time saved can actually be redeployed.
Common failure modes
| Failure mode | What it looks like | Countermeasure |
|---|---|---|
| Pilot purgatory | Many pilots, none reach production | A pre-agreed decision rule and a budget path set at kickoff |
| Solution in search of a problem | The technology was chosen first | Start from a measured workflow pain |
| No baseline | “It feels faster” | Measure before you build, from logs |
| Perception gap | Users report big savings that timestamps do not show | Use system data for savings; use surveys for adoption and trust |
| Demo data | Great pilot, poor production results | Test on messy, representative data |
| Shadow adoption | Staff use unapproved tools | Provide a sanctioned option and a clear policy |
| Over-automation | Humans rubber-stamp output | Design real review and track override rates |
| Agent washing | A product labeled “agentic” that is a scripted workflow with a chat box | Ask the vendor to demonstrate multi-step actions on your data |
| Evaluation rot | Quality drifts after a model or prompt change, unnoticed | Regression suite re-run on every change, refreshed with production failures |
| Hidden costs | ROI evaporates after launch | Full cost model, per-task costing, usage caps |
| Change neglect | Tool deployed, behavior unchanged | Training, workflow redesign, manager involvement |
| Vendor lock-in | You cannot switch when pricing or quality shifts | Abstraction layers and portable evaluation sets |
| Governance as afterthought | A late security or legal veto | Involve risk partners at the charter stage |
Build, buy, or partner
| Option | Best when | Watch for |
|---|---|---|
| Buy a packaged product | The use case is common, speed matters, and differentiation is low | Fit to your workflow, data handling, pricing growth |
| Configure a platform | You need flexibility without owning everything | Integration debt and the skills required |
| Build custom | The workflow or data is a competitive advantage | Maintenance burden, talent, evaluation discipline |
| Partner with a specialist | You lack internal capacity or domain expertise | Knowledge transfer, dependency, scope clarity |
Evaluate vendors on a short list of questions: data handling and retention terms, security attestations, model transparency, evaluation support, integration depth, pricing at ten times your pilot volume, whether “agentic” claims survive a live demo on your data, and exit terms.
One-page pilot charter template
Copy this into your own document and complete every line. Anything you cannot fill in is a readiness gap.
PILOT CHARTER
Use case:
Business owner (accountable):
Program lead:
Finance partner:
Risk / legal / security partners:
Hypothesis:
Using [tool] on [task] will reduce [primary metric] from [baseline] to [target]
without raising [risk metric].
Autonomy level (1 Retrieve, 2 Draft, 3 Act with approval,
4 Act within limits, 5 Autonomous):
Baseline
Measurement method and source system:
Measurement period (2-4 weeks minimum):
Baseline value, with range:
Metrics
Primary metric:
Guardrail metrics and floors:
Leading indicators:
Design
Control or comparison group:
Pilot users (and how they were chosen):
Minimum run time and volume:
Evaluation set size and owner:
Data and risk
Data used, owner, and permission basis:
Risk tier:
Stop conditions:
Decision rule (agreed before launch)
GO if:
ADJUST if:
STOP if:
Decision maker and date:
Costs (per quarter, one-time costs amortized over ___ quarters)
Licenses and usage:
Implementation:
Oversight and monitoring:
Governance and security:
Training and change:
Realization factor assumption (low / expected / high),
and what will happen to the saved time:
Sign-off: business owner ____ finance ____ risk ____ date ____Frequently asked questions
How long should an enterprise AI pilot run?
Plan for about 8 to 12 weeks of live use after a 2 to 4 week baseline. The right length depends on volume: you need enough transactions to see a stable result and to cover normal variation such as month-end peaks.
How many users should be in a pilot?
Enough to represent the real user population, including less enthusiastic staff. Small, hand-picked groups tend to overstate results because champions use the tool differently from the average employee.
What is a good ROI target for an AI project?
There is no universal number. Use your company’s normal investment hurdle, then compare the project with other uses of the same money and effort. Report a range (low, expected, high) rather than one figure.
Should we measure productivity or quality?
Both. Pair every efficiency metric with a quality guardrail, because speed gains that raise error or rework rates often cost more than they save.
When should we stop a pilot?
Stop when a pre-agreed stop condition triggers, when results show no meaningful improvement over the baseline, or when risk exceeds what the use case can justify. Stopping early with a documented lesson is a good outcome.
Who should own AI ROI?
The business leader whose operation changes, with finance as co-owner of the model. A central AI team can enable the work but should not own outcomes it does not control.
Why do AI productivity gains not show up in profit?
Because saved time only becomes money when staffing, throughput, or spend changes. McKinsey’s 2026 survey found 80% of respondents reported personal productivity gains while 37% attributed any EBIT impact to AI. Apply a realization factor to capacity gains and track the budget line that is supposed to move.
Do AI agent pilots need a different design from copilot pilots?
Yes. Agents can take actions, so add least-privilege access, per-action approval at first, spending caps, full action logging, and prompt-injection testing. Start at the lowest autonomy level that can prove the value and promote only on evidence.
Does the EU AI Act delay mean we can pause compliance work?
No. The Digital Omnibus moved Annex III high-risk obligations to 2 December 2027 and Annex I product-embedded obligations to 2 August 2028, but transparency duties apply earlier and classification work still takes time. Confirm current dates with counsel and the official text.
How do we avoid being misled by vendor-reported statistics?
Treat them as hypotheses. Ask for the study design, sample, baseline, and context, and prefer your own measured pilot data.
Sources and method
All sources were checked on 1 October 2026. Where a figure comes from secondary coverage rather than the primary document, we say so, and you should verify it before quoting it.
- McKinsey & Company, The State of AI survey series (2025 and 2026 editions). 2026 figures (1,719 respondents; 37% any EBIT impact; about 6% high performers; 80% personal productivity gains; 40% of $1B+ organizations scaling agents, up from 27%) as reported by The Register, with the 44% enterprise-scaling figure from secondary coverage. Primary report: mckinsey.com. Self-reported survey data.
- Gartner, Gartner Predicts Over 40% of Agentic AI Projects Will Be Canceled by End of 2027, 25 June 2025. A forecast, not a measurement.
- METR, Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity, July 2025. 16 developers, 246 tasks, early-2025 tools.
- Gibson Dunn, EU AI Act Omnibus Agreement: Postponed High-Risk Deadlines and Other Key Changes. Legal commentary on the provisional agreement.
- Jones Walker, Yes, August 2 Still Matters. Commentary on which Article 50 transparency obligations were not delayed.
- Praxikon, The Digital Omnibus and the postponement of high-risk obligations to December 2027. Secondary source for the Regulation (EU) 2026/1744 reference and dates. Confirm on EUR-Lex.
- NIST, AI Risk Management Framework and Generative AI Profile (NIST AI 600-1), July 2024.
- ISO, ISO/IEC 42001:2023, AI management system.
- OWASP, Top 10 for LLM Applications.
How this page is made
- It is a framework, and the numbers inside worked examples are illustrative assumptions, not findings. Market statistics are limited to those cited above.
- We state limits next to each statistic: sample, date, and whether it is a measurement, a survey, or a forecast.
- Regulatory and standards references should be checked against the current official text before you rely on them for compliance decisions.
- This page is reviewed at least quarterly. Corrections are welcome through the Contact page, and material changes will be logged here.
- AInvasion uses AI tools in parts of its editorial workflow, such as research discovery, outlining, drafting, and editing, and this guide was drafted with that assistance. AI does not decide whether a claim is true: claims are sourced, and the editor, Tom Morgan, is responsible for the published text, its interpretation, and its corrections. See the full methodology.
About the author
Tom Morgan, Founder & Editor
Tom is the editorial lead at AInvasion and is accountable for what it publishes, including its corrections. AInvasion presents itself as a named-editor publication, not a newsroom of specialists, so where a topic needs specialist expertise, this guide relies on primary sources, regulators, and published research, all linked below. More on his author page and on how AInvasion verifies claims.
Next steps: Request an enterprise briefing. Related reading: AI security, law and governance, military AI in 2026, AI in healthcare, our methodology.
Edited by Tom Morgan, AInvasion. Last reviewed 1 October 2026. Not legal, financial, or investment advice.