From AI Pilot to Production: The 90-Day Path

The Short Answer

Ninety days works as a scoping constraint rather than a schedule. MIT's Project NANDA study found mid-market top performers averaged 90 days from pilot to full implementation, while firms above $100M in revenue took nine months or longer and reported the lowest pilot-to-scale conversion of any segment. Pick one workflow. Write down its baseline number before day one. Build the evaluation set before the feature. Gate every irreversible action. If you cannot name the metric, do not start the clock.

Why 95% of Pilots Return Nothing

MIT's Project NANDA report, "The GenAI Divide: State of AI in Business 2025," found that despite $30 to $40 billion in enterprise generative AI investment, 95% of organizations were getting zero return, and only 5% of integrated pilots were extracting millions in value. The same report found adoption everywhere. Ninety percent of the companies studied had seriously explored buying an AI solution, and workers at more than 90% of surveyed companies were already using personal AI tools for work.

Adoption and return get measured differently, and the gap between those two measurements is the whole story. MIT counted success as deployment beyond the pilot phase with measurable KPIs, checked six months after the pilot. Seat counts and usage surveys clear a much lower bar. If your business case rests on how many people tried the tool, you are measuring the part that was always going to go well.

The Constraint Is the Point

Enterprises in the MIT study, defined there as firms above $100 million in revenue, led every segment in pilot count and assigned the most staff to AI initiatives. They also reported the lowest pilot-to-scale conversion rates. Resourcing was not the binding constraint. Attention was. A 90-day limit makes eight simultaneous pilots impossible, which is the mechanism.

When three pilots are running and none is in production, cancel two and move those people onto the one with an owner who holds production authority. The segment that converted fastest in that data was not the one placing the most parallel bets.

Days 1 to 15: One Workflow, One Baseline

Two gates before anything gets built. First, does this need an agent. OpenAI's "A Practical Guide to Building Agents" gives three qualifying criteria: decision-making that needs nuanced judgment, rulesets too intricate to maintain, and heavy reliance on unstructured data. If your case does not clearly hit one, the guide says a deterministic solution may suffice. Ship ordinary code with one model call inside it.

Second, write down the metric the pilot must move and its current value, dated. Six months in, nobody accepts a baseline reconstructed after the fact, and reconstruction is exactly what happens when a pilot works without one. A pre-registered number is the difference between proving a win and describing one.

Pick deliberately, too. Executives in the MIT survey put roughly half of generative AI budget into sales and marketing, while the largest savings the report documented sat in back office work: outsourcing elimination worth $2 to $10M a year in customer service and document processing. When two cases are tied on expected value, take the back office one.

Days 16 to 45: Build the Evaluation Before the Feature

This is where the 95% is decided, and the problem is evidence rather than scaling. The pilot dataset was almost always assembled by the person who wrote the prompt. Production sends blank fields, mixed languages, pasted email threads, adversarial users, and formats nobody curated. The system did not change. The input distribution did.

The tau-bench paper put a number on it. It introduced pass^k, which asks whether all k independent attempts at one task succeed, rather than pass@1. Agents including GPT-4o succeeded on under 50% of tasks at pass@1 and fell below 25% at pass^8 in the retail domain. Same tasks, same system, more attempts. Your demo number is pass@1. Your SLA is pass^k.

Three rules for the eval:

1. Run every case five to eight times and report pass^k, not the best run. Put that number in the contract.
2. Use at least 100 labeled cases and report error bars. Anthropic's paper "Adding Error Bars to Evals" found clustered standard errors can run over three times larger than naive ones when questions share a source passage. A three-point gain on 80 clustered questions is noise.
3. If your automated judge shares a model family with the system under test, add a human-labeled holdout. The MT-Bench paper measured strong LLM judges at over 80% agreement with human preferences, roughly the level at which humans agree with each other, alongside documented position, verbosity and self-enhancement bias. Agreement that good still leaves one judgment in five worth arguing about, and the disagreements are not randomly distributed.

Days 46 to 70: The Vendor Calendar and the Cost Model

Ninety days is roughly the maximum safe window between measuring a model and shipping what you measured. Anthropic's published policy is at least 60 days notice before retiring a public model, and practice tracks it: Claude Opus 4.1 was deprecated on June 5, 2026 and retired on August 5, 2026, 61 days later. A nine-month project benchmarks on a model that is legacy at launch, and migration is not free. Anthropic's docs state that temperature, top_p and top_k return a 400 error when set to a non-default value on Claude 4.7 and later, and that the tokenizer introduced with those models counts roughly 1x to 1.35x as many tokens for the same text as the generation before it. The price per million is unchanged. The invoice is not.

Build the cost model in tokens per request, then check your spend tier. Anthropic's Start tier caps at $500 a month and Build at $1,000, and hitting the cap returns HTTP 429 with error code enforced_spend_limit_reached and no retry-after header, so SDK retries keep failing until 00:00 UTC on the first of the next month. Request the increase in week two, not on launch day, and ramp traffic over days, because a sharp usage increase can trip acceleration limits below your published ceiling.

Before adding a second agent, exhaust the first with more tools. Anthropic's engineering team measured agents at roughly 4x the token usage of chat and multi-agent systems at roughly 15x.

Days 71 to 90: Guardrails, Escalation, and the Ramp

If the system reads any text the user did not author, treat indirect prompt injection as reachable. OWASP ranks prompt injection first in its Top 10 for LLM Applications 2025 and states that retrieval augmented generation and fine-tuning do not fully mitigate it. EchoLeak (CVE-2025-32711) is what that looks like in production: an AI command injection flaw in Microsoft 365 Copilot that let an unauthorized attacker disclose information over the network with no user interaction. Microsoft scored it 9.3, critical.

So rate every tool the system can call on four axes, per OpenAI's guidance: read versus write, reversibility, permission scope, and financial impact. Anything high on any axis needs human approval until you have production reliability. OWASP calls the opposite failure Excessive Agency, and its own example is a dependency default rather than a decision: you needed an extension that reads documents from a repository, and the third-party one you picked also modifies and deletes them. Set hard thresholds on retries, tool calls and turns, each escalating to a person rather than looping.

Anything the system can state as a price, a policy, or an eligibility decision is a representation your company is making. Review those paths the way you review a published pricing page, and have the system return values retrieved from your own records rather than values it composes. If this is customer support, build the human escalation path in week one and instrument its rate as a first-class metric. Klarna reported its assistant handling 2.3 million conversations in a single month, two-thirds of chat volume and the equivalent of 700 agents. At that volume the escalation path is load-bearing infrastructure, not a safety net you check quarterly.

Instrument OpenTelemetry gen_ai spans and token usage from day one. Those conventions still carry Development status rather than stable, so budget for attribute churn. Nothing traces itself.

Build In-House or Partner for the First One

MIT measured this directly. External partnerships with learning-capable, customized tools reached deployment about 67% of the time against about 33% for internally built tools, with employee usage nearly double for the external ones. The report is honest that this may reflect organizational capability rather than approach, which is the useful reading: if your team has never taken an LLM system to production, that first one is not the one to learn on.

The same report names the top barrier to scaling, and it is not infrastructure, regulation, or talent. Across 52 organizations, unwillingness to adopt new tools ranked first, and the strongest deployments started with frontline power users rather than a central AI lab.

The 90-Day Go/No-Go Checklist

If a gate fails, cut scope rather than extending the date. Extending the date is how a 90-day project becomes the nine-month one that does not convert.

What This Looks Like in Practice

At SASID the delivery model is built around this shape: proof of concept in days, production in four to eight weeks, then 90 days of monitoring. Across seven production AI systems in five industries, the number worth arguing about at handoff is coverage on real volume, not accuracy on a curated set. The call center QA system is the clearest case, running 100% automated coverage of calls where manual review had only ever reached under 5%.

Get a Free Technical Assessment

If you have a pilot that has not converted, or you are about to start one, we offer a free technical assessment: a 30-minute call about your use case, then a written roadmap within 48 hours covering scope, evaluation design, cost model, and timeline. There is no obligation, and the roadmap is yours to keep. Book at sasid.ai.