The AI Agent Scaling Gap: Moving From Pilot to Production

What AI agent adoption surveys measure and how to move a pilot into production with clear outcomes, live evaluation, review, and rollback.

Published · Updated
8 min read

Key Takeaways

Read the survey question before using the percentage: Scaling a deployment in one business function and running a fully autonomous agent in production are different milestones. Neither measures the share of pilots that failed.
Give the pilot a business outcome: Write down the eligible work, capture the current process baseline, and specify the outcomes and side effects that decide whether the rollout can expand.
Keep a review and rollback path: Start with read-only or human-reviewed operation, track outcomes and exceptions, and make sure the process owner can stop writes without waiting for a new release.

An agent that handles ten clean examples in a demo still has to cope with live records, expired credentials, inconsistent inputs, and work that needs judgement. The decision to expand a pilot should rest on evidence from that workflow. Adoption surveys can help a team understand how peers describe their progress. They cannot tell a team how its own deployment will perform.

What the surveys measure

In its 2025 global AI survey, McKinsey reported that 39% of respondents said their organization had begun experimenting with AI agents and a further 23% reported scaling an agentic AI system in at least one business function. McKinsey defined scaling as expanding deployment and adoption within a function. The survey collected 1,993 responses in 105 countries in June and July 2025 and weighted responses by each country’s contribution to global GDP. The 23% result allows for a deployment expanding within a single function, even if other departments have yet to adopt it. Read the McKinsey results and methodology.

DigitalOcean’s February 2026 Currents report separately found that 10% of its respondents had fully autonomous agents in production. The survey collected more than 1,100 responses from developers, CTOs and founders, with fieldwork in October and November 2025. DigitalOcean also reports that 67% of respondents experienced productivity improvements from using AI agents. The productivity figure and the autonomy figure came from different questions. They do not trace the outcomes of the same set of pilots. Read the DigitalOcean results and methodology.

To measure your own conversion rate, keep a register of pilots started in the same period and define the production milestone before you count anything. For example, an internal support agent might qualify once it routes live tickets for one business unit with monitoring and on-call ownership. Report outstanding and cancelled pilots separately. Include the observation period, so a newly started pilot does not count as an unsuccessful one.

Define what the pilot has to prove

Pick one workflow with a clear entry point, an owner, and an outcome someone can check. For an agent routing support tickets, define which queues it can see, which categories it can route, and who takes an ambiguous ticket. Keep higher-risk actions such as sending a reply or closing a complaint behind review unless they are explicitly in scope and evaluated separately.

Measure how the current process handles the same work before running the comparison. Capture routing accuracy, time until the customer reaches the right queue, reassignment, and the staff time spent reviewing mistakes. Record the ticket mix because an apparent improvement may come from giving the agent easier work. Define how the workflow should behave when the source record is stale or the queue has been renamed since the pilot started.

Write go and stop criteria together. A team could accept a release only if it meets the existing routing baseline, leaves sensitive tickets in the review queue, and consistently identifies cases it cannot route. Choose numerical thresholds using the cost of a mistake and the performance of the existing workflow. A global adoption statistic cannot supply those thresholds.

Test the outcome and the actions

Build an evaluation set from representative tickets, known exceptions, and cases in which escalation is the right answer. For each case, record the allowed outcome and any actions that would be unacceptable. Keep some examples aside from prompt development, and add failures observed during the rollout so they are tested before the next release.

Check the state in the target system. A fluent message saying a ticket was reassigned cannot prove that the help desk received the update. Anthropic’s guide to agent evaluations distinguishes the outcome from the transcript and describes combining code, model, and human graders. Use deterministic checks for write outcomes, permissions and duplicate actions. Have people adjudicate cases in which the correct answer depends on a policy or a judgement call.

Log enough information to reproduce a failure without copying unnecessary customer data into monitoring. Record the relevant tool response, action proposed and taken, outcome, and model and workflow version. An aggregate success rate alone can hide a regression concentrated in a small but sensitive ticket category.

Roll out in stages

First, replay historical examples in a test environment. Once those pass review, let the agent propose actions on live inputs while the existing process retains control. That shadow run reveals missing fields and permission failures the replay did not cover. Also check whether the person reviewing a proposed action can see enough evidence to catch an error quickly.

Next, grant access to a limited live queue with review for the actions a mistake would make hard to reverse. Expand only after the workflow meets the agreed quality, safety, and cost criteria. OpenAI’s guide to building agents describes human handoff for actions the agent cannot complete safely, including when repeated attempts fail or an action poses higher risk. Give the reviewer the context and action needed for that decision.

Test how the system behaves when a tool times out after committing a write. Before retrying, check whether the original action already applied. Establish how the owner can pause new actions, recover the affected records, and route queued work to the previous process. Rollout size should reflect how quickly the team can detect and contain a bad release.

Run it as an operational service

Separate support from release decisions. The process owner sets the quality threshold and decides which cases the agent can handle. The technical owner maintains service health, permissions, and version controls. Both need access to the same outcome and escalation data. A named team with an escalation path is enough for a bounded deployment if it can do this work reliably.

Track successful resolutions as a share of eligible work, and show how much work went to people. Report the cost of successful work after including model calls, tool calls, review, retries, and rework. A team may be able to handle more cases after rollout while paying more per completed case. Both are useful facts for a decision about funding and scope.

Run the release tests whenever a model, prompt, integration, or permission changes materially. Keep a way to restore a known configuration while reconciling actions that have already reached downstream systems. Use the agent observability guide to plan how evaluators, traces, and costs fit together.

Launch checklist

  • Define the eligible work, business owner, and production milestone.
  • Compare live outcomes against a baseline from the same workflow.
  • Check the downstream state and preserve traceable reasons for escalations.
  • Test partial writes, repeat actions, tool outages, and the manual fallback.
  • Agree on who can expand or pause the rollout and the criteria for doing so.

Expand the deployment once the team can show which work is being handled well, how it knows, and what happens to the next ticket when the agent is unavailable. Use the AI transformation service if you need help defining scope and testing the handoff.

Corrections

Correction, : An earlier version presented an unidentified March 2026 survey of 650 technology leaders as the source for a 78% pilot rate, a 14% scaling rate and related breakdowns. We could not substantiate that survey and have withdrawn those findings. A related article said 90% of AI agent pilots failed to reach production, citing DigitalOcean. Its February 2026 Currents report surveyed more than 1,100 respondents and found that 10% had fully autonomous agents in production. It did not measure the share of pilots that failed. This guide describes the actual survey measures and how to evaluate an individual rollout.

Frequently Asked Questions

Digital Applied newsletter

Deep dives on AI, marketing and development.

Practical guides and fresh insights by email. No recycled takes.

Related Articles

Continue exploring with these related guides