Production, Not Pilots: The New Benchmark for Generative AI Success

A year ago, it was all about experimentation. Enterprises were testing what generative AI (GenAI) could do: building chatbots, repurposing content, and automating summaries. The early results were intriguing, but inconclusive. Leaders called it promising. Others called it hype.

Fast forward to now, and something subtle has changed. Quietly, behind the scenes, a growing group of organizations is moving past pilots and prototypes. They’re going into production. And they’re seeing results.

This shift isn’t about chasing shiny objects. It’s about building something real and reaping the benefits. But not everyone is keeping pace. In fact, most aren’t even close.

That’s the shift we’re seeing. GenAI has a new dividing line, and it’s not between early adopters and laggards. It’s between those in production and those still stuck in pilots.

What the data reveals

Vertesia surveyed 400 enterprise leaders across IT and digital roles. What we found was telling: organizations with custom GenAI solutions in production are not only seeing value, they’re seeing more value than expected. These are teams that have already moved from exploration to execution. And their investment is paying off.

But they’re the minority, with only 30% of respondents having GenAI deployed in production. The rest were still testing, stuck in proof-of-concepts. And as a result, 23% admitted they already felt behind.

This scenario isn’t surprising. Pilots can take time, and it takes a certain level of strategic foresight to commit to creating an environment for long-term success rather than quick wins. But when you get into GenAI production mode, momentum builds, and the competitive gap really starts to widen.

Time becomes the constraint

It’s not that business leaders don’t see the potential. They do. But many underestimate what it takes to get there.

Rolling out a GenAI solution isn’t a weekend sprint. Preparing data, aligning stakeholders, and finding the right technologies all take time. In our survey, over a third of organizations said they spent more than three months just preparing data. Another third said the deployment phase took just as long, or longer.

That means even a well-structured initiative can take six to twelve months to move from idea to impact. Which raises two fundamental questions: 

1. If it takes this much time and investment to deploy one solution, is this a sustainable approach to deploying GenAI applications?
2. And more importantly, if you want to see value this time next year, have you started yet?

What GenAI in production looks like

When we talk about GenAI in production, we’re not talking about a single chatbot. We’re talking about multiple solutions, embedded in different parts of the business, and solving various problems.

The most mature organizations we surveyed had between two and five GenAI use cases already live. That might include generating personalized marketing content, surfacing insights from internal knowledge bases, or supporting decision-making in frontline operations.

These organizations aren’t aligning to just one inference provider (Amazon Bedrock, Google Vertex AI, etc.) or model family (OpenAI’s GPT, Anthropic’s Claude, etc.). They’re investing in a programmatic approach that allows them to scale GenAI solutions quickly, safely, and repeatedly. They’ve moved beyond experiments and into capability-building.

They’re also thinking seriously about accuracy. The further GenAI moves into core business processes, the more important it becomes to trust the outputs. That’s why we’re seeing a shift towards strategies like semantic RAG (retrieval-augmented generation) that ground large language models (LLMs) in verifiable enterprise data to ensure responses are not only plausible but also correct. According to Gartner, by 2028, 80% of enterprise GenAI apps will be powered by internal data using RAG-based architectures. Today, that figure is under 20%, which means the next three years will be a period of rapid change. 

Equally, organizations are increasingly adopting a multi-model approach to strengthen the accuracy of results. Each model family and inference provider has its own strengths and weaknesses, so the ability to create AI models from any of these frameworks and then compare and even combine the outputs to get optimal results is incredibly beneficial.

A narrowing window

If you haven’t started your journey towards production-ready GenAI, don’t worry, there’s still time to act. But don’t wait too long because that window is closing.

Organizations that move now have a chance to lead the charge. To define best practices, to learn what works best, and to reap the rewards.

Those who wait will likely find themselves in catch-up mode – struggling to cope with the pace of change, having to explain internally why others are moving faster, and lacking the ability to go beyond experiments to get the best from their GenAI efforts.

Moving from ambition to action

At Vertesia, we speak to teams every day that want to accelerate GenAI adoption but feel overwhelmed by where to begin. That’s completely understandable. The space moves fast, the stakes are high, and the challenges are real.

That’s why we created “From Experimentation to Execution: Accelerating Generative AI Deployments”, a step-by-step, practical, plain-language resource for business and technology leaders looking to make the most of GenAI. It walks through the steps needed to move from pilot to production, including:

• Getting alignment across business and technical stakeholders
• Preparing and governing your internal data
• Choosing the right delivery model (build, buy, partner)
• Avoiding the most common barriers to scaling

You can download the IT guide for free here.

Closing the GenAI Gap — Your Move

Despite GenAI being a relatively new technology, the bar for success has already been raised. It’s no longer about who’s experimenting. It’s about who’s executing, and doing so with purpose, governance, and strategic intent.

That shift isn’t theoretical. It’s here.