Barry Morris is chief product and strategy officer at Couchbase, where he oversees product direction and long-term technology direction.

getty

Enterprises have poured billions into faster models, smarter agents and sharper algorithms. Think of these as high-performance vehicles with bigger engines, wider tires and better aerodynamics. However, most organizations are trying to drive them through an overgrown jungle of data silos, slow retrieval paths and ungoverned access with no bridges between cloud and edge. The vehicles stall, and the passengers disappear.​

Winners don’t build tougher trucks. They build highways.

A team gets a compelling prototype working against a curated dataset, leadership gets excited, and then the project hits the wall of production requirements with multiturn conversational state, sub-millisecond retrieval across billions of records, real-time decisioning at global scale and compliance constraints that demand data never leave certain jurisdictions. The prototype’s stitched-together stack can’t carry the traffic. The problem was never the vehicle. It was the road.​

Gravel paths don’t scale.

Conventional wisdom says you pick the best tool for each job. Need vector search? Add a specialized vector database. Need session state? Bolt on a cache layer. Need structured queries alongside unstructured retrieval? Wire up another system and synchronize it with pipelines.

This sounds reasonable in a whiteboard session, but it gives you a network of gravel paths when what you need is a highway. Every integration point is a pothole. Every synchronization pipeline introduces drift. Every additional system adds another stretch of unpaved road that your AI traffic has to crawl across.​

The more systems you stitch together, the more your infrastructure resembles the Darién Gap. It’s impassable at speed and punishing at scale.​

What does the highway look like?

A production-grade data highway for AI rests on four properties engineered into the roadbed, not bolted on after traffic starts moving.​

1. Persistent operational memory is the surface itself. AI agents that lose context between sessions are vehicles with no traction. An agent handling procurement, compliance review or customer service needs to remember what happened in the last conversation, what decisions were made and what constraints apply. That memory can’t live in an ephemeral cache that evaporates when a container recycles. It needs to be durable, queryable and fast.​

2. Vector and hybrid search that scales is the lane capacity. Semantic search is foundational to retrieval-augmented generation, but production workloads require combining vector similarity with keyword matching, structured metadata filtering and geospatial constraints in a single query, returning results in sub-millisecond timeframes across billion-scale datasets. Split these across separate systems, and you get a bottleneck at every merge point.

3. Unified indexing across structured, unstructured and vector data eliminates the interchanges where traffic backs up. The ETL pipeline synchronizing your relational store, vector database and document store is a source of data staleness that directly undermines AI accuracy. When an agent retrieves context that’s even minutes old, it makes decisions on stale information. A highway doesn’t require you to exit, refuel at three different stations and reenter.​

4. Deployment flexibility is the on-ramp system. Regulated industries need on-premises routes. Latency-sensitive applications at the tactical edge need local lanes. Global organizations need the ability to place data and compute where compliance and performance requirements dictate. A highway that only connects two cities isn’t a highway. It’s a toll road with limited utility.​

We worked with Agora, whose conversational AI platform handles outbound sales, customer service and physical AI. All three depend on a single data layer that returns results fast enough for human-to-AI conversations to feel natural. The partnership started years before agentic workloads entered the picture. Extending it was straightforward once Agora needed the same retrieval speed and consistency for agents that it already relied on for its core platform.

One question changes the conversation.

For CIOs and CTOs evaluating AI infrastructure, this framework shifts the question from which vector database they should buy to whether their data plane carries the traffic patterns that agentic AI actually generates.

Those patterns are fundamentally different from traditional application workloads: unpredictable query volumes as agents explore and reason, multiturn conversational loads requiring persistent state, real-time decisioning where an extra hundred milliseconds of latency means a worse answer or a missed opportunity. Web-era infrastructure was built for predictable commuter traffic. Agentic AI generates freight-hauling volume at Formula 1 speeds.​

The “build vs. buy” calculus changes, too. Assembling five or six specialized systems and maintaining the glue between them isn’t cheaper than investing in an integrated data plane once you account for engineering time lost to synchronization, the incident-response burden of multisystem failures and the opportunity cost of slower iteration.​

Waiting comes with a toll.

In the vast majority of the companies we’re working with, the data infrastructure conversation for AI agents is consolidating around a small number of architectural patterns. I believe the organizations that pave their highways now will compound that advantage, not because first movers always win but because data infrastructure decisions are sticky. Once agent workflows are built on a particular data plane, ripping up the road is extraordinarily expensive.​

Your vehicles are almost certainly fast enough. The question is whether the road beneath them can carry the load when those vehicles are finally asked to haul real freight, at real speed, for real users who won’t tolerate gridlock.​​​

Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?