The Rise of the AI Gateway: Assessing CData's Connect AI Gateway

As enterprises move from AI pilots to production, Omdia is assessing how AI gateways can help address challenges around governance, cost, and business context. This Analyst View explores CData Connect AI Gateway's approach to connecting models, tools, and agents with enterprise data, examining how unified controls, record-level governance, and its Context Engine could support more accurate, efficient, and manageable AI deployments.

Every enterprise AI leader recognizes the pattern. The pilot impresses stakeholders. The demo generates enthusiasm. Then come the questions that determine whether AI reaches production: Who approved this agent's access to customer records? Why did last month's token bill double? And why does the sales team still verify every answer?

These questions center on control, cost, and context: the operational challenges that AI gateways are designed to address. CData's recently announced Connect AI Gateway, now open for early access, represents one approach to these challenges. Examining its architecture reveals how the AI gateway market is evolving from basic traffic management toward more sophisticated context and governance capabilities.

Understanding AI Gateways

An AI gateway functions as a control point between AI consumers (whether people, assistants, or autonomous agents) and the systems AI accesses. Every request passes through the gateway on the way to a model and into business systems. The gateway authenticates requests, routes them to appropriate models, enforces data and tool access policies, and logs all activity.

As organizations scale from isolated AI experiments to hundreds of assistants and agents operating across dozens of systems, this centralized control point becomes increasingly critical for managing complexity, ensuring security, and maintaining visibility.

CData's Data-First Architecture

CData's approach is characterized by its starting point: the connectivity layer where AI meets enterprise systems. This differs from gateways built primarily at the model edge to monitor prompts and responses. The distinction reflects CData's two-decade history building live, read-write connectivity to hundreds of enterprise systems, a foundation the company is now extending to AI governance.

The platform comprises four layers:

Gateway Layer: Three specialized gateways handle different interaction patterns. The Model Gateway registers LLMs and routes requests across providers like Anthropic, OpenAI, and Google, managing failover and token budgets. The MCP Gateway governs how AI accesses enterprise tools through the Model Context Protocol, an emerging standard for agent-to-system connectivity. The Agent Gateway manages multi-agent orchestration and agent-to-agent handoffs. A shared governance and observability layer provides unified logging and policy enforcement across all three.

Security and Controls: The platform enforces identity and policy before executing requests. Passthrough identity maps each request to individual user credentials in source systems, ensuring agents operate with the same permissions as the people they assist. Enterprise SSO, SCIM provisioning, PII detection, masking, and encryption provide additional security layers.

Context Engine: This component addresses the challenge of business-specific understanding. Models lack knowledge of company-specific terminology, authoritative data sources, and cross-system relationships. The Context Engine combines four types of context: company knowledge learned from user interactions, semantic definitions from existing business models in tools like dbt and Snowflake, system context describing source contents, and data modeling that creates derived views and cross-source joins. These elements form a queryable context graph that teams can visualize and refine.

Connectivity Foundation: The base layer provides live access to hundreds of enterprise sources through a consistent interface, leveraging CData's existing connectivity capabilities.

When a user asks "What enterprise revenue is at risk this quarter?", the platform authenticates the user, routes the request to an appropriate model, retrieves relevant definitions of "revenue" and "at risk," identifies join paths across systems like Salesforce and NetSuite, executes a governed federated query, and returns results. User refinements to definitions can be captured for organizational reuse.

The Economics of Context

Note: Performance figures cited below reflect CData's own testing and have not been independently validated by Omdia.

While discussions of AI cost typically focus on model pricing, significant cost and accuracy levers exist at the data layer. Four factors are particularly relevant:

Model Selection Economics: When gateways provide accurate business context and enforce safety controls, organizations can route requests to appropriately-sized models rather than defaulting to premium options. CData's testing across 22 models found cost variations up to 175× for equivalent correct answers, a range that suggests substantial optimization opportunity.

Semantic Efficiency: Models that understand business terminology require fewer clarifying exchanges and retries, reducing both token consumption and human verification time. In CData's study of 378 enterprise queries, the platform reported 98.5% accuracy compared with 65-75% for other MCP providers on identical questions. While these are vendor-reported figures, they illustrate how contextual grounding can affect both cost and trust.

Data Processing Location: Sending raw records to models for filtering and aggregation is resource-intensive. Processing data within the gateway platform and returning only results reduces prompt size and improves reliability.

Governance as Enabler: Strong governance (per-user identity, record-level policies, complete audit trails) gives security teams confidence to approve broader data access. Broader access enables more valuable AI applications. Cost attribution and rate limits provide financial visibility that supports rather than constrains AI investment.

These factors can reinforce one another: better context enables cheaper models, better data reduces consumption, stronger governance expands safe access, which improves accuracy further.

Market Evolution: From Traffic Control to Context Management

The AI gateway market is shifting from first-generation capabilities (routing, rate limiting, spend tracking) toward more sophisticated context and governance features. This evolution is reshaping competitive dynamics.

Model providers, hyperscalers, and API management vendors are all entering the gateway space, each with distinct advantages. However, deep contextual understanding requires extensive connectivity to enterprise systems, semantic modeling capabilities, and security that follows users to source systems. These are capabilities typically built over years.

Simultaneously, enterprise AI is becoming multi-model and multi-agent. Organizations want flexibility to use optimal models for specific tasks and to combine assistants from different providers. This trend increases the value of neutral, model-agnostic control points, since individual model vendors face challenges governing competitors' traffic.

CData's approach reflects this market direction. The company brings established connectivity, a growing partner ecosystem including Microsoft and ServiceNow, and financial stability (over $100M in annual recurring revenue, profitable since inception). The platform's model-agnostic positioning aligns with enterprises' desire to avoid vendor lock-in.

Considerations for Enterprise Evaluation

Organizations assessing AI gateway solutions should consider several factors:

Governance Depth: How does the solution handle record-level security across diverse source systems? Can it enforce policies that respect both gateway-level and source-system permissions?

Architectural Flexibility: Does the platform support multi-model and multi-agent architectures? Can it accommodate both current and emerging protocols?

Context Mechanisms: What capabilities exist for capturing and operationalizing business context? How much semantic modeling investment is required?

Cost Visibility: How transparent is token consumption and cost attribution? Can the platform support intelligent model routing based on request complexity?

Integration Requirements: What connectivity already exists versus what must be built? How does the solution handle both real-time and batch data access patterns?

The effectiveness of any AI gateway will depend on how well its architecture aligns with an organization's specific challenges, whether those center on model orchestration, data governance, contextual accuracy, or cost management.

Looking Ahead

AI gateways are becoming as fundamental to enterprise AI as API gateways became to modern applications. The organizations that implement effective gateway strategies will likely spend less per answer, trust their answers more, and move from pilots to production with greater confidence across IT, finance, and business stakeholders.

CData's Connect AI Gateway illustrates how providers are addressing the gap between AI capability and enterprise requirements. The platform's data-first architecture, emphasis on business context, and comprehensive governance represent one path forward as the market matures.

As enterprises move beyond experimentation, the question is less whether they need gateway capabilities and more which architectural approach best fits their requirements. The shift from traffic management to context management suggests that solutions built on strong data foundations may have advantages, but success will ultimately depend on execution, ecosystem development, and how well platforms adapt to rapidly evolving enterprise needs.

The AI gateway market is still early, and multiple approaches will likely find success serving different enterprise priorities. What's clear is that the operational challenges of production AI (governance, cost, and context) require systematic solutions, and gateways designed to address all three dimensions will be better positioned as enterprises scale their AI deployments.