Skip to content

Meet DeepintShield at Bengaluru Tech Summit 2026, 17 to 19 November, BIEC Hall 1, booth H1A106. Book 20 minutes

Start For Free

Platform · AI Gateway

AI gateway: one OpenAI-compatible API for all your model providers

An AI gateway is a single entry point between your applications and the LLM providers they call. The DeepintShield AI gateway puts 29 built-in model providers behind one OpenAI-compatible API, with weighted load balancing, fallbacks, budgets and request logging built in. Run it as a managed service or inside your own environment.

  • 29 built-in model providers
  • Bring your own keys, 0% token markup
  • Managed, hybrid or self-hosted
AI gateway illustration: one central gateway connected to many model provider endpoints

Why teams need an AI gateway

Once more than one team ships an LLM feature, the same three problems appear.

Every app wired to a different SDK

Each team calls providers in its own way, so switching models or adding a control means changing every codebase.

Outages and rate limits reach users

When one provider returns errors or 429 responses, an app with no fallback path fails in front of customers.

Keys and spend scattered across teams

Provider keys live in many places, and nobody can see or cap what each team or customer spends.

How the AI gateway handles a request

1. Your app calls one API

Requests reach one OpenAI-compatible endpoint, authenticated with a DeepintShield virtual key.

2. Governance checks run

Model access, budgets and rate limits are checked for the customer, team, virtual key and provider.

3. Routing picks a provider and key

Weighted load balancing selects a key; errors and timeouts move the call to your fallbacks.

4. Usage and cost are recorded

Tokens, cost and latency are logged and exposed to OpenTelemetry and Prometheus.

AI gateway features

Routing, resilience and spend control live in the gateway, so each application stays simple.

One OpenAI-compatible API

Keep OpenAI, Anthropic, Google GenAI, Bedrock or LiteLLM client code; change the base URL and the key.

Weighted load balancing

Spread traffic across keys and providers by weight, with per-key model lists and Azure or Bedrock deployment mappings.

Fallbacks and circuit breaking

Errors, 429s and timeouts move the call to your backup providers in order; a circuit breaker removes failing keys.

Adaptive load balancing

On Team and above, traffic shifts to the best-performing keys by error rate, latency and utilisation.

Budgets and rate limits

Set budgets and request or token limits per customer, team, virtual key and provider; over-budget providers leave the route.

Routing rules by plan

Add rules that steer requests between providers: 1 on Developer, 10 on Team, 25 on Business, unlimited on Enterprise.

One console for providers, models and spend

The Model Hub lists every configured provider and model, and GET /v1/models returns the live list. Override prices for negotiated rates, and see cost by virtual key, team and customer. Provider prompt caching is on by default, which lowers input cost on supported providers with no change to your app.

  • Model catalogue with custom pricing overrides
  • Per-model budgets and rate limits on Team and above
  • Prometheus metrics for cost, cache hits and time to first token

Change one line of code

Point your existing OpenAI client at the gateway and use a virtual key instead of a provider key. Select any provider with a provider/model prefix, such as openai/gpt-4o-mini. Provider credentials stay on the gateway. For new projects, the Quick Start walks through the first call.

  • Python and server-side JavaScript clients
  • Streaming, async and custom headers keep working
  • Provider-native passthrough routes when you need them
import openai

client = openai.OpenAI(
    base_url="https://app.deepintshield.com/openai",
    api_key="sk-ds-your-virtual-key"
)

Mapped to

  • OWASP LLM Top 10
  • NIST AI RMF
  • ISO/IEC 42001

Product facts

  • 29 built-in model providers
  • OpenAI-compatible API
  • No logging surcharge
  • 0% token markup

AI gateway questions

What is an AI gateway?

An AI gateway is a single entry point between your applications and the model providers they call. It gives every app one API and applies routing, fallbacks, budgets and logging in one place, instead of in each codebase. See the full request path.

Do I need to change my code?

Usually only the base URL and the API key change. Point an OpenAI, Anthropic, Google GenAI or Bedrock client at the gateway and authenticate with a virtual key. Compatibility is route-specific, so test the exact endpoints and SDK version your application uses.

Which LLM providers are supported?

DeepintShield ships 29 built-in model providers, plus custom OpenAI-compatible endpoints. They include OpenAI, Anthropic, Azure, AWS Bedrock, Vertex AI, Google Gemini, Mistral, Cohere, Groq, DeepSeek, xAI, Ollama and vLLM. Browse all integrations.

What happens when a provider fails?

The gateway tries your fallback providers in the order you set. Network errors, 5xx responses, 429 rate limits, timeouts and unavailable models trigger a fallback; validation errors and policy blocks do not. If every attempt fails, you get the original error. Read the fallback docs.

Does the gateway add latency?

It adds some, and the amount depends on your workload, policies and deployment. DeepintShield publishes mechanisms rather than an end-to-end figure: speculative dispatch makes allow-path latency max(guards, model) instead of guards + model, and eligible deterministic guard decisions are reused in about 1.42 µs (local microbenchmark, no I/O). Measure it on your own traffic.

How much does it cost?

The Developer plan is free for 50,000 requests a month, and Cloud Team starts at $49 a month. You bring your own provider keys and pay providers directly, with 0% token markup. Compare plans and pricing.

Further reading on AI gateways

Route every model call through one gateway

Start free with 50,000 requests a month, or talk to us about running the gateway in your own cloud.

Start For Free