Skip to content

Meet DeepintShield at Bengaluru Tech Summit 2026, 17 to 19 November, BIEC Hall 1, booth H1A106. Book 20 minutes

Start For Free

Buyer’s guide to self-hosted AI gateways

How to compare self-hosted and hybrid AI gateways: deployment models, questions to ask, what to test and who operates what.

An AI gateway sits between your applications and the model providers they call. It routes requests, applies policy and records what happened. Choosing a self-hosted gateway is also a choice about who operates it and where information travels. This self-hosted AI gateway buyer’s guide helps platform, security and procurement teams compare options on evidence rather than labels.

The guide is vendor-neutral. It covers the three common deployment models, the questions to put to every vendor, the tests to run in a proof of concept, and the operating work that stays with you after installation. Use the same requests, prompts and expected outcomes for every candidate, and decide your mandatory requirements before the first demonstration. A long feature list does not make up for one missing control your application cannot run without.

Hosting location supports a data-handling design; it does not settle it. A gateway inside your network can still send prompts to an external model provider, and an external trace exporter or classifier adds another destination. So start every evaluation by drawing the data path for one complete request.

Deployment models to compare

  • Vendor-managed (SaaS). The vendor runs the control plane and the data plane. Fastest to start and least operating work; request content is processed in the vendor’s environment.
  • Hybrid (split plane). The vendor runs the control plane; you run the data plane in your cloud account. Check exactly what crosses the link between them, and what happens when it is down.
  • Fully self-hosted. You run both planes, sometimes air-gapped. Most direct control; you own upgrades, certificates, capacity and recovery.

Draw the data path first

List the application, gateway, model providers, retrieval system, log store, backups, telemetry destinations and any external safety services. For each connection, write what it carries: request content, credentials, usage metadata or configuration. Then confirm the real route with harmless marker requests. A diagram drawn before a new provider or exporter was enabled is no longer accurate.

Eight questions to ask every vendor

  1. What content can it inspect? Test prompts, responses, retrieved passages and tool data, including attachment limits.
  2. What authorises an agent action? Ask for caller identity, exact resource and tool permission, independent approval, and behaviour when authorisation is unavailable.
  3. How does it manage usage? Test model restrictions and budgets, then measure caching or routing against answer quality.
  4. Where does the data go? Trace every destination and name an operator for each component.
  5. Which API operations work? Verify your models, streaming, tool continuation and stored-resource lifecycle, not a count of provider logos.
  6. What can operators observe? Follow one request through timing, outcome and usage records.
  7. Can reviewers verify the evidence? Export a record and check its integrity method, retention and access controls.
  8. What does continued operation need? Confirm plan, dependencies, upgrade process, support terms and exit options.

For self-hosted options, add: which databases, caches and detector services must you run; how upgrades are delivered; whether the product calls home; and how it behaves when its control plane or licence service cannot be reached.

What to test in a proof of concept

  • Performance with controls on. Measure time to first output, total time and task success with the inspection and authorisation you will run in production. Keep cached and uncached requests apart.
  • Failure. Make a provider time out, make authorisation unavailable, and take the database or cache away. Record which requests finish, fail or need an operator.
  • Change. Change a tool schema, a source or an approved provider route, and watch the review and rollout process.
  • Recovery. Restore a backup into a disposable environment and confirm configuration, access and records return.

Assign the operating work

Name who patches software, renews certificates, rotates secrets, monitors capacity, restores backups and answers incidents. Include every supporting service, not just the gateway container. Managed operation moves some of this work to a vendor, but application ownership, data classification and an incident contact stay with you.

Score without hiding hard requirements

Weight the questions before testing and mark the requirements that must pass regardless of score. Keep the raw cases, versions and configuration with each result so the comparison can be re-run after a product update.

Where DeepintShield fits

Apply the same tests to DeepintShield. It offers a managed cloud, a hybrid Enterprise VPC option with the data plane in your cloud, and Enterprise Self-Hosted with an air-gapped option. In hybrid and self-hosted modes, no data leaves your environment. See Hybrid Cloud, Self-Hosted and the deployment docs.

Further reading: AI gateway comparison: eight questions to test, self-hosted vs SaaS AI security and the comparison pages.

Start For Free