Skip to content

Meet DeepintShield at Bengaluru Tech Summit 2026, 17 to 19 November, BIEC Hall 1, booth H1A106. Book 20 minutes

Start For Free

vLLM

Self-hosted vLLM servers: chat, embeddings, transcription and reranking.

Put your own vLLM inference servers behind DeepintShield and govern them like any hosted provider.

What works

  • Chat Completions, with Responses through a chat fallback
  • Text completions and embeddings
  • Rerank (needs a rerank-capable model on the server) and transcription (STT)
  • List models and streaming

Limits to know: No image generation, speech, files or batch. An API key is optional; when set, it is sent as Authorization: Bearer.

How to connect

Set the server address in network_config.base_url in Model Hub → AI Providers, create a Virtual Key, then call vllm/<model>.

Read the vLLM provider docs. Requests get the same Virtual Key budgets, guardrails and logs as every other provider on the AI gateway.

Read the integration docs

Start For Free