vLLM
Self-hosted vLLM servers: chat, embeddings, transcription and reranking.
Put your own vLLM inference servers behind DeepintShield and govern them like any hosted provider.
What works
- Chat Completions, with Responses through a chat fallback
- Text completions and embeddings
- Rerank (needs a rerank-capable model on the server) and transcription (STT)
- List models and streaming
Limits to know: No image generation, speech, files or batch. An API key is optional; when set, it is sent as Authorization: Bearer.
How to connect
Set the server address in network_config.base_url in Model Hub → AI Providers, create a Virtual Key, then call vllm/<model>.
Read the vLLM provider docs. Requests get the same Virtual Key budgets, guardrails and logs as every other provider on the AI gateway.