GoModel

GoModel is an open-source AI gateway written in Go that puts a single OpenAI- and Anthropic-compatible endpoint in front of 31 AI model providers. Applications keep using the OpenAI or Anthropic SDK and simply change the base URL, while GoModel handles authentication, workflow resolution, guardrails, caching, budgets, rate limits, provider routing, and failover behind that endpoint. It ships as one self-contained binary with an embedded admin dashboard, released under the MIT license, and is positioned as a self-hosted alternative to OpenRouter and LiteLLM. Its stated purpose is to move provider switching, debugging, and usage tracking out of application code and into one gateway layer. The GoModel site frames the problems it solves around what happens when AI integrations mature. Teams become coupled to one provider, so switching vendors turns into a code project instead of a configuration change. A single runtime behavior rarely fits every team or application: one path needs caching, another needs audit logging, and another needs guardrails. Identical prompts burn budget twice because nothing intercepts duplicates. Provider dashboards show one aggregate total, so costs cannot be attributed to teams, tenants, or features. When a fallback fires during an incident, nobody can reconstruct why. And the gateway itself can become its own project if it needs a separate deployment, admin tooling, and database to operate. GoModel answers each of these by moving that logic into one gateway layer. In routing and provider coverage, GoModel places 31 providers behind one endpoint, including OpenAI, Anthropic, Google Gemini and Vertex AI, Azure OpenAI, Amazon Bedrock, OpenRouter, Cohere, Groq, xAI, DeepSeek, Fireworks AI, Alibaba Bailian, MiniMax, Kimi Code, Z.ai, Xiaomi MiMo, Meta Muse Spark, Kilo AI, OpenCode Go, Oracle GenAI, ElevenLabs, Ollama, and vLLM, each configured through environment variables such as OPENAI_API_KEY or OLLAMA_BASE_URL. Hundreds of models are read from live provider catalogs, and model counts are approximate. Multiple API keys per provider rotate round-robin, and suffixed environment variables register extra instances of the same provider type, so any OpenAI-compatible backend can join as its own provider instance. Aliases and virtual models let teams publish stable names such as smart-chat and remap the real provider and model behind them with a config change rather than an application change. Load balancing spreads a virtual model across targets with weighted round-robin, or lets cost-based routing pick the cheapest capable model for each request. Automatic failover sends availability errors to the next model or provider, with retries, backoff, and a circuit breaker to absorb flaky upstreams. Provider passthrough lets you call any provider's native API through /p/:provider/* while keeping GoModel's auth, usage tracking, and audit on the way through. Control and safety features decide how each request behaves. Scoped workflows toggle cache, audit, usage, budgets, guardrails, and failover per provider, model, or user path, with versioned definitions where the most specific scope wins, so one gateway runs different runtime policies for different workloads. Guardrails inject system prompts or rewrite messages with an LLM before dispatch, running in ordered steps that execute as parallel groups. Virtual API keys give teams managed keys bound to a user path and labels instead of raw provider credentials, and they can be revoked and rotated from the admin UI. Rate limits cap request rate and concurrency per user path, provider, or model; saturated routes are routed around when alternatives exist, and return 429 with Retry-After when they do not. Cost controls are built around tracked usage. Budgets set hard spend limits per user path or label, evaluated from tracked usage cost and enforced before a request is dispatched, so the run stops at the cap rather than at the invoice. Response caching works in two ways: exact-match caching returns identical non-streaming requests straight from the gateway with no provider call and no cost, while semantic caching matches similar prompts and is backed by Qdrant, pgvector, Pinecone, or Weaviate. Cache lookups run after alias and workflow resolution so policy decisions still apply, and cache hits are visible in the dashboard. Usage and cost tracking performs token and dollar accounting per request, user path, and label, with per-model pricing overrides when list prices do not match your contract. On the site's example, a repeated prompt that took 1.9 seconds and cost $0.42 on a cache miss returned in 38 milliseconds at no cost on the second call. Observability covers what happened on every request. Audit logs record each request with its resolved route, workflow, cache result, and provider attempts, with bodies and headers logged only when explicitly enabled. The admin dashboard is an embedded UI for live request logs, usage breakdowns, keys, budgets, workflows, and provider status, so there is no separate deployment to run. Request tagging flows labels from headers or key metadata into usage and audit so spend and incidents map to teams, tenants, and features. Prometheus metrics are exposed at /metrics with request, provider, and circuit-breaker gauges, alongside health endpoints and optional pprof profiling. OpenTelemetry traces and metrics cover every inbound request and provider call on the GenAI semantic conventions, and Jaeger, Tempo, Honeycomb, or Datadog read them as they are. Beyond chat completions, GoModel serves the fuller OpenAI surface including embeddings, the Responses API with gateway-managed conversations, files, and batches, plus the Anthropic Messages API at /v1/messages with token counting, so the Anthropic SDK can be pointed at GoModel and routed to any provider behind it. Audio and realtime coverage includes text-to-speech, transcription, and realtime speech over WebSocket and WebRTC through the same gateway pipeline. An MCP gateway aggregates MCP servers behind one endpoint with namespaced tools, and every tool call gets usage tracking and audit like any other request. A built-in playground sends a real request from the dashboard against any model or alias, streaming or not, and shows the exact JSON both ways while routing, logging, and metering it like any client call. Deployment is a single Go binary with Docker, Compose, and Helm recipes and an embedded admin UI. Storage starts on SQLite with zero setup and moves to PostgreSQL or MongoDB when traffic and retention demand it, using the same binary with a different config. Session keeping pins requests from one conversation or agent task to the target and key that served the first, warming provider prompt caches and keeping audit logs threaded. Streaming is first-class: SSE responses record usage and audit from the stream itself, with no buffering. GoModel authenticates each request, applies the matching workflow covering guardrails, cache, budgets, and rate limits, and routes it to the right provider with automatic failover, all behind OpenAI- and Anthropic-compatible APIs. Requests arrive from the OpenAI SDK, the Anthropic SDK, or plain HTTP and curl against endpoints such as POST /v1/chat/completions, POST /v1/responses, and POST /v1/messages. Cache hits are returned instantly without a provider call. Every response records usage and cost, an audit trail, a cache write, and a live dashboard entry. Provider attempts are protected by retries, backoff, and a circuit breaker. The stated benefits map directly to those mechanisms: provider choice is decoupled from the application so models can be swapped with a configuration change; caching and cost-based routing cut spend without code changes; budgets prevent end-of-month surprises; per-request tracking attributes spend to teams, tenants, and features; audit logs let compliance reviews replay any request including the resolved route, guardrail versions, and provider attempts; and failover turns a provider incident into a routing event rather than a customer-facing one. Local models served by Ollama or vLLM can sit behind the same endpoint the cloud providers use in production, so moving from a laptop to production is configuration, not code. The site lists six concrete jobs teams use the gateway for. A multi-tenant SaaS issues a virtual key per customer, tracks usage by user path, and enforces per-tenant budgets, so invoices come from the dashboard rather than guesswork. A platform team publishes aliases such as smart-chat with scoped workflows behind them, letting product teams ship features without ever holding provider keys. Production traffic rides failover chains with retries and circuit breakers to stay up through provider outages. Caching absorbs duplicate prompts, cost-based routing picks the cheapest capable model, and budgets stop end-of-month surprises. Compliance reviews replay any request with its resolved route, guardrail versions, provider attempts, and full bodies where logging is explicitly enabled. Developers run Ollama or vLLM locally behind the same endpoint the cloud providers serve in production. GoModel targets engineering and platform teams, multi-tenant SaaS operators, and developers who want a self-hosted layer between their applications and AI providers, since the content describes platform teams publishing internal endpoints, product teams shipping without provider keys, and compliance reviewers replaying requests. It integrates with the OpenAI SDK, the Anthropic SDK, plain HTTP clients, MCP servers, and monitoring stacks through Prometheus and OpenTelemetry, with storage on SQLite, PostgreSQL, or MongoDB and semantic cache backends including Qdrant, pgvector, Pinecone, and Weaviate. It runs on macOS, Linux, and Windows, in Docker, Docker Compose, or Kubernetes with a Helm chart. The core gateway is MIT licensed and free; GoModel Pro is the commercial distribution at $4,999 per year or $499 per month, flat per company, backed by a 30-day money-back guarantee and an offline signed license token. Pro adds prompt compression, OIDC single sign-on, per-child quota templates, and intelligent routing in beta. GoModel's value proposition is consolidation: one small, self-hosted Go binary that replaces per-provider integration code with a single OpenAI- and Anthropic-compatible endpoint, then adds the caching, budgets, guardrails, failover, audit, and usage tracking that teams would otherwise build themselves. The benchmark figures in the content, 2.35 ms median latency overhead versus 42.4 ms, 3,610 versus 250 requests per second, 42.7 MB versus 2,173 MB of RAM under load, and a 0.58 second cold start versus 31.25 seconds, illustrate why that consolidation is practical rather than theoretical.