GoModel

GoModel puts OpenAI, Anthropic, Gemini, Bedrock, Vertex, Azure, Groq, Ollama, vLLM, and others behind a single endpoint. That's 33 providers. It stays compatible with the OpenAI and Anthropic request formats, so you change the base URL and keep the SDK you already know. It also rotates across multiple keys per provider with round-robin, which spreads load and reduces the blast radius of a single throttled key.

You publish stable names like smart-chat and remap the real provider and model behind them without touching application code. Why does that matter? Teams get a predictable interface while you experiment with what's underneath. The project frames this as a config change, not an application change, and that's the whole point.

When a provider returns an availability error, GoModel fails over to the next model or provider, retries with backoff, and uses a circuit breaker to absorb flaky upstreams. It can also spread a virtual model across targets with weighted round-robin, or let cost-based routing pick the cheapest model that can handle a request. Failover is automatic. That's the whole appeal.

Cache, audit, usage tracking, budgets, guardrails, and failover are all toggles you set per provider, model, or user path. Workflows are versioned, and the most specific scope wins, so one route can require audit logging while another runs lean. That's flexible. It's how you apply different behavior to different teams without a pile of conditional logic.

GoModel can inject system prompts or rewrite messages with an LLM before a request goes out. The steps run in ordered groups, and parallel groups execute together. It's a way to enforce tone, block certain content, or normalize inputs centrally instead of asking every service to do it. Less duplicated logic.

Exact-match caching returns identical requests faster and cheaper, so the same prompt doesn't pay full price twice. This matters most in agent loops and retries, where near-duplicate calls pile up and quietly burn budget. Small change, real savings.

Per-request tracking maps spend to teams, tenants, and features, going past the single total a provider dashboard shows. Audit logs and runtime metadata make every request traceable, which is what you want when a fallback fires and nobody remembers why. No guessing.

The gateway ships as one Go binary with a built-in admin UI and storage that scales with your workload through SQLite, Postgres, or MongoDB. Prometheus metrics and SSE streaming are included, so observability hooks exist from day one. Nothing extra to bolt on.