IonRouter
Cumulus Labs · Coding
IonRouter is an inference API built by Cumulus Labs that serves open-source and open-weight AI models through OpenAI-compatible endpoints. Point your existing client at its base URL, swap the key, and your code keeps working. The draw is speed and price: the team's custom IonAttention stack multiplexes several models onto one GPU and claims throughput numbers well above typical hosted providers, with pay-per-token billing and no idle costs.
About IonRouter
What Is IonRouter
IonRouter is a hosted inference service for developers who need to call AI models without running their own GPUs. It speaks the OpenAI API format, so the migration cost from another provider is usually one line of code. Under the hood, Cumulus Labs runs a custom engine called IonAttention on NVIDIA Grace Hopper hardware. That engine is the core pitch, and it matters to you because throughput and startup time are what determine your bill and your users' wait times.
The service covers language, vision, image, video, and audio models. You can call flagship options like GLM-5 or Qwen3.5-122B-A10B. You can also bring your own finetunes and LoRAs onto dedicated streams, where they run on their own GPU allocation without sharing a queue. Billing is per million tokens, and you pay only for what you use. There's no free tier to lean on, which is the main thing to plan around before you commit.
The honest limit: this is a developer tool. There's no drag-and-drop builder and no polished end-user chat product beyond a playground. So what does that leave you with? A clean API and a playground, nothing more. If you don't write code, IonRouter isn't for you. It's also a young platform, so if you need a decade of uptime history before trusting a provider, you won't find it here yet.
Getting Started
- Create an account at ionrouter.io and sign in.
- Add credits from the Billing page. Credits are drawn down as you make calls.
- Generate an API key on the Keys page and copy it, since it's shown only once.
- Point your OpenAI client at the base URL and pass your key as a Bearer token.
- Send a chat completion request and start building.
Product Information
A quick look at IonRouter's pricing, supported platforms, and performance.
Best for
The users, tasks, and scenarios where this tool fits best.
Users
- Backend and ML engineers
- Startups watching inference costs
- Robotics and video teams
Tasks
- Swapping out a pricier inference provider
- Running open-source LLMs on your own finetunes
- Processing long documents
Scenarios
- Prototyping an AI feature on a tight budget
- Scaling a production chatbot past a spike
- Building real-time video or surveillance pipelines
Key features
OpenAI-Compatible Endpoints
Any language or framework that already talks to the OpenAI API can talk to IonRouter, which means you point your existing client at a new base URL, swap in a fresh key, and watch the same calls keep working without a rewrite. That means no new SDK to learn and no rewrite of your request handling. For teams already on a hosted model, this is the whole migration.
IonAttention Engine
IonAttention is Cumulus Labs' own inference stack. It multiplexes models on a single GPU, swaps them in milliseconds, and adjusts to traffic as it changes. According to the company, a single GH200 running Qwen2.5-7B hits around 7,167 tokens per second, versus roughly 3,000 for a top inference provider, which the team frames as proof its engine squeezes more work out of the same silicon than the usual hosted setup does. Treat that as a vendor figure. The design goal is real: less idle hardware, lower cost per token.
Per-Second Billing With No Idle Costs
You pay per million tokens rather than per reserved hour. There's no charge for keeping a GPU warm while nothing runs, and that single difference reshapes how you budget for a feature whose traffic you can't predict. For workloads that burst and then go quiet, that pricing shape usually beats a fixed reservation. Your cost scales with what you actually serve. Nothing more.
Custom Models and LoRA Streams
You can deploy your own finetunes, custom LoRAs, or any open-source model on the fleet, and each one gets its own dedicated GPU stream with per-second billing instead of a shared queue that makes your latency unpredictable. This matters if your product depends on a model tuned for your data and you don't want to babysit your own cluster.
Broad Model Catalog
The catalog spans language, vision, image, video, and audio models, so you can pick the cheapest model that clears the bar for a given task and reserve the expensive heavyweights for work that genuinely needs them. Flagship options include GLM-5 for reasoning and coding, Kimi-K2.5 for long-document work, and MiniMax-M2.5 with a million-token context. Cheaper small models like Qwen3-8B and GPT-OSS-120B handle lighter jobs. Prices run from a few cents to a few dollars per million tokens depending on the model.
Dedicated GPU Infrastructure on Grace Hopper
IonRouter runs on NVIDIA Grace Hopper GPUs, and the company is part of the NVIDIA Inception program. Dedicated streams mean your traffic isn't sharing a queue with strangers in a way that spikes your latency. Sub-second cold starts keep the first request of a session from stalling users.
Pros and cons
Pros
- OpenAI-compatible endpoints cut migration to a URL and a key change, so there's little code to rewrite.
- Per-token billing with no idle costs fits spiky or unpredictable traffic better than reserved GPUs do.
- Custom finetunes and LoRAs get dedicated streams. That helps when your product depends on a tuned model.
- A wide model range, from cheap small models to flagship reasoning models, lets you trade cost against capability per task.
- Vendor-reported throughput is high, and sub-second cold starts help interactive and real-time workloads.
Cons
- No free plan, so you have to add credits before you can test anything, which raises the cost of evaluating it.
- Newer and less proven than the big hosted providers, so there's no long uptime track record to check.
- It's built for developers. There's no no-code path or ready-made end-user app if you don't write code.
Frequently asked questions
It's a hosted inference API for calling AI models in your own software. You send requests and get model outputs back, the same way you'd use OpenAI's API, but pointed at open-source and open-weight models running on Cumulus Labs' GPUs.
Related content
Explore related tools, skills, and articles for IonRouter.
IonRouter Alternatives
Forefront
Forefront · CodingForefront is a web platform for building with open-source AI. It lets you fine-tune leading open-source language models on your own data, evaluate how they perform, and run them through an API or export them to host yourself. Developers who want the convenience of a closed-source platform but insist on owning their models and data are the target audience here.
Startkit
StartKit.AI · CodingStartkit is a boilerplate for building AI SaaS and AI wrapper products. Think of it as an AI startup boilerplate with the boring parts already wired up: authentication, Stripe and Lemon Squeezy payments, usage limits, transactional email, and an AI API starter that talks to OpenAI, Anthropic, Groq, or Llama. You clone the repo, set your price, and start on the part of your product that people actually pay for. It's Next.js under React and Tailwind, so most of the boilerplate code already feels familiar.
Testim
Tricentis · CodingTestim is an AI-powered test automation platform for building and running end-to-end tests across web, mobile, and Salesforce applications. It leans on machine learning to keep tests stable when an interface changes, so teams spend less time fixing broken selectors. Not bad for an automated testing tool you can start using today. You create tests by recording actions in a browser, then optionally add JavaScript when you need more control. It's a solid pick for busy QA teams.