Substrate AI Inference Platform

You defined a program as nodes with dependencies, not as a sequence of HTTP calls. Substrate read the whole graph first, then scheduled independent branches to run at the same time. On a job with three parallel text generations and one join step, that meant the join waited for the slowest branch instead of running everything one after another. This is what the product meant by multi-inference: one request, many model calls, one result.

The platform shipped ready-made nodes so you didn't have to host a model to get started. Text generation, JSON extraction that returns typed fields, image generation, speech transcription, and embeddings were all callable with the same client. A common pattern was to draft with a text node, force the output into a JSON schema, and store the result, all in a single run without moving data between providers.

Two nodes, one for Python and one for JavaScript, let a program run ordinary code next to the model calls. That covered the parts models are bad at: parsing a CSV, calling an internal endpoint, reshaping a payload. Because the code node ran in the same cluster as the model nodes, you could pass large intermediate objects between them without serializing them through your own server.

Substrate promised that every node of a program executed in the same cluster. For users, the practical effect was fewer round trips and no egress bill for passing a transcript from the speech node into a text node. It also made latency more predictable on long chains, since each hop was a local call rather than a public internet request.

The Python SDK and the TypeScript SDK were the main interface. Both let you build nodes as objects, chain them by passing one as the input to another, and inspect the run afterward. Ben Guo spent eight years on API and SDK design at Stripe, and it shows in the ergonomics: typed inputs, short method names, and a run object that holds every intermediate value for replay.

Model calls, custom code, and storage of run results drew from a single usage bill. For a small team that was the difference between one invoice to reconcile and four, and it removed the need to pre-buy credits from each model vendor separately. Usage was metered per node, so you could see which step in a graph was eating the budget.