Union.ai

The core idea is that a run doesn't need to restart when the hardware under it fails. Union records each task, its inputs and outputs, and a replay log. If a GPU disappears mid-training or a container dies from an out-of-memory error, the run resumes at the point of failure. On retry it can also request different resources, so the same step can move from an A100 to an H100 without editing a line of code. That's the difference between a retry and a recovery, and it's the reason long jobs finish more often.

Flyte 2 drops the workflow language. Tasks are Python functions, and tasks call tasks. Loops, conditionals and try/except blocks are just Python, and fan-out is asyncio.gather. There's no YAML file describing the graph, and no compile step before a run. Teams pick this up in an afternoon, which cuts the cost of onboarding new hires. The same program runs locally, on a devbox or across a cluster, so the debugging loop stays fast.

Every dataset, checkpoint and report becomes a named, versioned artifact, and the platform records which run produced it and which components consume it downstream. You can wire components together with artifact events instead of a scheduler. A new checkpoint version fires its eval job, a promoted model fires the scoring job, and a fresh batch of production traces fires the dataset build. Each component can be written and owned by a different team, and neither team imports the other's code. Union publishes reference implementations on GitHub that are tested in CI.

One environment declaration turns a task into a torchrun cluster, with nodes and workers wired up for you. A single run can fan out across many nodes and host thousands of actions, which is what GRPO and LoRA training runs need. Built-in plugins cover Ray, Spark and Dask for workloads that sit outside the standard task model. Where the platform shows its value is in failure handling: these clusters run on spot capacity by default, and a reclaimed node only costs the work in progress.

You can serve a model from your own GPUs with an OpenAI-compatible endpoint, declared the same way as any other task. Scaling goes from zero replicas when traffic is quiet up to the configured maximum, and weights stream from object storage straight to GPU memory, which keeps cold starts short. The same gateway can govern calls to external API providers, so a team sees one place for model spend and routing. The endpoint that serves the model is fed directly by the training run's output, which removes the manual copy to a separate registry.

Batch jobs and long-lived services share a programming model, a packaging model and one GPU scheduling layer. That's not a small convenience. It means a batch scorer and a live endpoint can point at the same checkpoint artifact, and the GPU pool can be shared between them. When a serving endpoint scales down, its capacity goes back to the training queue rather than idling. Teams that used to run two stacks get one, and cost tracking gets simpler.

Union runs in your VPC on AWS, Azure or GCP in what the company calls a BYOC setup, with an on-prem and air-gapped option on Enterprise. The control plane reaches the cluster through an outbound tunnel, and the data proxy that signs URLs and fetches logs lives inside your network. Enterprise adds SSO and SAML, custom RBAC, customer-managed encryption keys, bring-your-own-key, private endpoints and a single-tenant control plane. Compliance coverage includes SOC 2 Type II and a HIPAA BAA on request.