BaseRT
Base Compute · Coding · Other
BaseRT is a from-scratch local LLM inference runtime built for Apple Silicon, tuned at the Metal kernel level to squeeze more speed out of Mac hardware than general-purpose engines like MLX or llama.cpp. It ships as a single `basert` LLM CLI that pulls models from HuggingFace, converts them to its own `.base` format, and lets you chat locally or serve an OpenAI-compatible API. Developers who want fast, private, on-device AI on a Mac get the most out of it, because there's no cloud dependency, no API keys, and no per-token cost.
About BaseRT
What Is BaseRT
BaseRT is a local inference runtime from Base Compute, a small AI lab split between Melbourne and Berlin. The project is built from scratch rather than bolted onto existing engines, which is how the team claims prefill speedups up to 6.4x over llama.cpp and 3.9x over MLX, with decode up to 1.33x over MLX on an Apple M5 Pro. Those figures come from Base Compute's own benchmarks, so treat them as vendor numbers rather than independent lab results.
The runtime is aimed at engineers and hobbyists who run open source models on their own Macs. It handles the whole loop: pull a model, convert it into a format the engine loads quickly, then either chat with it or expose it over HTTP. Everything stays on your machine. Nothing goes to a provider.\n\nWhy does that matter? No per-token cost once the model is local. Simple as that.
The main limits are practical. BaseRT is macOS-only right now, with Windows and Linux listed as coming soon. The core engine ships as a prebuilt binary rather than open source code, so you can't rebuild the kernels yourself. If you're on a non-Apple machine or need a fully open stack down to the silicon, this isn't it yet. Not yet, anyway.
Getting Started
- Install the CLI with the one-line script:
curl -LsSf https://basecompute.co/install.sh | sh. - Pull a model from HuggingFace using the launcher, for example
basert serve basecompute/gemma-4-E4B-it. - Convert or load the model into the
.baseon-disk format the runtime reads. - Chat with the model directly, or point an app at the OpenAI-compatible endpoint the server exposes.
- Optionally install the coding-agent plugin (
pi install git:github.com/basecompute/pi-basert) to run a local coding agent against your served model.
Product Information
A quick look at BaseRT's pricing, supported platforms, and performance.
Best for
The users, tasks, and scenarios where this tool fits best.
Users
- Mac developers
- Privacy-focused engineers
- AI hobbyists
Tasks
- Serving a local LLM
- Running a local coding agent
- Benchmarking on Apple Silicon
Scenarios
- Working offline on a plane or train
- Handling sensitive prompts
- Cutting recurring API bills
Key features
Native Metal Inference Engine
BaseRT is built from the ground up for Apple Silicon's Metal framework instead of being ported from CUDA or CPU code. The engine embeds its Metal kernels in a prebuilt libbaseRT.dylib, so the runtime talks to the GPU the way Apple intends. In practice that's where the speed claims come from: up to 6.4x faster prefill than llama.cpp and up to 33% faster decode than MLX on some models, per Base Compute's benchmarks.
OpenAI-Compatible Serving
The basert-serve tool turns any loaded model into an HTTP server that speaks the OpenAI API format. That means existing tools, SDKs, and scripts that already target api.openai.com can be repointed at your local endpoint with a config change. You skip rewrites and get a familiar interface while keeping the actual model on your own machine.
Multi-Language Bindings
Beyond the CLI, BaseRT exposes a C API in baseRT.h with bindings for Python, Node, Rust, and Swift. If you'd rather embed inference directly in an app than shell out to a server, these bindings let you call the engine from the language your project already uses. It's the difference between a command line tool and a library you build on top of.
HuggingFace Model Hub and Converter
The basert command bundles a model hub and a converter. You pull weights straight from HuggingFace and the tool converts them into the .base on-disk format the runtime loads. That keeps setup to a couple of commands instead of a manual export chain, and it works with common quantized builds like Q4 and Q8.
Local Coding-Agent Integration
Base Compute ships a plugin (pi-basert) that wires a coding agent directly to a BaseRT-served model. You run three commands. That's it. You end up with an agent that reads and writes code without sending anything to a remote API. For teams wary of leaking source code to hosted providers, that's the headline use case.
Zero-Cost, No-Account Operation
There's no sign-up, no API key, and no usage meter. Once a model is on your disk, each token costs nothing to generate because the only resource consumed is your own hardware. So what's the catch? There basically isn't one. For heavy users who'd otherwise pay per token, the savings add up fast.
Pros and cons
Pros
- Prefill and decode speeds that beat MLX and llama.cpp on the same Mac, according to vendor benchmarks.
- Fully local inference, so prompts and data never leave your device.
- OpenAI-compatible server means existing tools work with minimal changes.
- Bindings for Python, Node, Rust, and Swift open it up to many stacks.
- No accounts, no API keys, and no per-token cost after setup.
Cons
- macOS only for now, so Windows and Linux users have to wait.
- The core engine is a prebuilt binary rather than open source, which limits how far you can customize it.
- Benchmarks are vendor-reported, and independent results aren't available yet.
- Setup assumes comfort with a terminal, which puts off less technical users.
Frequently asked questions
BaseRT runs large language models locally on Apple Silicon Macs. You use it to chat with open models, serve them over an OpenAI-compatible API, or embed local LLM inference into an app through its C API bindings.
Related content
Explore related tools, skills, and articles for BaseRT.
BaseRT Alternatives
AI Mock Interview
SQLPad · Coding · LeaningAI Mock Interview is a practice tool built into SQLPad that simulates real job interviews and gives you instant feedback on both what you say and how you say it. You pick a role template or upload a job description, answer questions out loud in real time, and then review a transcript with notes on structure, clarity, and grammar. It's aimed at data professionals prepping for SQL, Python, data engineering, machine learning, and system design roles, and it works entirely in the browser. No install needed.
Codeflying
Kuafu Technology (Codeflying) · Coding · Marketing · ChatbotCodeflying is an AI app builder that turns a plain-language description into a working website, mobile app, or mini app. It works as a no-code app builder, so you type what you want and a set of AI agents handle requirements, architecture, front-end screens, back-end logic, and deployment. The goal is simple: build an app from a prompt, even with zero coding background. Marketing tools and a customer-facing chat agent come bundled too, so the result is more than a prototype stuck on a hard drive.
Runware
Runware, Inc. · Image · Video · CodingRunware is a generative AI inference platform that gives developers one API for image, video, audio, 3D, and language models. Instead of signing up with a dozen providers, you call a single endpoint, switch models with a one-line string change, and pay only for the requests you send. No servers to run. It's aimed at teams that want to ship AI features fast without building or babysitting their own GPU infrastructure.