FuriosaAI
FuriosaAI · Coding
FuriosaAI builds the RNGD NPU and the NXT RNGD server, energy-efficient AI inference hardware that runs LLMs and multimodal models through OpenAI-compatible APIs.
About FuriosaAI
What Is FuriosaAI
FuriosaAI is a South Korean chip company that makes AI accelerators for inference, the part of AI work that serves a model to real users. Its current product is RNGD, pronounced "renegade," a second-generation Neural Processing Unit built on the company's Tensor Contraction Processor architecture. Think of RNGD as an NPU for LLM inference: add AI serving capacity to a data center without adding much power.
Where a typical AI GPU draws hundreds of watts per card, RNGD runs at 180W and still claims 512 TFLOPS in FP8 with 48GB of HBM3 memory. FuriosaAI pairs the chip with a software stack that speaks the same APIs as vLLM and OpenAI, so an application that already points at a GPU endpoint can often move over without rewrites. The company sells the card, a turnkey 8-card server called NXT RNGD, and evaluation access for teams that want to test before buying.
The limits matter before you plan around it. This is hardware for data centers, not a cloud subscription you sign up for on a website, and you'll talk to FuriosaAI's team to evaluate or buy. RNGD targets inference only, so it won't train models. Some figures differ between the product page and the vendor's own documentation, and pricing isn't published. The company has real production deployments, but its library of ready-made models and tools is much smaller than the CUDA world around Nvidia.
Getting Started
- Request an evaluation through the FuriosaAI website and pick bare-metal access on an NXT RNGD server or an OpenAI-compatible API endpoint.
- Install the software stack with pip, starting from
pip install furiosa-llmon a machine with RNGD hardware. - Load your model and launch the server with
furiosa-llm serve [model], which exposes an OpenAI-compatible endpoint on port 8000. - Point your existing application at that endpoint, keeping the same client code you use for GPU-backed APIs.
- Benchmark throughput, latency, and power against your own service targets, then decide whether to move it into production.
Product Information
A quick look at FuriosaAI's pricing, supported platforms, and performance.
Best for
The users, tasks, and scenarios where this tool fits best.
Users
- Cloud providers and data center operators
- Enterprises with strict data rules
- Foundation model teams
- Software vendors and neoclouds
Tasks
- Serving large language models in production
- Running multimodal and vision models
- Cutting inference power and cost
- Validating a GPU alternative before committing
- Deploying multi-tenant AI
Scenarios
- Adding AI capacity without more power
- Keeping proprietary models in-house
- High-volume consumer AI features
- Sovereign and regional AI
Key features
Tensor Contraction Processor Architecture
RNGD uses the Tensor Contraction Processor, an architecture that runs tensor contractions, the math behind deep learning, directly in hardware. A tensor contraction is a general form of matrix math, so the chip handles many model shapes instead of only fixed-size operations. FuriosaAI says this design keeps compute utilization high as batch sizes and model structures change.
High Throughput at 180W
One RNGD card, an AI accelerator card for the PCIe Gen5 slot, delivers 512 TFLOPS in FP8, 512 TOPS in INT8, and 1024 TOPS in INT4, with 48GB of HBM3 memory and 1.5 TB/s of bandwidth. The rated power draw is 180W, roughly half that of comparable GPU accelerators. Lower power per card means more compute inside a fixed data center power budget.
OpenAI-Compatible and vLLM-Compatible Software
Furiosa-LLM is the inference engine that sits on the hardware, and it mirrors the vLLM API along with an OpenAI-compatible server. You run furiosa-llm serve and point existing apps at the endpoint. For teams, this means an application written against a GPU endpoint can often migrate without client-side changes.
NXT RNGD Server
The NXT RNGD is a turnkey 4U rackmount server with 8 RNGD chips, 384GB of accelerator memory, 1TB of DDR5 system memory, dual AMD EPYC 9354 processors, and 4096 TFLOPS of FP8 performance. It ships with Ubuntu 24.04 LTS and the full Furiosa software stack preinstalled. It's built for teams that want inference capacity without sourcing and integrating parts themselves.
Multi-Instance and Virtualization Support
SR-IOV splits one physical chip into 2, 4, or 8 independent instances, each with dedicated compute cores and private memory bandwidth. This suits multi-tenant Kubernetes clusters, where several workloads share hardware but need isolation and steady latency. ECC memory and secure boot add reliability and a hardware root of trust.
Compiler with an Accurate Cost Model
The Furiosa compiler plans memory allocation and instruction scheduling ahead of time, rather than relying on hand-tuned kernels per model. Because the hardware is predictable, the compiler searches a bounded space of shapes and tactics and builds one execution plan for the whole graph. New models can run well without someone writing kernels for each one.
Full Software Stack and Reference Apps
The toolchain covers compilation, runtime, and serving: the Furiosa Compiler, Furiosa Runtime, and Furiosa-LLM. Reference applications for agents and retrieval-augmented generation are published on GitHub, so developers have starting points instead of a blank file. The stack integrates with existing Kubernetes clusters and serving pipelines.
Pros and cons
Pros
- Runs LLM and multimodal inference at 180W per card, around half the power of comparable GPU accelerators.
- Speaks OpenAI-compatible and vLLM-compatible APIs, so existing applications can often migrate without code changes.
- SR-IOV splits each chip into up to 8 isolated instances, which fits shared and multi-tenant setups.
- Evaluation offers real hardware and an API endpoint, so you can test against your own service targets before buying.
- Production references include large Korean portals, a telco-adjacent cloud, and a sovereign AI deployment.
Cons
- It's data center hardware sold through a sales conversation, not a self-serve cloud product, so getting started takes planning and a quote.
- RNGD handles inference only, which means you still need separate hardware or services to train or fine-tune models.
- The software library is far smaller than Nvidia's CUDA stack, so unusual models or custom kernels may need FuriosaAI's help.
- Published specs differ in places, such as a 180W rating on the product page and 150W in older documentation, so confirm exact figures during evaluation.
Frequently asked questions
FuriosaAI makes AI inference accelerators: the RNGD NPU, the Furiosa RNGD PCIe card, and the NXT RNGD server. These run models that are already trained, serving them to users rather than training new ones.
Related content
Explore related tools, skills, and articles for FuriosaAI.
FuriosaAI Alternatives
Base Code
Base44 (Wix) · Coding · ProductivityBase Code is a shared cloud development environment from Base44. Teammates describe changes to an existing GitHub codebase in chat, preview them live and send them back as pull requests.
Google EmbeddingGemma 2
Google DeepMind · Writing · Image · CodingGoogle EmbeddingGemma 2 is a 740M-parameter open model that turns text, code, images, audio, and video frames into vectors in one 768-dimension space, running on phones and laptops.
GitGlow
GitGlow · Coding · ProductivityGitGlow is a local-first desktop Git client that lets independent developers review their own code and their AI agents' changes in one place before shipping.