Dream 7B
HKU NLP Group and Huawei Noah's Ark Lab · Business · Other
Dream 7B is a free, open diffusion language model built by HKU and Huawei. It writes whole passages in parallel instead of one word at a time, and it plans complex tasks better than same-size models.
About Dream 7B
What Is Dream 7B
Dream 7B, sometimes written as Dream LLM, is a 7-billion-parameter diffusion language model (a dLLM, short for diffusion large language model) from the University of Hong Kong and Huawei's Noah's Ark Lab. The checkpoint is an open source language model you can download and run free. Most chatbots, including GPT-4 and Claude, write text left to right, one token at a time. Dream 7B works differently. It starts with a fully noisy sequence and refines the whole passage in parallel over a number of steps, which lets every position see the full context at once.
That design pays off in two places. On general writing, math, and coding, Dream 7B matches or beats strong autoregressive models of the same size, such as Qwen2.5 7B and LLaMA3 8B. On planning puzzles like Countdown and Sudoku, it pulls ahead of those models and sometimes even tops DeepSeek V3, a model with far more parameters. It also supports infilling, so you can ask it to fill a gap in a paragraph or hold a specific ending sentence.
The catch is that this is a research release, not a polished app. There is no hosted chatbot, no commercial license for the weights, and no API from the team. You download the weights from Hugging Face, run it yourself, and you need a GPU with at least 20GB of memory to start.
Getting Started
- Install the right libraries: transformers 4.46.2 and torch 2.5.1, since Dream relies on the SdpaAttention built into torch.
- Download a checkpoint from Hugging Face, either Dream-v0-Base-7B for fine-tuning or Dream-v0-Instruct-7B for chat.
- Load the model with the AutoModel and AutoTokenizer classes, with trust_remote_code enabled, then move it to a CUDA device.
- Call model.diffusion_generate with your prompt, and set steps to trade speed against quality.
- For fine-tuning, clone the repo and run the SFT script on a single node with 8 GPUs.
Product Information
A quick look at Dream 7B's pricing, supported platforms, and performance.
Best for
The users, tasks, and scenarios where this tool fits best.
Users
- AI researchers
- Developers testing non-autoregressive models
- Students learning how diffusion language models work
Tasks
- Text infilling
- Constraint-heavy planning puzzles
- Fine-tuning for a niche domain
Scenarios
- Trying a diffusion language model without paying for a commercial service
- Researching an alternative to left-to-right generation
- Building a coding or math assistant on a small budget
Key features
Parallel Text Generation
Dream 7B writes many tokens per step instead of one at a time. The sequence starts fully masked and gets refined in parallel, so the model draws on the whole context at each pass. This is the core difference from autoregressive models and the reason it plans well.
Energy and Planning Tasks
The model shines on problems with several constraints at once. On Countdown and Sudoku, it beats Qwen2.5 7B and LLaMA3 8B, and sometimes edges out DeepSeek V3 despite that model's far larger size. According to the HKU team, diffusion is better suited to problems with multiple goals and constraints.
Infilling and Arbitrary Order
You are not locked into left-to-right output. Dream 7B can fill a gap between a given start and a fixed ending sentence, and it can generate in an order of your choosing. That makes it handy for editing tasks where you keep one line and rewrite the middle.
Quality-Speed Trade-off
The number of diffusion steps controls how much work each answer gets. Fewer steps return faster but rougher text, while more steps raise quality at higher cost. This gives you a speed dial that autoregressive models do not offer, and you can move it per request.
Open Weights and Training Code
Both the base and instruct checkpoints are on Hugging Face, and the training, evaluation, and SFT code are on GitHub. You can fine-tune Dream 7B on your own dataset rather than renting it. The team also published Dream-Coder for code and DreamOn for variable-length generation.
Remasking Strategies
The diffusion_generate call lets you pick how tokens get committed. You can go fully random, or use confidence-based strategies like maskgit_plus, topk_margin, or entropy, and add temperature for variety. Language model inference quality shifts with these choices, so the confidence strategies matter when accuracy counts.
2048-Token Context and Single-GPU Runs
Dream 7B has a context length of 2048 tokens for input plus output, which fits most single-prompt tasks. It runs on one GPU with at least 20GB of memory, so it stays within reach of a single workstation rather than a cluster.
Pros and cons
Pros
- Free, open weights for both base and instruct models, with no commercial license tied to a paid plan.
- Matches or beats same-size autoregressive models on general, math, and coding benchmarks.
- Leads on planning tasks like Countdown and Sudoku, where it sometimes beats models with far more parameters.
- Native infilling and arbitrary-order generation open up editing workflows that left-to-right models do not support.
- Adjustable diffusion steps give a clear speed-versus-quality dial for each request.
Cons
- There is no hosted product, so you must run it yourself and supply a GPU with at least 20GB of memory.
- The 2048-token context is small, which rules out long documents and extended conversations.
- The team ships no API, so adding it to an app means hosting the model and writing the inference path yourself.
- The random default decoding strategy can lower accuracy, so you have to tune remasking for serious work.
- Being a research release, it lacks the polish, safety layers, and support you get from commercial models.
Frequently asked questions
Yes. Both the base and instruct checkpoints are free on Hugging Face, and the training code is open on GitHub. There is no paid plan and no hosted subscription, but you do cover your own hardware.
Related content
Explore related tools, skills, and articles for Dream 7B.
Dream 7B Alternatives
Knocket
Knocket (Tencent RTC team) · Productivity · Business · ChatbotKnocket is a free live chat and contact page tool that lets visitors message you from your website, app, or a shareable link, with every reply handled from Telegram or one inbox.
shippedit.me
Krawlup · Marketing · Businessshippedit.me now redirects to Krawlup, a web platform that audits SEO, AEO and page speed every week, then hands the fixes to AI coding agents over MCP.
Ayven
Alknoma · Productivity · BusinessAyven is an AI control centre by Alknoma that turns a plain-language goal into a dated plan, hands the work to your people and AI agents, and asks for your approval only when the call is yours.