Phi-4-multimodal and Phi-4-mini
Microsoft · Coding
Phi-4-multimodal and Phi-4-mini are Microsoft's small open models: one reads text, audio, and images at once, the other runs fast text reasoning on 3.8B parameters.
About Phi-4-multimodal and Phi-4-mini
What Is Phi-4-multimodal and Phi-4-mini
Phi-4-multimodal and Phi-4-mini are two small language models from Microsoft Research. They joined the Phi family in February 2025 as lighter companions to the 14B Phi-4. Mini is the text workhorse at 3.8B parameters. Multimodal is the same backbone with vision and speech wired in, at 5.6B parameters. Both are open source SLMs, meaning Microsoft publishes the weights for anyone to download.
Microsoft built them for devices, not data centers. Both fit memory tight phones, laptops, and cars, and both run with low latency. That's the point of the family. You get a capable multimodal AI model without renting a cloud GPU.
The limits are honest. These are small models, so they store less factual knowledge than big ones. Mini can't read images at all. Multimodal handles text, images, and audio, but it can't combine all three in one prompt. Both ship under the MIT license, so you can fine-tune and ship commercially.
Getting Started
- Grab the weights from Hugging Face or Azure AI Foundry, or pick the ONNX build if you're targeting edge hardware. The Phi-4 API is also available if you'd rather call a hosted endpoint.
- Load the model with the transformers library, or call it through Azure AI Foundry, GitHub, or the NVIDIA API catalog.
- Feed it a chat prompt. Mini takes text. Multimodal also accepts image and audio inputs for tasks like OCR or transcription.
- Fine-tune on your own data with the Phi Cookbook recipes, then deploy on-device or serve it from the cloud.
Product Information
A quick look at Phi-4-multimodal and Phi-4-mini's pricing, supported platforms, and performance.
Best for
The users, tasks, and scenarios where this tool fits best.
Users
- App developers
- Researchers
- Speech and vision builders
Tasks
- Transcribing and translating speech
- Reading documents and charts
- Math and logic prompts
- Function and tool calling
Scenarios
- Building an offline assistant
- Prototyping on a budget
- Edge and IoT projects
Key features
Unified text, vision, and audio in one model
Phi-4-multimodal processes text, images, and audio through a single network instead of chaining a transcription model to a vision model. That means one component to load and one pipeline to maintain. The gain shows in tasks where the inputs interact, like reading a chart while listening to a spoken question.
Mixture-of-LoRAs architecture
Microsoft keeps the base language model frozen and bolts on modality-specific LoRA adapters with a router that picks the right one. This is what lets the model add speech and vision without wrecking its text skills. For builders, it means the base model stays intact and each modality trains separately.
128K token context window
Both models accept up to 128K tokens, enough for long documents or extended conversations. Grouped-query attention keeps the memory cost of that long context down, which matters when the model runs on a device with limited RAM.
Strong speech recognition and translation
Multimodal reached the top of the Hugging Face OpenASR leaderboard with a 6.14% word error rate, according to Microsoft. It was the first open model able to do speech summarization, and Microsoft reports its performance there is close to GPT-4o.
Built-in function calling
Both models support function and tool calling out of the box, so they can trigger external APIs and act as agents in a workflow. Mini's instruction following was tuned specifically for this, which makes it a practical choice for automation.
200,000 token vocabulary and multilingual support
The vocabulary jumped to 200K tokens, up from Phi-3.5, which sharpens multilingual handling. Text spans more than 20 languages, while audio covers eight and vision runs in English.
Pros and cons
Pros
- Downloads and runs for free under the MIT license, with no API fees if you self-host.
- Small enough for phones, laptops, and edge hardware, so features work offline and stay on the device.
- Multimodal handles speech recognition, translation, OCR, and image reasoning in a single model.
- Mini matches much larger models on math and coding benchmarks, according to Microsoft.
- The open weights and technical reports let researchers inspect and fine-tune everything.
Cons
- Small size means limited factual knowledge, so answers on niche topics can be wrong without a search or RAG layer.
- Multimodal can't take text, audio, and images in the same prompt, so some combined-input workflows need workarounds.
- Vision support is English only, which narrows it for non-English image tasks.
- Cloud pricing depends on the Azure AI Foundry plan, so costs vary by region and usage tier.
Frequently asked questions
No, not always. The models are small enough to run on CPUs and edge hardware, and Microsoft ships ONNX builds for that purpose. A GPU speeds things up, especially for the multimodal model and long contexts, but it isn't required.
Related content
Explore related tools, skills, and articles for Phi-4-multimodal and Phi-4-mini.
Phi-4-multimodal and Phi-4-mini Alternatives
Udon
Udon · Coding · ProductivityUdon turns an Apple Silicon Mac into a home server you run from a browser, with container controls, file shares, and an optional AI sysadmin.
offstage
offstage cloud · Coding · Productivityoffstage gives AI coding agents and CI jobs a real Mac: a fresh macOS account on Apple silicon, driven over an API and billed by the second.
BotBus
Chengdu Tuyang Tusen Technology Co., Ltd. · Coding · ProductivityBotBus lets you control AI coding agents from your phone and watch: approve commands, start tasks and read results while the code stays on your desktop.