Molmo
Allen Institute for AI (Ai2) · Image · Other
Molmo is an open source vision language model family from the Allen Institute for AI. This multimodal AI model reads images and video and points to what it sees.
About Molmo
What Is Molmo
Molmo is a family of open multimodal models built by the Allen Institute for AI (Ai2). It handles images, multi-image sets, and video, and it answers questions while pointing to the regions it used to decide. Most vision models return text alone. Molmo also returns coordinates.
The first release arrived in September 2024 with models from 1B to 72B parameters. Molmo 2 followed in December 2025 with 4B and 8B variants based on Qwen 3, plus a fully open 7B version built on Olmo. Weights, datasets, and training code ship under Apache 2.0, so anyone can download a model and run it on their own hardware.
The catch is what it is. Running Molmo on a laptop means handling model files and setup work that a chat app hides. Several released datasets carry academic and non-commercial terms, so you need to check licenses before commercial deployment.
Getting Started
- Pick a variant on the Molmo or Molmo 2 Hugging Face page. Sizes run from about 1B to 72B parameters, so match the model to your GPU memory.
- Download the weights, or open the Ai2 Playground if you only want to test one before committing.
- Load the model with the transformers library, or with vLLM for faster serving. The Molmo 2 repository ships scripts for both.
- Send an image, a set of images, or a video clip along with your question. Ask where, when, or how many to get coordinates and timestamps back with the text.
- Fine-tune on your own data if you need a narrower task. The training code and recipes are public.
Product Information
A quick look at Molmo's pricing, supported platforms, and performance.
Best for
The users, tasks, and scenarios where this tool fits best.
Users
- Researchers
- Developers
- Students and educators
- Robotics and automation teams
Tasks
- Image pointing and counting
- Video tracking
- Dense video captioning
- Grounded question answering
- Document and chart reading
- Fine-tuning for a niche task
Scenarios
- Checking a long match or lecture recording
- Sorting product photos for a store
- Spotting flaws in AI-generated video
- Reviewing satellite or drone footage
- Teaching a class about multimodal models
- Prototyping a robot vision pipeline
Key features
Image and Video Pointing
Molmo points to what it sees instead of only describing it. Give this image pointing model a photo and ask which object is which, and it returns 2D coordinates for each answer. In Molmo 2, that pointing extends to time. Ask when a cup fell or which player scored, and you get a timestamp alongside the location.
Video Tracking With Persistent IDs
Molmo 2 follows objects through a clip while keeping a stable ID for each one. That matters for counting, because the same cup should not get counted twice after it leaves the frame and comes back. The model handles occlusions and re-entries, which older open models struggled with. Ai2 reports it beats Gemini 3 Pro on its video tracking evals. Those are Ai2's own benchmarks, so treat the gap as a starting point, not a final score.
Multi-Image and Long-Video Reasoning
The models take a single image, a set of two to five related images, or video clips that stretch past a few minutes. As a video understanding model, Molmo 2 handles footage through a SlowFast approach: full resolution on key frames, less on the rest. You keep accuracy on long-video tasks while spending fewer vision tokens per request.
Open Weights, Data, and Training Code
Molmo ships under Apache 2.0 with weights, training code, and recipes that Ai2 published. Molmo 2 adds nine new datasets covering captioning, QA, pointing, and tracking, with more than 9 million training examples. Nothing here depends on data generated by a closed model, which keeps the pipeline reproducible for anyone who wants to build on it.
Runs Without API Fees
Because the weights are open, you can run Molmo on your own hardware and skip per-call charges. Ai2 notes that the smaller models run locally without a data center. The tradeoff is that you supply the GPU and the setup, and the largest variants still need serious memory to load.
Grounded Counting and Anomaly Detection
The model answers "how many?" by returning visual evidence, not just a figure. That makes counts auditable: you can see which objects it included. The same grounding supports anomaly detection, where the model flags rare or surprising events and points to the frame that holds them.
Fine-Tunable End to End
Molmo 2-O (7B) is built on Olmo, so the vision encoder, the connector, and the language model are all under your control. That suits teams that want to retrain every part of the stack rather than fine-tune a fixed backbone. The Molmo 2 repository includes pretraining, SFT, and long-context SFT scripts.
Pros and cons
Pros
- Open weights, datasets, and training code under Apache 2.0, so you can inspect and reuse the full pipeline.
- Pointing and grounding return coordinates and timestamps, not just text.
- Smaller models match or beat the earlier 72B Molmo on several grounding benchmarks.
- No per-call API fees when you self-host.
- Video tracking and counting rank at or near the top of Ai2's open-model evaluations.
Cons
- No official pricing or public API yet, so a hosted option is not something you can plan around today.
- Running the models yourself means managing GPU hardware and model files, which rules out quick low-end laptop use.
- Several training datasets are limited to academic and non-commercial research, so commercial deployment needs a license check.
- Video grounding stays hard: Ai2 reports that no tested model, Molmo included, reaches 40 percent counting accuracy.
Frequently asked questions
Molmo handles image and video understanding where you also need to know where things are. Typical uses include pointing at objects, counting them, tracking them across a clip, and answering questions about photos, charts, or recordings. It suits anyone who wants those abilities on their own hardware rather than through a paid API.
Related content
Explore related tools, skills, and articles for Molmo.
Molmo Alternatives
Google EmbeddingGemma 2
Google DeepMind · Writing · Image · CodingGoogle EmbeddingGemma 2 is a 740M-parameter open model that turns text, code, images, audio, and video frames into vectors in one 768-dimension space, running on phones and laptops.
Tractionwave
TractionWave · Image · MarketingTractionWave is an AI attention heatmap tool for static ads. Upload a PNG or JPG and it returns predicted attention, ranked hotspots, and fixes before you spend budget.
Imejis.io
Imejis.io · Image · MarketingImejis.io is an AI design studio for agents and marketing teams. Describe an image in plain words, and it writes the copy, builds the layout, reflows it to every size, then exports a finished file.