ToonCrafter AI
Tencent AI Lab, CUHK, City University of Hong Kong · Video
ToonCrafter AI is an open-source research model that fills in the frames between two cartoon drawings, turning keyframes into short animated clips on your own GPU.
About ToonCrafter AI
What Is ToonCrafter AI
ToonCrafter AI is an open-source AI animation tool built on a generative cartoon interpolation model, and it comes from researchers at The Chinese University of Hong Kong, City University of Hong Kong, and Tencent AI Lab. It was published at SIGGRAPH Asia 2024. You hand it a starting drawing and an ending drawing, and it predicts the drawings in between. The result is a short clip at 512x320 with up to 16 frames.
The model starts from an image-to-video diffusion system trained on real footage, then retunes that motion knowledge for flat, line-heavy 2D art. A second decoder pulls fine detail back into each generated frame. That work matters because generic video models blur outlines and smear flat colors on cel-shaded characters, which makes their output unusable for real animation work.
Three limits shape how you use it. The model doesn't reliably understand what objects are, so a rigid part can bend like cloth. It struggles when something enters or leaves the frame. And it stops at 16 frames, so you get about two seconds of motion, not a finished scene. The project also has no official hosted app. The README says so directly, which means any site charging money for "ToonCrafter" access is a third party.
Getting Started
- Clone the GitHub repository, create a conda environment with Python 3.8.5, and install the packages listed in requirements.txt.
- Download the model checkpoint from Hugging Face and place it in checkpoints/tooncrafter_512_interp_v1.
- Pick two frames of the same character in the same art style, cropped close to the 512x320 ratio, and write a short prompt that describes the motion.
- Run scripts/run.sh from the terminal, or start the browser interface with python gradio_app.py and upload your images there.
- Adjust ddim_steps for speed and frame_stride for how much movement happens between frames, then collect the video from the results folder.
Product Information
A quick look at ToonCrafter AI's pricing, supported platforms, and performance.
Best for
The users, tasks, and scenarios where this tool fits best.
Users
- 2D animators
- Indie game developers
- Students and research teams
Tasks
- Keyframe inbetweening
- Sketch to animation
- Colorizing line art
Scenarios
- Pitching a shot
- Prototyping a game animation
- Testing an art style
Key features
Two-Keyframe Cartoon Interpolation
This is the core function, and it's where AI frame interpolation earns its keep: two images in, a sequence out. The model handles occlusion and the large, non-linear motion that cartoon acting relies on, which is where ordinary frame blending falls apart. Everything happens in a single inference pass, so there's no training and no per-shot fine-tuning.
Sparse Sketch Guidance
You can add one or more rough sketches for specific moments in the clip and the model follows them. That gives frame-level control over timing and direction, not just a text hint. For a walk cycle or a hand gesture, a couple of scribbled lines often communicate intent better than a paragraph of prompt text.
Reference-Based Sketch Colorization
Feed a colored reference image alongside line art animation and the tool applies that palette to the moving frames. It accepts a single reference or two references for more control over shading. Studios use this to test color scripts quickly, before a colorist touches the shot.
Cartoon-Tuned Diffusion Base
The underlying image-to-video model learned motion from real footage. The team adapted it with a cartoon rectification stage trained on hundreds of thousands of curated animation clips, so the motion prior fits drawn characters. A dual-reference 3D decoder then restores detail that the compressed latent space throws away.
Local Browser Interface
A Gradio app ships with the repository. You upload the two frames, type a prompt, and press generate without touching a config file. The command line route exposes the same settings plus the sampler controls, so you can keep the UI for quick tests and the script for batch runs.
Open Code and Weights
Both the inference code and the checkpoint are public. You can read how the sampler works, swap components, or wire the model into a bigger tool. That openness is the main reason the project spread through the animation and ComfyUI communities within weeks of release.
Pros and cons
Pros
- No subscription and no credit system. Once the checkpoint is on your disk, generation costs only electricity.
- It handles occlusion and wide motion better than generic interpolation models, and the project page compares results against AnimeInterp, FILM, and SEINE.
- Sketch guidance and reference colorization cover two jobs that prompt-only video tools can't target directly.
- The install is small for a diffusion project: one Python environment, one checkpoint, one YAML config.
- Output is deterministic enough to iterate, since the seed controls variation and you can rerun a shot until the motion reads right.
Cons
- The 16 frame cap at 512x320 means roughly two seconds of low resolution motion. You can't finish a shot here, only preview it.
- The reference implementation asks for about 24GB of GPU memory, and even community forks that fit in 10GB still want a recent NVIDIA card. Laptop owners are out unless they rent a cloud GPU.
- The model has no real understanding of objects, so rigid parts can wobble and anything entering or leaving the frame can break. Expect to reroll several seeds per usable clip.
- Setup is developer work. There's no official web app, and the README warns that paid sites using the name aren't connected to the team.
Frequently asked questions
It generates the missing frames between two cartoon images, so you get a short animation from a pair of keyframes. Think of it as a narrow cartoon video generator rather than a general purpose video model. The team also demonstrates sketch-guided motion and reference-based colorization on the project page.
Related content
Explore related tools, skills, and articles for ToonCrafter AI.
ToonCrafter AI Alternatives
Griffin by Tavus
Tavus · Voice & Language · Video · ChatbotGriffin by Tavus is the first Human Interaction Model: a full-duplex, video-to-video system that watches, listens, and answers during a live video call in real time.
Syllaby AI Avatar 2.0
Syllaby · Video · MarketingSyllaby AI Avatar 2.0 turns a script into a presenter-led video with a reusable AI avatar, natural voice, B-roll and automatic subtitles, all in one browser workflow.
Vitra.ai
Vitra.ai · Voice & Language · Video · MarketingVitra.ai is an agentic content platform that creates, translates, reviews, and publishes video, images, documents, and app content in 120 languages from one shared brief.