Google Genie 2

Google Genie 2 starts from a single image and builds a 3D environment around it. So how do you get one? The image can come from a text-to-image model or from a real-world photo. You describe a world in text, pick a rendering you like, and then step into it. That low input bar is the point, since building worlds by hand is the bottleneck that the model is trying to remove for anyone who needs a steady supply of fresh, varied environments rather than a handful of carefully crafted ones.

The model responds to keyboard and mouse input and works out which object the player is actually controlling. Press an arrow key in a forest scene, and the robot moves, not the trees or the clouds. That sounds trivial. It isn't, because the model was never told which pixels are the character, and it has to infer that from the image alone before any action means anything at all. That inference is what makes the worlds playable rather than just watchable.

Genie 2 remembers parts of a world that have left the screen and redraws them correctly when you come back. Walk behind a house. Turn around. The house is still there, with the same details, and that's not something a plain video generator can promise. This consistency is what separates a world model from a video generator, and it's why generated worlds can hold up as agent testbeds rather than crumbling the moment your view changes.

From the same starting frame, you can take different actions and get different outcomes. That gives researchers counterfactual experiences to train against, which is hard to get from recorded video. Video only records the path that was actually taken. It has no idea what would have happened if you'd turned left instead, and that blind spot is exactly where counterfactuals help. If you want to know how an agent handles a branch it didn't take, this is one way to simulate it.

Genie 2 models water, smoke, gravity, lighting, reflections, and bloom without being explicitly taught each one. Balloons burst, doors open, barrels explode. The behaviors show up at scale rather than being hand-coded. They're not perfect, but they're enough to make the worlds feel like places with rules, and those rules shape how agents plan around obstacles they can see and predict, which is a surprising amount of structure to get from a model that was only ever trained to guess the next frame.

The model can simulate NPCs and interactions with them, plus character animation for a range of avatars. For agent research, that matters because many tasks involve dealing with something else that moves and reacts. Genie 2 gives those scenarios a home without a game studio.

Worlds can render in first-person, isometric, or third-person driving views, and different avatars animate based on the prompt. A paper plane, a dragon, a hawk, and a parachute all get their own movement style. That range comes from the out-of-distribution generalization that lets a piece of concept art turn into a working, playable scene, and it's the difference between a model that copies its training data and one that can improvise when the prompt takes it somewhere new and unfamiliar.