WEBVTT

00:00:18.340 --> 00:00:24.062
This is OneWM: one model that learns
to imagine the world, and to act in it.

00:00:24.743 --> 00:00:27.728
Robots need two kinds of understanding.

00:00:27.768 --> 00:00:32.590
A world model asks: if I take this action,
what happens next?

00:00:32.630 --> 00:00:37.153
A policy asks the reverse:
given what I see, what should I do?

00:00:37.193 --> 00:00:39.065
These are usually learned apart.

00:00:39.105 --> 00:00:42.853
So, can learning one make the other better,
in both directions?

00:00:43.945 --> 00:00:46.555
That's where OneWM comes in.

00:00:46.595 --> 00:00:50.880
It brings world prediction and action generation
into one causal framework,

00:00:50.920 --> 00:00:52.968
where actions play two roles:

00:00:53.145 --> 00:00:56.980
a condition that says what will be done,
and a target to learn.

00:00:57.960 --> 00:01:03.045
Inside, a video expert and an action expert
share joint causal attention,

00:01:03.098 --> 00:01:05.433
and both read the task instruction.

00:01:06.782 --> 00:01:08.704
That attention follows time.

00:01:08.794 --> 00:01:12.204
Future frames read the past
and their aligned actions,

00:01:12.257 --> 00:01:16.517
while actions read only what has been observed,
never the future.

00:01:17.497 --> 00:01:20.332
Training mixes three action regimes.

00:01:20.372 --> 00:01:24.869
With every action given,
the model learns their visual consequences.

00:01:24.909 --> 00:01:27.644
With only the first few, it completes the rest.

00:01:27.684 --> 00:01:31.919
With none, it generates every action
from what it sees.

00:01:33.268 --> 00:01:37.328
One objective learns future video
and unknown actions together.

00:01:38.308 --> 00:01:41.268
So, does learning the world help the robot act?

00:01:41.308 --> 00:01:43.631
Under matched comparisons, yes.

00:01:43.671 --> 00:01:46.493
Success rises on all three benchmarks,

00:01:46.533 --> 00:01:50.668
with the largest gain, over eight points,
on the most challenging one.

00:01:52.174 --> 00:01:55.309
On a real robot, across three tasks,

00:01:55.436 --> 00:02:00.596
overall success climbs from 48.9 to 66.7 percent.

00:02:03.614 --> 00:02:05.412
Now, the other direction.

00:02:05.452 --> 00:02:10.024
Given the same actions, does learning to act
help the model predict the future?

00:02:11.829 --> 00:02:17.877
It does. OneWM improves all three
image-quality metrics on every benchmark,

00:02:17.979 --> 00:02:21.027
so predicted futures stay closer to reality.

00:02:22.419 --> 00:02:25.504
Put the two together, and OneWM can dream.

00:02:25.569 --> 00:02:28.704
From a single observation, it predicts actions,

00:02:28.744 --> 00:02:33.567
imagines their effects,
and feeds that imagined view back in to act again.

00:02:33.607 --> 00:02:36.692
Sixteen rounds, over five hundred actions,

00:02:36.894 --> 00:02:39.579
all inside its own imagination.

00:02:42.597 --> 00:02:47.020
OneWM. Understanding the world
by learning to act in it.

00:02:47.060 --> 00:02:49.970
Explore the paper and the project page.
