Most discussion of world models treats “the world state” as a given object you read off and predict forward. I keep getting stuck one step earlier: what is a world state, concretely?
The open questions I can’t yet answer cleanly:
- There’s no uniform representation of a 3D world, and no uniform representation of its state. So which one do you commit to, and what does that choice quietly assume?
- Does the state have to be maintained in real time, or is it something you reconstruct on demand?
- Where does the data defining it even come from?
The reason this isn’t just bookkeeping: the representation is the problem, not a detail you settle before the real work. That’s the same lesson as Every conditioning choice is a hidden assumption, the encoding has to match the thing it’s trying to steer. A “world state” you can’t ground in a representation is just a word.
Update, Oct 2026. I’ve committed to an answer to the first question, at least for now. The state holds the 3D layout and appearance of the scene, its physical properties, and the positions and goals of the agents in it, and a generative model renders it into video. Goals are the part I would have left out earlier. The Reachy pilot and CosmosAlign both broke on what the agents were trying to do, not on geometry.
The other two questions are still open, and so is a new one: how much of the state should be learned, and how much written down? Back to World models need an explicit world state, and to A director, not a script for what acts on the state once it exists.