AI models form "mental models" of the world. An AI trained to play a board game called Othello was opened up and found to be forming a mental model of the board. The AI system here was actually language model. It was given valid games as training data, and its job was to output a word that would represent a move in the game -- except the AI system was never told there was a game. It was simply given training data that to it looks like sequences of words, and output words. Without knowing what it was dealing with was a board game, you would expect it would develop a statistical model of likely good moves. But surprisingly, when the researchers opened up the box and looked at what was actually happening inside the language model, they found it was creating a representation of the board, keeping track of whose pieces are in which positions.
In another study, diffusion models, the kind of models that create images, such as DALL-E 2, Midjourney, and Stable Diffusion, are opened up, and inside it is found that they form 3D depth models of images. In other words, even though diffusion models are trained on 2-dimensional images only, they form "mental models" that are 3-dimensional. Early on in the process of generating an image, they concieve of how the objects in the scene are related in 3 dimensions.
I previously told you all about an interview where Ilya Sutskever, leader of the research team that created GPT-4, who said he believes large language models have real understanding. He said people say these models just learn statistical regularities and have no model of the world, but he disagrees, and says the best way to predict what words will come next is to genuinely understand what is being talked about. So asking models to predict what word comes next is a far bigger deal than meets the eye. To predict, you need to understand the true underlying process that produced the data. Even though language models only see the world through the shadow of text as expressed by human beings on the internet, they are able to develop a shocking degree of understanding.
That was back in April. Now we are starting to see evidence, from looking inside, that language models do indeed form "mental models" of the world they are predicting. Not just language models but diffusion models as well. This may be a general feature of generative models and maybe we will find it more and more.
Beyond surface statistics - AI secretly builds visual models of the world - Wes Roth
