Latent Space as a New Medium
Just to be clear, no human action is doing the mapping. The system itself, the LLM, is mapping each bit of the world, all things, all attributes, all art, all words, all ideas. And astoundingly it creates this map, this latent space, not piecemeal, but all at once simultaneously. (To do so requires an immense, energy-hungry, massive cluster of chips, all connected together with miles of wires — the famous data center now in short supply.)
While training, the LLM is fed millions of books, billions of web pages, and billions of pages of text from social media. It reads every word on each of them, and once this entire library of material is loaded into its mind, it massively calculates all the interconnecting vectors, all the relative directions pointing to each other. The scale of this vast synchronized parallel calculation is staggering. It then throws away the books, the text, the images, and only keeps this tangled web of directions and vectors. These billions of directions are called its parameters. As we build larger and larger models, mapping more and more material, the parameters increase. The latest models on the frontier of AI contain trillions of parameters, meaning there are trillions of directions, or trillions of attributes that it uses to map every idea or thing it has seen.
