This podcast episode delves deep into the intricacies of how large language models (LLMs) store knowledge, using the example of Michael Jordan to explain complex concepts like multilayer perceptrons and the transformer architecture. By breaking down the operations within MLPs, the speaker emphasizes the challenges of interpreting their functions despite simple computations, introduces the notion of superposition for representing multiple features, and hints at future explorations into training methodologies, providing a comprehensive understanding of the mechanics behind LLM functionality.
