You can gain access to a world of education through Stanford Online, the Stanford School of Engineering’s portal for academic and professional education offered by schools and units throughout Stanford University. https://online.stanford.edu/ Our robust catalog of degree programs, credit-bearing education, professional certificate programs, and free and open content is developed by Stanford faculty, enabling you to expand your knowledge, advance your career, and enhance your life. Stanford Online is operated and managed by the Stanford Engineering Center for Global & Online Education (CGOE). CGOE expands access to Stanford teaching and research, working in collaboration with faculty in the School of Engineering and throughout Stanford University to design and deliver extensive global, online, and enterprise education to a global audience.
In this podcast, Div Garg discusses human-inspired approaches to AI agents and the path to AGI, emphasizing the need to rethink the design, evaluation, and deployment of intelligence. He explores the abstract concept of AGI, its potential forms, and its applications in everyday life. Garg breaks down the architecture o...
The podcast features a speaker presenting research on human-machine interactions, focusing on predicting and shaping outcomes when humans control machines. The speaker discusses models people learn when controlling machines, using paradigms from fighter pilot studies to modern interfaces like joysticks and myoelectric ...
Magdalena Wojcieszak discusses the dangers of the current communication environment, arguing that the focus on radicalization and echo chambers online is too narrow and overshadows the more fundamental problem of interest bias in recommender systems. She presents a study using sock puppets on YouTube, which found that ...
The lecture focuses on multi-machine optimization, specifically parallelism across machines, to train large models that exceed single GPU capacity. It covers networking basics, mapping hardware to parallelization strategies, and case studies. The discussion includes compute and memory concerns, heterogeneous communicat...
In this lecture, the speaker details how to write high-performance code for GPUs, particularly for Assignment 2, which involves profiling and writing a Triton kernel for FlashAttention2. The lecture includes a review of GPU components, benchmarking, and profiling techniques using CUDA kernels in C++, Triton, and PyTorc...
The podcast discusses GPUs and their importance in language models, aiming to demystify CUDA and GPU performance. It covers why GPUs get slow, how to create fast algorithms like Flash Attention, and the key components needed for acceleration. The lecture emphasizes the differences between CPUs and GPUs, GPU anatomy (SM...
In the first part of their series on generative AI for healthcare, Dong-han Yao and Shivam Vedak, both clinical informaticists and physicians at Stanford, aim to provide healthcare professionals with a foundational understanding of large language models (LLMs) and their applications in medicine. They discuss the challe...
Karina Nguyen from OpenAI delivers a lecture on the co-design of product and research in AI, highlighting AI's potential in education, personalized tool creation, and augmented creativity. She discusses two scaling paradigms: next token prediction and chain of thought, emphasizing the importance of building and creatin...
In this episode of the Stanford Healthcare AI podcast, Dr. Amy Abernethy joins the hosts to discuss the rapid advancements in AI and their implications for healthcare. The conversation revolves around the underutilization of AI capabilities by clinicians, the challenges of keeping up with the technology, and the need f...
The podcast discusses Mixture of Experts (MOEs) architectures in modern high-performance systems, highlighting their advantage over dense architectures in terms of parameter efficiency and performance. It covers the basic components of MOEs, focusing on sparsely activated experts within MLPs, and explains how MOEs achi...
This is an introductory lecture for the course "CS336, Language Models from Scratch," co-taught by Percy Liang and Tatsu. The lecture introduces the core staff, including TAs, and outlines the course's goals, which include providing a foundational understanding of language models by building them from scratch. It addre...
The podcast features a presentation by a speaker detailing their work on algorithmic audits, usable control, and autonomy in sociotechnical systems. The speaker begins by discussing the history of algorithm regulation, using the Sabre system as an example, and then transitions to their work on housing audits using sock...
This is the fifth iteration of the CS25 Transformers class, focusing on Transformers and AI. The class invites leading researchers to speak on state-of-the-art topics. Instructors Steven, Curran, Chelsea, and Jenny introduce themselves and their research interests. The course logistics are explained, including the new ...
The lecture focuses on the architecture and training of Large Language Models (LLMs), diving into the details often overlooked in other courses. It begins with a recap of transformers, contrasting standard and modern variants, and then shifts to a data-driven analysis of LLM architectures, examining changes and commona...
Pax Haymeier and Petra Parikova from Stanford Engineering Center for Global and Online Education (SIGO) provide an overview of AI programs, covering both credit-bearing graduate certificates and professional programs. They detail course content, prerequisites, application processes, and the flexibility of each program,...
In this lecture, the speaker discusses building language models from scratch using Pytorch, focusing on efficiency in resource utilization (memory and compute). The lecture covers Pytorch primitives such as tensors, models, optimizers, and training loops. The speaker explains memory accounting, including different floa...
This is an introductory lecture for the CS336 course, "Language Models from Scratch," co-taught by Percy Liang and Tatsunori. The lecture outlines the course's motivations, which stem from a perceived disconnect between researchers and the underlying technology of language models due to increasing abstraction. The cour...
Follow this podcast in Podwise
Sign in to get AI summaries, transcripts and mind maps for any episode, including new ones.
