Ep. 002 - InferenceX 2.0 Release (Technical Staff) | Cam Quilici, Bryan Shan, Doug O'Laughlin, Jordan Nanos

InferenceX provides a transparent, iterative benchmarking platform for AI inference, tracking the performance of frontier hardware like NVIDIA’s GB200/B300 and AMD’s MI355. By implementing advanced techniques such as disaggregated prefill, expert parallelism, and multi-token prediction (MTP), the project demonstrates that modern hardware can achieve up to 80 times higher throughput per GPU compared to older architectures at high interactivity levels. The platform emphasizes the critical role of software optimization, showing that kernel-level tuning can triple performance in just one month. Beyond raw throughput, the team is shifting toward realistic, agentic benchmarks that incorporate multi-turn conversations and multi-modal data. This ongoing analysis highlights that while hardware remains central, the rapid evolution of inference runtimes and software recipes is the primary driver of cost reduction and efficiency gains in production environments.