Ep. 014 - Finding Miscompiles For Fun, Not Profit (AI Infrastructure) | Justin Lebar & Jordan Nanos

Compiler engineer Justin Lebar details his process for identifying critical bugs in NVIDIA’s PTX compiler and LLVM backends using a combination of traditional fuzzing and LLM-assisted code analysis. By leveraging AI to generate random programs and directly inspect source code, he uncovered significant vulnerabilities, including a high-severity flaw where atomic operations were incorrectly split into non-atomic instructions. While traditional fuzzing faces diminishing returns as bug patterns become harder to exclude, LLM-assisted analysis offers a powerful, albeit token-intensive, alternative for identifying complex compiler errors. This approach demonstrates that AI can significantly accelerate the triage and remediation of software defects. Ultimately, the project underscores the potential for developers to drastically improve the quality of foundational software infrastructure by integrating LLMs into their testing workflows, provided they commit the necessary resources and human oversight to verify results.