Will Wilson is the founder and CEO of Antithesis, which is trying to change how people test software. The idea is that you run your application inside a special hypervisor environment that intelligently (and deterministically) explores the program’s state space, allowing you to pinpoint and replay the events leading to crashes, bugs, and violations of invariants. In this episode, he and Ron take a broad view of testing, considering not just “the unreasonable effectiveness of example-based tests” but also property-based testing, fuzzing, chaos testing, type systems, and formal methods. How do you blend these techniques to find the subtle, show-stopper bugs that will otherwise wake you up at 3am? As Will has discovered, making testing less painful is actually a tour of some of computer science’s most vexing and interesting problems.

You can find the transcript for this episode on our website.

Some links to topics that came up in the discussion:

Antithesis, Will’s company
FoundationDB’s deterministic simulation framework
QuickCheck — the original Haskell property-based testing library, by Koen Claessen and John Hughes
Hypothesis — property-based testing for Python, created by David MacIver
QuviQ — John Hughes’ company commercializing QuickCheck, including automotive testing work
Netflix Chaos Monkey
Goodhart’s law — “When a measure becomes a target, it ceases to be a good measure”
CAP theorem — the impossibility result for distributed systems that FoundationDB claims to have in some sense violated.
Paxos — the consensus algorithm FoundationDB reimplemented from scratch
Large cardinals, an area Will studied before abandoning mathematics
Lyapunov exponent — measure of chaotic divergence
Chesterton’s fence
The Story of the Flash Fill Feature in Excel
Building a C compiler with a team of parallel Claudes
Barak Richman, “How Community Institutions Create Economic Advantage: Jewish Diamond Merchants in New York”

Why Testing is Hard and How to Fix it with Will Wilson

Signals and Threads

Why Testing is Hard and How to Fix it with Will Wilson