Suppose you’re not happy with the quality of your sleep. You’ve already stopped doing the obviously harmful things (no more coffee at night), and your sleep has improved - but you’d like to work on it further. A coworker gives you an herbal mix with St. John's wort and lavender. You try drinking it at night instead of coffee, and it does seem that sometimes your sleep really does get deeper than before. But sometimes it doesn’t. You’re willing to experiment, but how do you actually check whether the herbs work, or whether it's just random variation?

Or suppose you’re not particularly satisfied with your productivity at work. Following the advice from "Atomic Habits" and books on workflow organization, you’ve introduced a few useful micro‑habits and ergonomic improvements. But what do you do once the low‑hanging fruits are picked up? Time is limited - you can’t implement everything that someone somewhere calls "useful". Some habits are even mutually exclusive: it's impossible to both socialize during lunch and sit alone in silence at the same time.

Or, for example, you want to achieve better results in fishing… you get the idea.

"Don’t underestimate the power of small things taken in [...]

---

Outline:

(02:20) Notation

(03:11) Minimal background for people who want to read the article but dont know probability theory

(10:30) How large of effects should you expect from self‑experimentation?

(13:41) What does d = 0.1-0.4 actually mean in real life?

(15:56) How many observations, exactly?

(20:23) p‑value

(25:02) Additional complications

(25:15) Non‑linear interactions

“Most likely you won’t be able to perform a data-driven self-improvemnet” by siarshai

LessWrong (30+ Karma)