Quantifying Uncertainty
Making good decisions in the face of uncertainty is the focus of these two classes. Statistics and probabilistic thinking can help us identify faulty reasoning, bias, and uncertainty in everyday decisions. But statistics can also mislead: we will examine conditional probability, causation, and Simpson’s paradox.
We often treat computation as deterministic, with fixed answers. Probabilistic reasoning lets us represent incomplete knowledge, weigh prior information, and update our beliefs as evidence arrives. These ideas connect our work on predictive models to human judgments about risk and evidence.
Our goal is to assess the present and make better predictions about the future. We will consider how cognitive biases and research practices affect the evidence we encounter, and what makes a forecast useful. Work through the interactive examples; use the equations to support your understanding rather than getting stuck on notation.
Due Thursday, November 5: Probability, Bayes, and Causality
1. Seeing Theory: chapters 1–2
Try the interactive examples and change their settings. Pay attention to expectation, variance, and how conditioning changes the outcomes you count.
2. Watch
- A Visual Guide to Bayesian Theory — Julia Galef (11:24).
- Bayes’ Theorem — 3Blue1Brown (15:45).
- Your Brain Is Not a Bayes Net — Julia Galef (8:57).
- The Danger of Mixing Up Causality and Correlation — Ionica Smeets (5:56).
- Correlation CAN Imply Causation — MinutePhysics: first 3 minutes only, plus the footnote (1:25).
How does evidence change a prior belief? Why is the probability of evidence given a hypothesis different from the probability of the hypothesis given that evidence? When does a correlation support a causal explanation?
3. DataCamp: Intermediate Python, chapter 1
Complete chapter 1: Matplotlib. This gives you practice building and interpreting the plots we use to examine data and models.
Mini-Project 2 is due Friday, November 6, at 11:59 p.m. Eastern.
Due Tuesday, November 10: Evidence, Bias, and Statistical Pitfalls
1. Seeing Theory: chapter 5
Work through Chapter 5: Bayesian Inference. Explore how the base rate and test accuracy affect a positive result, and how new observations turn a prior into a posterior.
2. Adventures in Cognitive Biases
Complete Adventures in Cognitive Biases. Update your beliefs using the control in the upper-left corner to finish the activity.
3. Watch
- This Is How Easy It Is to Lie with Statistics — Zach Star (about 19 minutes).
- This Is How Easy It Is to Manipulate Public Perception — Zach Star (23:35; formerly titled You’re Wrong About More Than You Think).
- How Simpson’s Paradox Explains Weird COVID-19 Statistics (about 8 minutes).
- Is Most Published Research Wrong? — Veritasium (about 12 minutes).
How can correct numbers support a misleading conclusion? Distinguish selection bias, survivorship bias, availability, and confirmation bias. What changes when we separate groups rather than combine them? How can research practices affect the reliability of published findings?
4. DataCamp: Intermediate Python, chapter 2
Complete chapter 2: Dictionaries & Pandas. Practice organizing and selecting data with dictionaries and DataFrames.
We will discuss the Good Judgment Project and forecasting in class; no forecasting reading is assigned.