Probability & Statistics

A visual introduction

Gabriel Cabrera^‡

‡ The University of Manchester

September 2026

About Me

Where to Find Me

gabriel.cabreraguzman@postgrad.manchester.ac.uk
gcabrerag.rbind.io
gcabrerag LinkedIn
gcabrerag.bsky.social Bluesky
@gabbocg GitHub
AMBS, The University of Manchester

How These Decks Work

Nine seminars, one idea each

The R runs here, in this page

No install · no account · nothing to download

The grey boxes below are real R. Press Run and the code executes inside your browser — there is no server, and nothing you type is sent anywhere.

Press Run a few times without changing anything. The mean comes out different every time.

Nine seminars One idea each, in the order we meet them. Nothing here needs the seminar before it to make sense.

Moving around Arrows or space step through. Esc shows every slide at once, M opens the contents.

Writing on it B opens a blackboard and C draws straight over a slide, so worked examples happen in front of you.

Describing Data I

Seminar 1 · Data visualisation

Which Picture for Which Data?

Tick the display that suits each set of data. The reasoning appears underneath.

ccOpts = ["Bar or pie", "Histogram", "Scatter plot", "Time series", "Box plot"]

// Exercise 1 of the beamer Seminar 1, answers and reasoning carried over.
ccQ = [
  {q: "The tax paid by a light engineering company and its annual profit, for 20 consecutive years.",
   a: "Scatter plot",
   hint: "Look again at how many measures there are, and at what pairs them up.",
   why: "Two numerical measures, and the question is whether they move together. The twenty years supply the pairs, they are not the thing being plotted."},
  {q: "The types of premises occupied by all the small businesses in a town: office, factory, warehouse, and so on.",
   a: "Bar or pie",
   hint: "Ask first whether any of this can go on a number line at all.",
   why: "The variable is categorical. There is nothing to put on a number line, only counts in named boxes, so a bar chart or a pie chart."},
  {q: "The salary of a trainee accountant in the first year of training, at 30 major accountancy firms.",
   a: "Histogram",
   hint: "One measure, thirty values of it, and the shape is what you are after.",
   why: "One numerical measure, and what you want to see is the shape of its spread. A stem-and-leaf does the same job by hand and keeps the digits."},
  {q: "The number of trainee accountants enrolling for professional examinations each year since 1960.",
   a: "Time series",
   hint: "There is one measure here, but it is useless without the year beside it.",
   why: "One measure again, but it only means anything read against the year it belongs to."},
  {q: "The years of experience and the current salary of a random sample of economists at a New York bank.",
   a: "Scatter plot",
   hint: "Count the measures recorded for each economist.",
   why: "Two numerical measures per person, so each economist is one point."}
]

// What each display is FOR. A wrong answer is explained by naming the job the
// chosen chart actually does, which is one line per option rather than one
// per (question, wrong option) pair — twenty of those would say the same five
// things over and over.
ccJob = ({
  "Bar or pie":   "A bar or pie chart compares counts across named categories",
  "Histogram":    "A histogram shows the shape of one numerical measure",
  "Scatter plot": "A scatter plot shows how two numerical measures move together",
  "Time series":  "A time series plot follows one measure against time",
  "Box plot":     "A box plot summarises the spread of one numerical measure"
})

ccStyle = ({fontSize: "13px", fontFamily: "Roboto, sans-serif",
            color: "#6E7681", background: "transparent"})

ccRD = [6.0, 10.4, 10.5, 9.0, 7.3, 6.6, 6.9, 8.2, 7.1, 8.1,
        7.9, 6.8, 7.4, 9.5, 8.1, 13.5, 9.9, 6.9, 11.1, 8.2,
        8.0, 7.7, 7.4, 6.5, 9.5, 8.2, 6.9, 7.2, 8.2, 6.7]

ccSalary = [846, 457, 563, 1466, 1200, 2164, 746, 1611, 824, 824, 1310,
            1007, 1367, 575, 1252, 1354, 2479, 1238, 927, 1253, 925,
            1284, 1279, 1660, 860]

// Five of the twenty-six, all with SHORT names. The bar chart is the only
// plot with words on its y axis, so it used to set the shared left margin at
// 88 — which left the other four, and the box plot worst of all, with a band
// of empty space down their left side and the drawing pushed off centre.
// Picking short names lets every plot share a 44px margin instead.
ccPay = [["CBS", 60.25], ["Exxon", 34.90], ["AT&T", 21.00],
         ["Apple", 4.17], ["Tesco", 1.90]].map(([k, v]) => ({k, v}))

ccPairs = [1, 2, 3, 4, 5, 6, 7].map((x, i) => ({x, y: [2, 4, 8, 16, 32, 64, 128][i]}))

ccMonths = [23, 39, 36, 26, 36, 12, 26, 30, 42, 35, 36, 28]
  .map((x, i) => ({month: i + 1, x}))

// One geometry for all five: identical margins mean the axes are the same
// length and the ticks the same size from plot to plot, so clicking between
// displays changes the picture and nothing else. marginLeft is set by the
// longest company name on the bar chart; the numeric plots spend that space
// on their y-axis label.
ccBase = ({
  width: 400, height: 152,
  marginTop: 10, marginRight: 16, marginBottom: 56, marginLeft: 66,
  style: ccStyle
})

// labelOffset 38 against a 48 bottom margin: Plot's default sets the axis
// label directly under the tick text and the two touched.
// The two labelOffsets are not equal because they are not measured the same
// way: Plot pushes the x label DOWN from the axis and the y label LEFT from
// it, past tick text of different lengths. These two land both labels the
// same distance clear of their tick numbers, which is what reads as even.
ccAxX = (label) => ({label, labelAnchor: "center", labelOffset: 46, tickSize: 5})
ccAxY = (label, ticks, domain) => ({label, labelAnchor: "center",
                                    labelOffset: 55, tickSize: 5,
                                    ticks, domain, grid: true})
// Explicit tick values, not a count. The shared geometry leaves a 94px plot
// area and `ticks: n` is only a HINT — d3 rounds to a nice step, so both 4
// and 5 came back with six labels 15px apart under 13px type, all but
// touching, while leaving it automatic dropped to two. Naming the values is
// the only way to fix the pitch, and these datasets never change.

ccDemo = ({
  "Bar or pie": () => Plot.plot({...ccBase,
    x: ccAxX("Executive pay ($m)"),
    y: {label: null, tickSize: 5},
    marks: [
      Plot.barX(ccPay, {y: "k", x: "v", fill: "#325D88",
        sort: {y: "x", reverse: true}}),
      Plot.ruleX([0], {stroke: "#B0B7C0"})
    ]}),
  "Histogram": () => Plot.plot({...ccBase,
    x: ccAxX("R&D spend (% of revenue)"),
    y: ccAxY("Firms", [0, 4, 8]),
    marks: [
      Plot.rectY(ccRD, Plot.binX({y: "count"},
        {x: d => d, thresholds: d3.range(6, 15), fill: "#325D88"})),
      Plot.ruleY([0], {stroke: "#B0B7C0"})
    ]}),
  "Scatter plot": () => Plot.plot({...ccBase,
    x: ccAxX("X"), y: ccAxY("Y", [0, 40, 80, 120]),
    marks: [Plot.dot(ccPairs, {x: "x", y: "y", fill: "#325D88", r: 3.4})]}),
  "Time series": () => Plot.plot({...ccBase,
    x: {...ccAxX("Month"), tickFormat: "d"}, // From zero: a demand axis that starts at 15 exaggerates every dip.
    y: ccAxY("Demand", [0, 25, 50], [0, 50]),
    marks: [Plot.line(ccMonths, {x: "month", y: "x",
      stroke: "#325D88", strokeWidth: 2})]}),
  "Box plot": () => Plot.plot({...ccBase,
    x: {...ccAxX("Executive salary (£000s)"), grid: true},
    y: {label: null, ticks: [], tickSize: 5},
    marks: [Plot.boxX(ccSalary, {x: d => d, fill: "#CBD9E6", stroke: "#325D88"})]})
})

ccIcon = {
  const NAVY = "#325D88";
  const wrap = inner =>
    `<svg width="42" height="26" viewBox="0 0 42 26" aria-hidden="true">${inner}</svg>`;
  // Bars on a 40-wide strip sitting on a baseline at y = 22. A histogram's
  // bars touch and a bar chart's do not, which is the one difference a
  // student is meant to read off these thumbnails.
  const bars = (hs, pitch, w) => hs.map((h, i) =>
    `<rect x="${2 + i * pitch}" y="${22 - h}" width="${w}" height="${h}" fill="${NAVY}"/>`
  ).join("");
  const line = (x1, y1, x2, y2) =>
    `<line x1="${x1}" y1="${y1}" x2="${x2}" y2="${y2}" stroke="${NAVY}" stroke-width="1.6"/>`;
  return ({
    "Bar or pie": wrap(bars([16, 9, 13, 6], 10, 7)),
    "Histogram": wrap(bars([5, 12, 18, 11, 5], 8, 8)),
    "Scatter plot": wrap([[5,20],[11,16],[16,18],[21,13],[26,11],[32,8],[38,5]]
      .map(([x, y]) => `<circle cx="${x}" cy="${y}" r="2.1" fill="${NAVY}"/>`).join("")),
    "Time series": wrap(
      `<polyline points="3,20 10,12 17,16 24,7 31,11 39,4" fill="none"
         stroke="${NAVY}" stroke-width="2" stroke-linejoin="round"/>`),
    "Box plot": wrap(line(3, 13, 12, 13) + line(28, 13, 39, 13) +
      `<rect x="12" y="7" width="16" height="12" fill="none" stroke="${NAVY}" stroke-width="1.6"/>` +
      line(21, 7, 21, 19))
  });
}

viewof ccI = Inputs.radio([1, 2, 3, 4, 5], {value: 1, label: "Question"})

ccPrompt = html`<div class="quiz-q">${ccQ[ccI - 1].q}</div>`

viewof ccPick = {
  ccI;   // naming it here rebuilds the input when the question changes, which
         // is the only way to clear the previous answer's ticks
  // html([...]) rather than html`...`: the sketch is a string of markup and
  // the tagged form would escape it into visible angle brackets.
  // radio, not checkbox: every question has exactly one right display, so
  // letting the room tick three of them only invites a wrong kind of answer.
  return Inputs.radio(ccOpts, {
    label: "Suitable display",
    format: v => html([`<span class="quiz-opt">${ccIcon[v]}<span>${v}</span></span>`])
  });
}

ccFeedback = {
  const want = ccQ[ccI - 1].a;
  if (!ccPick) return html`<div class="quiz-fb quiz-idle">Pick a display to see the reasoning.</div>`;
  const ok = ccPick === want;
  // A wrong pick names the job the chosen chart does and points back at the
  // data. It does NOT name the answer: handing it over ends the thinking, and
  // the radio is still there to try again with.
  const text = ok
    ? html`<b>Yes.</b> ${ccQ[ccI - 1].why}`
    : html`<b>Not quite.</b> ${ccJob[ccPick]}. ${ccQ[ccI - 1].hint}`;
  // The plot always shows the display that was PICKED, not the right one —
  // on a wrong answer it is the thing they asked for, which is an argument
  // rather than a giveaway.
  return html`<div class="quiz-answer">
    <div class="quiz-fb ${ok ? "quiz-ok" : "quiz-no"}">${text}</div>
    <figure class="quiz-plot">
      <figcaption>${ccPick}</figcaption>
      ${ccDemo[ccPick]()}
    </figure>
  </div>`;
}

Describing Data II

Seminar 2 · Descriptive statistics

Thirteen Salaries at a Small Firm

The Same Thirteen Salaries, in R

Seminar 2 · Descriptive statistics

Every summary you just watched is one line of R.

Change the last salary to 96 and run it again. The mode and the median count positions, so they do not move; the mean and the standard deviation measure distances, so they do.

A Bound that Holds for Any Shape

Seminar 2 · Chebyshev

Chebyshev promises at least \(1 - 1/k^2\) of any distribution lies within \(k\) standard deviations. No normality, no symmetry, no assumptions at all.

The bound holds, but loosely — that is the price of assuming nothing. Swap rexp(n, 1)^2 for rnorm(n) and the gap widens further: for a normal, \(k = 2\) captures 95%, not the 75% Chebyshev guarantees.

Basic Concepts

Seminar 3 · Conditional probability and Bayes

Probability of an Event

Count how many ways the event can happen, then how many things can happen at all. The probability is the ratio of the two.

\[P(\color{#93C54B}{A}) = \frac{\#\ \text{ways } \color{#93C54B}{A} \text{ can happen}}{\#\ \text{of things that can happen}} = \frac{|\color{#93C54B}{A}|}{|\Omega|}\]

Flip a coin twice

\(\Omega = \{hh,\ ht,\ th,\ tt\} \quad \color{#6E7681}{\text{4 in all}}\)

At least one head

\(\color{#93C54B}{A} = \{hh,\ ht,\ th\} \qquad P(\color{#93C54B}{A}) = \tfrac{3}{4} = 75\%\)

No heads

\(\color{#325D88}{B} = \{tt\} \qquad P(\color{#325D88}{B}) = \tfrac{1}{4} = 25\%\)

Roll two dice

\(\Omega = \{11,\ 12,\ \ldots,\ 66\} \quad \color{#6E7681}{\text{36 in all}}\)

At least one five

\(\color{#93C54B}{A} = \{15, 25, 35, 45, 51, 52, 53, 54, 55, 56, 65\}\)

\(P(\color{#93C54B}{A}) = \tfrac{11}{36}\)

Or split on the first die

\(P(\color{#93C54B}{A}) = \tfrac{1}{6} + \tfrac{5}{6} \times \tfrac{1}{6} = \tfrac{6}{36} + \tfrac{5}{36} = \tfrac{11}{36}\)

A tempting answer is 6 + 6 = 12, six rolls with a five on the first die and six with a five on the second. Five and five is in both lists, so twelve counts it twice. The split on the right avoids that, because its second case needs the first die not to be a five.

Sample Spaces and Events

Roll two dice and every possible result is on the board. The sample space \(\Omega\) is that whole board, and an event is any part of it you can point at.

Unions, Intersections and Complements

Two events on the same board. \(\color{#93C54B}{A} \cup \color{#325D88}{B}\) is “either”, \(\color{#93C54B}{A} \cap \color{#325D88}{B}\) is “both”, and \(\color{#93C54B}{A}^{c}\) is everything left over.

The Same Picture, Drawn as Circles

A Venn diagram is the board with the grid rubbed out. The four counts are the ones you just read off it, and they still add to 36.

Set Notation for Probability

Counting gives \(P(\color{#93C54B}{A}) = |\color{#93C54B}{A}|/|\Omega|\) and \(P(\color{#93C54B}{A}^{c}) = 1 - P(\color{#93C54B}{A})\). Two rules do the rest.

Either · \(\color{#93C54B}{A} \cup \color{#325D88}{B}\)

\[\begin{array}{@{}l@{\;\;}c@{\;\;}l@{}} P(\color{#93C54B}{A} \cup \color{#325D88}{B}) & = & P(\color{#93C54B}{A}) + P(\color{#325D88}{B}) - P(\color{#93C54B}{A} \cap \color{#325D88}{B})\\[4pt] \phantom{P(\color{#93C54B}{A} \cap \color{#325D88}{B})} & = & P(\color{#93C54B}{A}) + P(\color{#325D88}{B}) \qquad \color{#6E7681}{\text{if mutually exclusive}} \end{array}\]

Both · \(\color{#93C54B}{A} \cap \color{#325D88}{B}\)

\[\begin{array}{@{}l@{\;\;}c@{\;\;}l@{}} P(\color{#93C54B}{A} \cap \color{#325D88}{B}) & = & P(\color{#93C54B}{A}) \times P(\color{#325D88}{B} \mid \color{#93C54B}{A})\\[4pt] \phantom{P(\color{#93C54B}{A} \cap \color{#325D88}{B})} & = & P(\color{#93C54B}{A}) \times P(\color{#325D88}{B}) \qquad \color{#6E7681}{\text{if independent}} \end{array}\]

Conditional Probabilities

Being told that \(\color{#325D88}{B}\) happened throws away every outcome outside it. What is left is a smaller sample space, and the question is how much of that is \(\color{#93C54B}{A}\).

\[P(\color{#93C54B}{A} \mid \color{#325D88}{B}) = \frac{P(\color{#93C54B}{A} \cap \color{#325D88}{B})}{P(\color{#325D88}{B})}\]

Knowing it changes nothing \(\color{#325D88}{B}\) is “the second die is a five”

Six rolls survive and one of them starts with a three, so \(P(\color{#93C54B}{A} \mid \color{#325D88}{B}) = \tfrac{1}{6}\), which is \(P(\color{#93C54B}{A})\) over again. The dice are independent.

Knowing it changes everything \(\color{#325D88}{C}\) is “the total is six”

Now only five rolls survive and one of them starts with a three, so \(P(\color{#93C54B}{A} \mid \color{#325D88}{C}) = \tfrac{1}{5}\). The total tells you something about the first die.

Here \(\color{#93C54B}{A}\) is “the first die is a three”. Reversing the bar asks something else, and that is what Bayes is for.

Conditioning Narrows the Sample Space

Eight in ten office managers would buy the software (\(\color{#93C54B}{A}\)), and four in ten of those would also upgrade (\(\color{#325D88}{B}\)). Of the rest, one in ten would. Given \(\color{#325D88}{B}\), how likely is \(\color{#93C54B}{A}\)?

The Law of Total Probability

Cut \(\Omega\) into pieces that do not overlap and leave nothing out. Every outcome in \(\color{#93C54B}{A}\) then lies in exactly one piece, so the probability of \(\color{#93C54B}{A}\) is its slices added back up.

\[P(\color{#93C54B}{A}) = \sum_{i} P(\color{#93C54B}{A} \mid \color{#325D88}{B_i})\, P(\color{#325D88}{B_i})\]

Supplier 1 Half the parts, two per hundred faulty

\[0.50 \times 0.02 = 0.010\]

Supplier 2 Three parts in ten, four per hundred faulty

\[0.30 \times 0.04 = 0.012\]

Supplier 3 One part in five, seven per hundred faulty

\[0.20 \times 0.07 = 0.014\]

Adding the three routes gives \(P(\text{faulty}) = 0.036\). Ask instead which supplier sent the faulty part in your hand, and you are asking for Bayes.

Librarian or Farmer?

Steve is very shy and withdrawn, invariably helpful but with very little interest in people or in the world of reality. A meek and tidy soul, he has a need for order and structure, and a passion for detail.

Twenty Farmers for Every Librarian

The Formula is Just That Picture

Move the Prior, Watch the Answer Move

Seminar 3 · the base rate does the work

The evidence is identical every time — only the prior changes. 1/21 is the real ratio, one librarian for every twenty farmers; at 0.50 you get 0.80. The description never had an opinion about how common librarians are, so the base rate has to supply it.

Monty Hall Problem

A car sits behind one of three doors and a goat behind each of the others. You pick door 1. The host, who knows where the car is, opens door 3 on a goat and offers you the swap. Do you take it?

Combinatorics and the Factorial

There are \(n\) choices and you are taking \(r\) of them. Two questions settle which formula you need, whether you sample with replacement and whether the order matters.

\[n! = n \times (n-1) \times \cdots \times 2 \times 1 \qquad 0! = 1\]

With replacement Order matters

\[n^{\,r}\]

Ten coin flips in a row

\(2^{10} = 1{,}024\)

Without replacement Order matters Permutation · \(^{n}P_{r}\)

\[\dfrac{n!}{(n-r)!}\]

Five cards off the top of the deck

\(\begin{array}{@{}r@{\;}c@{\;}l@{}} \dfrac{52!}{47!} & = & 52 \times 51 \times 50 \times 49 \times 48 \\[3pt] & = & 311{,}875{,}200 \end{array}\)

Without replacement Order does not matter Combination · \(^{n}C_{r}\)

\[\dfrac{n!}{(n-r)!\,r!} = \dbinom{n}{r}\]

One hand, dealt either way

K Q 2 3 7K 2 Q 7 3

\(\dbinom{52}{5} = 2{,}598{,}960\)

The last two differ by \(r!\), the orders the \(r\) you picked could have come in. Those 311,875,200 runs collapse to 2,598,960 hands because one hand can be dealt \(5! = 120\) ways.

Probability Distributions

Seminar 4 · Binomial, Poisson, exponential, normal

Random Variables

A random variable is not really a variable. It is a rule that hands every outcome in the sample space a number, which is what lets us do arithmetic with chance.

\[X : \Omega \to \mathbb{R} \qquad\qquad P(X = x)\]

Discrete Values you can list

Roll two dice and let \(\color{#93C54B}{X}\) be the total. It is the number already written in every cell of the board.

3 + 42 + 56 + 1

Six of the thirty-six cells carry a 7, so

\(P(\color{#93C54B}{X} = 7) = \tfrac{6}{36}\)

Continuous Values filling a range

Wait at a bus stop and let \(\color{#325D88}{X}\) be the minutes until the next arrival. It can be 3, or 3.1, or 3.0004.

3.03.13.0004

No single value can carry any probability, so ask for a stretch instead

\(P(2 \le \color{#325D88}{X} \le 4)\)

Which kind you have decides the tool. A discrete variable gives every value its own probability and you add them up. A continuous one spreads probability along a line and you measure the area under a curve. The binomial and the Poisson ahead are discrete, the exponential and the normal are continuous.

A Distribution Is a Function

Hand \(P\) a value of \(X\) and it hands back the probability of that value. That function is the distribution, and writing it down once replaces all the counting.

\[\sum_{k} P(X = k) = 1 \qquad\qquad \int p(x)\,dx = 1\]

Flip a coin four times \(\Omega\) has sixteen outcomes, \(\color{#93C54B}{X}\) counts the heads

Why bother Three things the function buys you

One summary. Write \(P(\color{#93C54B}{X})\) once and every probability is already in it, with no sample space left to count.

A model. The distribution is a claim about the process, so data can test it and estimate the numbers inside it.

Arithmetic. Add the bars over a range, or measure the area under the curve, and you have \(P(a \le \color{#93C54B}{X} \le b)\).

The two examples ahead already have names. Heads in \(n\) flips is the binomial, and a height or a measurement error is the normal, written \(\color{#93C54B}{X} \sim \text{Binomial}\) and \(\color{#325D88}{X} \sim \text{Normal}\).

Which Seller Do You Trust?

Three sellers on the same marketplace, same item, same price. The only thing you have to go on is what previous buyers said about them. Which one do you buy from?

Laplace’s Rule of Succession

In 1774 Laplace said not to rate a seller \(k/n\). Credit one imaginary good review and one bad one, then rate them \((k+1)/(n+2)\).

Where the Binomial Comes From

Every review is an independent draw at one constant unknown rate \(S\). We cannot work back to it, only forwards. Suppose \(S\) and see what it predicts.

What If the Rate Were Different?

One guess at \(S\) gives one number. Turn the dial and the top plot reshapes, while the bottom traces that number against every \(S\) it could be.

viewof s = Inputs.range([0.50, 1.00], {value: 0.95, step: 0.01, label: "True success rate s"})

n = 50
k = 48

// The deck's palette, so the plots match the hand-drawn stages either side.
PAL = ({olive: "#93C54B", brick: "#B94A48", navy: "#325D88",
        orange: "#F47C3C", soft: "#6E7681", dim: "#B0B7C0"})

// Binomial pmf by recurrence: P(0) = (1-p)^n, then
// P(j) = P(j-1) * (n-j+1)/j * p/(1-p). No factorial is ever formed, so
// nothing overflows — the same route stage-kit.html uses.
pmf = function (p) {
  const q = 1 - p;
  if (q <= 0) return Array.from({length: n + 1}, (_, i) => (i === n ? 1 : 0));
  let v = Math.pow(q, n);
  const out = [v];
  for (let j = 1; j <= n; j++) { v = v * ((n - j + 1) / j) * (p / q); out.push(v); }
  return out;
}

pk = pmf(s)
value = pk[k]
lik = d3.range(0, 1.0005, 0.004).map(sv => ({s: sv, p: pmf(sv)[k]}))

PSTYLE = ({fontSize: "15px", fontFamily: "Roboto, sans-serif",
           color: PAL.soft, background: "transparent"})

Plot.plot({
  // marginTop 16, not 4: at 4 the 30% tick label has no room above the top
  // gridline and reads as cut off. Height grows to match so the plotting
  // area is unchanged.
  width: 950, height: 154, marginLeft: 64, marginBottom: 24, marginTop: 16,
  style: PSTYLE,
  x: {domain: [-0.6, n + 0.6], ticks: d3.range(0, n + 1, 5), label: null, tickSize: 4},
  // labelAnchor centre rotates the label down the axis; Plot's default puts
  // it at the top, where it sits on the top tick.
  // Both plots are pinned to 0-30% so the two heights read against the same
  // scale. This one does exceed it once s passes about 0.97 — at s = 1 every
  // draw is fifty ticks, so that bar is 100% — hence clip, which flattens it
  // at the top of the frame instead of letting it paint over the equation.
  y: {domain: [0, 0.30], ticks: [0.1, 0.2, 0.3], grid: true,
      tickFormat: d => d3.format(".0%")(d), label: "P(k | s)",
      labelAnchor: "center", labelOffset: 56},
  marks: [
    Plot.ruleX(d3.range(n + 1), {x: d => d, y2: d => pk[d], strokeWidth: 5,
      stroke: d => (d === k ? PAL.olive : PAL.dim), clip: true}),
    Plot.ruleY([0], {stroke: PAL.dim})
  ]
})

html`<div class="ojs-eq">P(<span class="hit">48&#10003;</span>, <span class="miss">2&#10007;</span> | s = <span class="rate">${s.toFixed(2)}</span>) = <b>${value.toFixed(4)}</b></div>`

Plot.plot({
  // marginTop 16, not 4: at 4 the 30% tick label has no room above the top
  // gridline and reads as cut off. Height grows to match so the plotting
  // area is unchanged.
  width: 950, height: 154, marginLeft: 64, marginBottom: 24, marginTop: 16,
  style: PSTYLE,
  x: {domain: [0, 1], ticks: [0, 0.2, 0.4, 0.6, 0.8, 1], label: null, tickSize: 4},
  y: {domain: [0, 0.30], ticks: [0.1, 0.2, 0.3], grid: true,
      tickFormat: d => d3.format(".0%")(d),
      label: "P(data | s)", labelAnchor: "center", labelOffset: 56},
  marks: [
    Plot.line(lik, {x: "s", y: "p", stroke: PAL.navy, strokeWidth: 2.5}),
    Plot.ruleX([s], {stroke: PAL.soft, strokeDasharray: "4 4"}),
    Plot.dot([{s: s, p: value}], {x: "s", y: "p", fill: PAL.olive, r: 6,
      stroke: "white", strokeWidth: 1.5}),
    Plot.ruleY([0], {stroke: PAL.dim})
  ]
})

The Binomial in Standard Notation

Everything so far, in the notation the worksheet uses. The rate we have been calling \(S\) is written \(\color{#F47C3C}{p}\); its complement is \(\color{#B94A48}{q} = 1 - \color{#F47C3C}{p}\).

\[P(X = \color{#93C54B}{k}) \;=\; \binom{n}{\color{#93C54B}{k}}\; \color{#F47C3C}{p}^{\,\color{#93C54B}{k}}\; \color{#B94A48}{q}^{\,\color{#B94A48}{n-k}} \qquad X \sim \operatorname{Binomial}(n, \color{#F47C3C}{p})\]

One review is a Bernoulli trial, a single attempt with two outcomes, scored \(1\) for good and \(0\) for bad.

\[P(B_i = 1) = \color{#F47C3C}{p}, \quad P(B_i = 0) = \color{#B94A48}{q}, \quad \operatorname{E}[B_i] = \color{#F47C3C}{p}, \quad \operatorname{Var}(B_i) = \color{#F47C3C}{p}\color{#B94A48}{q}\]

\[X \;=\; B_1 + B_2 + \cdots + B_n, \qquad B_i \sim \operatorname{Bernoulli}(\color{#F47C3C}{p})\]

The Rare Event Limit of a Binomial Distribution

Hold the average \(\color{#F47C3C}{\lambda} = n\,\color{#F47C3C}{p}\) still and push \(n\) up. Each trial gets rarer, but you expect the same number of events.

viewof nTrials = Inputs.range([10, 400], {value: 10, step: 1, label: "Trials n"})
// Floor of 10, not 5: lam runs to 8, and n < lam would ask for p > 1, which
// the recurrence below happily turns into a 1971% gap. NB a comment must NOT
// be the first line of an {ojs} cell: it shadows the '//|' directive and
// Quarto then emits the whole slide as plain source, with no runtime and no
// error anywhere.

viewof lam = Inputs.range([1, 8], {value: 3, step: 0.5, label: "Mean λ = np"})

PPAL = ({olive: "#93C54B", brick: "#B94A48", navy: "#325D88",
         orange: "#F47C3C", soft: "#6E7681", dim: "#B0B7C0"})

pp = lam / nTrials                       // the success rate, shrinking as nTrials grows
KMAX = Math.max(12, Math.ceil(lam + 4 * Math.sqrt(lam)))
ks = d3.range(0, KMAX + 1)

// Binomial(nTrials, pp) by the same recurrence used everywhere else in this deck
binom = {
  const q = 1 - pp;
  if (q <= 0) return ks.map(k => (k === nTrials ? 1 : 0));
  let v = Math.pow(q, nTrials);
  const out = [v];
  const top = Math.min(nTrials, KMAX);   // KMAX <= 20; no point walking to 20,000
  for (let j = 1; j <= top; j++) { v = v * ((nTrials - j + 1) / j) * (pp / q); out.push(v); }
  return ks.map(k => (k <= nTrials ? out[k] : 0));
}

// Poisson(lam): P(0) = e^-lam, then P(k) = P(k-1) * lam / k
pois = {
  const out = []; let v = Math.exp(-lam);
  for (let k = 0; k <= KMAX; k++) { out.push(v); v = v * lam / (k + 1); }
  return out;
}

gap = d3.max(ks, k => Math.abs(binom[k] - pois[k]))
YMAX = Math.max(d3.max(binom), d3.max(pois)) * 1.18

Plot.plot({
  width: 950, height: 205, marginLeft: 64, marginBottom: 26, marginTop: 16,
  style: {fontSize: "15px", fontFamily: "Roboto, sans-serif",
          color: PPAL.soft, background: "transparent"},
  x: {domain: [-0.6, KMAX + 0.6], label: null, tickSize: 4},
  y: {domain: [0, YMAX], grid: true, tickFormat: d => d3.format(".0%")(d),
      label: "P(k)", labelAnchor: "center", labelOffset: 56},
  marks: [
    Plot.ruleX(ks, {x: d => d, y2: d => binom[d], strokeWidth: 9,
      stroke: PPAL.dim, clip: true}),
    // "step" is Plot's midpoint step (d3.curveStep); "step-mid" is not a
    // curve name and throws, taking the whole plot cell with it.
    Plot.line(ks, {x: d => d, y: d => pois[d], stroke: PPAL.brick,
      strokeWidth: 2, curve: "step", clip: true}),
    Plot.dot(ks, {x: d => d, y: d => pois[d], fill: PPAL.brick, r: 3.2,
      clip: true}),
    Plot.ruleY([0], {stroke: PPAL.dim})
  ]
})

html`<div class="ojs-eq">Binomial(${nTrials}, <span class="rate">${pp.toFixed(4)}</span>) &nbsp;→&nbsp; <span class="miss">Poisson(${lam.toFixed(1)})</span> &nbsp;·&nbsp; gap <b>${(100*gap).toFixed(2)}%</b></div>`

The Poisson in Standard Notation

Put \(\color{#F47C3C}{p} = \color{#F47C3C}{\lambda}/n\) into the binomial and let \(n\) run away.

\[P(X = \color{#93C54B}{k}) \;=\; \binom{n}{\color{#93C54B}{k}}\big(\tfrac{\color{#F47C3C}{\lambda}}{n}\big)^{\color{#93C54B}{k}}\big(1-\tfrac{\color{#F47C3C}{\lambda}}{n}\big)^{n-\color{#93C54B}{k}}\]

↓ Group the factors by whether they still mention \(n\)

\[=\;\frac{\color{#F47C3C}{\lambda}^{\,\color{#93C54B}{k}}}{\color{#93C54B}{k}!}\;\cdot\;\frac{n(n-1)\cdots(n-\color{#93C54B}{k}+1)}{n^{\color{#93C54B}{k}}}\;\cdot\;\big(1-\tfrac{\color{#F47C3C}{\lambda}}{n}\big)^{n}\;\cdot\;\big(1-\tfrac{\color{#F47C3C}{\lambda}}{n}\big)^{-\color{#93C54B}{k}}\]

↓ As \(n \to \infty\) those three go to \(1\), to \(e^{-\color{#F47C3C}{\lambda}}\), and to \(1\)

\[P(X = \color{#93C54B}{k}) \;=\; \frac{\color{#F47C3C}{\lambda}^{\,\color{#93C54B}{k}}\, e^{-\color{#F47C3C}{\lambda}}}{\color{#93C54B}{k}!} \qquad X \sim \operatorname{Poisson}(\color{#F47C3C}{\lambda})\]

Check the approximation

Seminar 4 · exact vs approximate

Three mistakes per 10,000 transactions; 10,000 audited. What is the chance of four or more? Compute it both ways.

Both read 0.3527681, so every digit R prints agrees and the approximation costs you nothing here. Now set n <- 20; p <- 0.15 (still \(np = 3\)) and watch it open up. That is what “large \(n\), small \(p\)” is protecting you from.

Time Between Poisson Events

Count arrivals, or time the wait. The bridge is the \(\color{#93C54B}{k = 0}\) bar.

viewof lamE = Inputs.range([0.5, 4], {value: 2, step: 0.5, label: "Rate λ per minute"})

viewof winT = Inputs.range([0.1, 2.5], {value: 1, step: 0.05, label: "Window t minutes"})

muE = lamE * winT                     // arrivals expected in the window
KMAXE = Math.max(8, Math.ceil(muE + 4 * Math.sqrt(muE)))
ksE = d3.range(0, KMAXE + 1)

// Poisson(muE) by the deck's usual recurrence: P(0) = e^-mu, P(k) = P(k-1) mu/k
poisE = {
  const out = []; let v = Math.exp(-muE);
  for (let k = 0; k <= KMAXE; k++) { out.push(v); v = v * muE / (k + 1); }
  return out;
}

// The whole point of the slide: the k = 0 bar IS the exponential's survival
// curve. P(no arrival in [0, t]) = e^(-lambda t) = P(wait > t).
p0E = Math.exp(-muE)
TE = 3                                // minutes on the bottom axis
densE = t => lamE * Math.exp(-lamE * t)
gridE = d3.range(0, 241).map(i => TE * i / 240)
tailE = d3.range(0, 161).map(i => winT + (TE - winT) * i / 160)
YPOIS = d3.max(poisE) * 1.18
YDENS = lamE * 1.12
ESTYLE = ({fontSize: "15px", fontFamily: "Roboto, sans-serif",
           color: PPAL.soft, background: "transparent"})

Plot.plot({
  width: 950, height: 154, marginLeft: 64, marginBottom: 24, marginTop: 16,
  style: ESTYLE,
  x: {domain: [-0.6, KMAXE + 0.6], label: null, tickSize: 4},
  y: {domain: [0, YPOIS], grid: true, tickFormat: d => d3.format(".0%")(d),
      label: "P(k arrivals in t)", labelAnchor: "center", labelOffset: 56},
  marks: [
    Plot.ruleX(ksE, {x: d => d, y2: d => poisE[d], strokeWidth: 7,
      stroke: d => (d === 0 ? PPAL.olive : PPAL.dim), clip: true}),
    Plot.ruleY([0], {stroke: PPAL.dim})
  ]
})

html`<div class="ojs-eq">P(<span class="hit">0 arrivals</span> in ${winT.toFixed(2)} min) &nbsp;=&nbsp; <b>${p0E.toFixed(4)}</b> &nbsp;=&nbsp; P(<span class="hit">wait &gt; ${winT.toFixed(2)}</span>) &nbsp;·&nbsp; <span class="rate">λ = ${lamE.toFixed(1)}</span></div>`

Plot.plot({
  width: 950, height: 154, marginLeft: 64, marginBottom: 24, marginTop: 16,
  style: ESTYLE,
  x: {domain: [0, TE], label: null, tickSize: 4},
  y: {domain: [0, YDENS], grid: true, tickFormat: d => d3.format(".1f")(d),
      label: "density of the wait", labelAnchor: "center", labelOffset: 56},
  marks: [
    // This olive area is the same number as the olive bar above it.
    Plot.areaY(tailE, {x: t => t, y: densE, fill: PPAL.olive,
      fillOpacity: 0.3, clip: true}),
    Plot.line(gridE, {x: t => t, y: densE, stroke: PPAL.brick,
      strokeWidth: 2.5, clip: true}),
    Plot.ruleX([winT], {stroke: PPAL.soft, strokeDasharray: "4 4"}),
    Plot.ruleY([0], {stroke: PPAL.dim})
  ]
})

The Exponential in Standard Notation

The wait beats \(t\) exactly when nothing arrives in \([0, t]\), so read the Poisson at \(\color{#93C54B}{k = 0}\).

\[P(T > t) \;=\; P\big(N(t) = \color{#93C54B}{0}\big), \qquad N(t) \sim \operatorname{Poisson}(\color{#F47C3C}{\lambda} t)\]

↓ Put \(\color{#93C54B}{k = 0}\) into the Poisson formula

\[=\;\frac{(\color{#F47C3C}{\lambda} t)^{\color{#93C54B}{0}}\, e^{-\color{#F47C3C}{\lambda} t}}{\color{#93C54B}{0}!} \;=\; e^{-\color{#F47C3C}{\lambda} t}\]

↓ That is the survival curve, so the density is the slope of \(1 - e^{-\color{#F47C3C}{\lambda} t}\)

\[f(t) \;=\; \color{#F47C3C}{\lambda}\, e^{-\color{#F47C3C}{\lambda} t} \qquad T \sim \operatorname{Exponential}(\color{#F47C3C}{\lambda})\]

The Limit of a Binomial Distribution for Large n

Hold \(\color{#F47C3C}{p}\) still and push \(n\) up, and the bars settle onto a bell.

viewof nBig = Inputs.range([5, 200], {value: 10, step: 1, label: "Trials n"})

viewof pBig = Inputs.range([0.05, 0.95], {value: 0.35, step: 0.05, label: "Success rate p"})

muB = nBig * pBig
sdB = Math.sqrt(nBig * pBig * (1 - pBig))
kLo = Math.max(0, Math.floor(muB - 4 * sdB))
kHi = Math.min(nBig, Math.ceil(muB + 4 * sdB))
ksB = d3.range(kLo, kHi + 1)

// Binomial(nBig, pBig) by the deck's usual recurrence, walked only as far as
// the plot shows: P(0) = (1-p)^n, P(j) = P(j-1) (n-j+1)/j p/(1-p).
binomB = {
  const q = 1 - pBig;
  let v = Math.pow(q, nBig);
  const out = [v];
  for (let j = 1; j <= kHi; j++) { v = v * ((nBig - j + 1) / j) * (pBig / q); out.push(v); }
  return out;
}

// De Moivre's curve: N(np, np(1-p)), read as a density at each whole k.
normB = x => Math.exp(-((x - muB) ** 2) / (2 * sdB * sdB)) / (sdB * Math.sqrt(2 * Math.PI))
gapB = d3.max(ksB, k => Math.abs(binomB[k] - normB(k)))
YMAXB = d3.max(ksB, k => Math.max(binomB[k], normB(k))) * 1.18
BWB = Math.max(1.5, Math.min(9, 880 / (kHi - kLo + 1) - 1))
gridB = d3.range(0, 241).map(i => kLo + (kHi - kLo) * i / 240)
BSTYLE = ({fontSize: "15px", fontFamily: "Roboto, sans-serif",
           color: PPAL.soft, background: "transparent"})

Plot.plot({
  width: 950, height: 205, marginLeft: 64, marginBottom: 26, marginTop: 16,
  style: BSTYLE,
  x: {domain: [kLo - 0.6, kHi + 0.6], label: null, tickSize: 4},
  y: {domain: [0, YMAXB], grid: true, tickFormat: d => d3.format(".0%")(d),
      label: "P(k)", labelAnchor: "center", labelOffset: 56},
  marks: [
    Plot.ruleX(ksB, {x: d => d, y2: d => binomB[d], strokeWidth: BWB,
      stroke: PPAL.dim, clip: true}),
    Plot.line(gridB, {x: d => d, y: normB, stroke: PPAL.navy,
      strokeWidth: 2.5, clip: true}),
    Plot.ruleY([0], {stroke: PPAL.dim})
  ]
})

html`<div class="ojs-eq">Binomial(${nBig}, <span class="rate">${pBig.toFixed(2)}</span>) &nbsp;→&nbsp; <span class="miss">N(${muB.toFixed(1)}, ${(sdB*sdB).toFixed(1)})</span> &nbsp;·&nbsp; gap <b>${(100*gapB).toFixed(2)}%</b></div>`

The Galton Board

Galton’s 1889 machine: a coin flip at every peg, and a binomial built out of falling balls.

The Idea Behind the Central Limit Theorem

Let’s Try Some Simulations

Throw \(\color{#F47C3C}{n}\) of them and add them up, 2,500 times.

The Normal Distribution

That bell has a formula. \(\color{#B94A48}{\mu}\) slides it, \(\color{#F47C3C}{\sigma}\) stretches it, and the area under it is always 1.

\[f(x) \;=\; \frac{1}{\color{#F47C3C}{\sigma}\sqrt{2\pi}}\; e^{-\frac{1}{2}\left(\frac{x-\color{#B94A48}{\mu}}{\color{#F47C3C}{\sigma}}\right)^{2}}\]

viewof nf = Inputs.form({
  sig: Inputs.range([0.4, 2], {value: 0.84, step: 0.01, label: htl.html`Spread <b style="color:${PPAL.orange}">σ</b>`}),
  mu:  Inputs.range([-2, 2], {value: 0.34, step: 0.01, label: htl.html`Centre <b style="color:${PPAL.brick}">μ</b>`})
}, {template: ins => htl.html`<div class="nf-dials">${Object.values(ins)}</div>`})
nfSig = nf.sig
nfMu = nf.mu

// Three readings. Only the first moves: the area and the ±σ share are the
// same for every normal, which is the point of showing them beside it.
html`<div class="nf-card">
  <div><span>Peak height</span><b>${nfPeak.toFixed(3)}</b></div>
  <div><span>Area under the curve</span><b>1</b></div>
  <div><span>Within μ ± σ</span><b class="nf-band">68.3%</b></div>
</div>`

Plot.plot({
  width: 640, height: 330, marginLeft: 56, marginBottom: 28, marginTop: 20, marginRight: 10,
  style: NFSTYLE,
  x: {domain: [-4, 4], label: null, tickSize: 4},
  // up to 1.1: room for σ = 0.4, whose peak is 0.997, and the arrows over it
  y: {domain: [0, 1.1], grid: true, ticks: [0, 0.5, 1], label: "f(x)",
      labelAnchor: "center", labelOffset: 46},
  marks: [
    // the reference: N(0, 1), dashed, behind everything
    Plot.line(nfGrid, {x: d => d, y: nfStd, stroke: PPAL.dim, strokeWidth: 1.5, strokeDasharray: "5 5"}),
    Plot.areaY(nfGrid, {x: d => d, y: nfDens, fill: PPAL.navy, fillOpacity: 0.08}),
    // within one σ of the centre: always 68.3% of the area
    Plot.areaY(nfBand, {x: d => d, y: nfDens, fill: PPAL.orange, fillOpacity: 0.18}),
    Plot.line(nfGrid, {x: d => d, y: nfDens, stroke: PPAL.navy, strokeWidth: 2.5}),
    // the peak, read across to the axis: 1/(σ√2π), never 1 unless σ ≈ 0.4
    Plot.ruleY([nfPeak], {x1: -4, x2: nfMu, stroke: PPAL.navy, strokeDasharray: "2 4", strokeOpacity: 0.7}),
    Plot.dot([nfMu], {x: d => d, y: nfPeak, r: 4.5, fill: PPAL.navy}),
    Plot.text([nfPeak], {x: -4, y: d => d, dx: 4, dy: -9, textAnchor: "start",
      text: d => d.toFixed(3), fill: PPAL.navy, fontSize: 14, fontWeight: 700}),
    Plot.text(["N(0, 1)"], {x: 2.6, y: nfStd(2.6) + 0.07, text: d => d,
      fill: PPAL.soft, fontSize: 14}),
    Plot.ruleX([nfMu - nfSig, nfMu + nfSig], {y1: 0, y2: nfArrowY + 0.05,
      stroke: PPAL.orange, strokeWidth: 1.5}),
    Plot.ruleX([nfMu], {y1: 0, y2: nfArrowY + 0.05, stroke: PPAL.brick, strokeWidth: 2}),
    Plot.arrow([-1, 1], {x1: nfMu, x2: d => nfMu + d * nfSig, y1: nfArrowY, y2: nfArrowY,
      stroke: PPAL.orange, strokeWidth: 2.5, headLength: 10, inset: 4}),
    Plot.text([-1, 1], {x: d => nfMu + d * nfSig / 2, y: nfArrowY, dy: -12, text: () => "σ",
      fill: PPAL.orange, fontSize: 20, fontWeight: 700}),
    Plot.text([nfMu], {x: d => d, y: nfArrowY + 0.05, dy: -12, text: () => "μ",
      fill: PPAL.brick, fontSize: 20, fontWeight: 700}),
    // in the left half of the band, clear of the μ line at every σ
    Plot.text([nfMu - nfSig / 2], {x: d => d, y: nfPeak * 0.3, text: () => "68.3%",
      fill: PPAL.orange, fontSize: 16, fontWeight: 700}),
    Plot.ruleY([0], {stroke: PPAL.dim})
  ]
})

Expected Value, Variance and Standard Deviation

The Law of Large Numbers, Intuitively

Unpacking the Normal Formula

Every piece of the formula has a job. The \(e\) and the square make the bell, \(\sigma\) sets its width, \(\mu\) moves it, and the constant in front keeps the area at 1.

Where the \(e\) in the Normal Comes From

Forget the binomial for a moment. Ask only that errors in two directions be independent and that no direction be special, and the bell has no choice about its shape.

Where the \(\pi\) in the Normal Comes From

Call \(C\) the area under the bell \(e^{-x^2}\). Build the round hill from two of them, then measure its volume in two different ways.

The Box Plot on a Normal Curve

A box plot is five numbers and no assumptions. Lay it over a normal curve and all five land in fixed places, which is where the \(1.5 \times \text{IQR}\) rule comes from.

\(Q_1\) and \(Q_3\) sit at \(\mp 0.6745\sigma\), so the fences fall at \(\mp 2.698\sigma\) and only \(0.7\%\) of a normal lies beyond them. In a sample of a thousand, about seven points get flagged with nothing wrong.

Functions of a Random Variable

For \(\color{#93C54B}{Y} = g(X)\), work through the CDF and never the density. Ask which values of \(X\) make \(\color{#93C54B}{Y}\) small enough.

\[F_{\color{#93C54B}{Y}}(y) \;=\; P(a X + b \le y) \;=\; P\big(X \le \tfrac{y-b}{a}\big)\]

↓ The right-hand side is just \(X\)’s own CDF, read at a shifted point

\[F_{\color{#93C54B}{Y}}(y) \;=\; F_X\big(\tfrac{y-b}{a}\big)\]

↓ Differentiate in \(y\); the chain rule brings down the \(1/a\)

\[f_{\color{#93C54B}{Y}}(y) \;=\; \tfrac{1}{|a|}\, f_X\big(\tfrac{y-b}{a}\big) \qquad\Longrightarrow\qquad \color{#93C54B}{Y} \sim \operatorname{N}\big(a\mu + b,\; a^{2}\sigma^{2}\big)\]

The Chi-Square Distribution

Square \(k\) independent standard normals and add them up. The total is never negative, and it is the yardstick for sample variances, since \((n-1)s^2/\sigma^2 \sim \chi^2_{n-1}\).

The Student’s \(t\) Distribution

Swap the unknown \(\sigma\) for the sample’s \(s\) and the bell gets fatter tails. Formally \(T = Z/\sqrt{V/k}\) with \(V \sim \chi^2_k\), which is why the chi-square came first.

Relationships Among Probability Distributions

Each distribution is one step from another: a sum, a limit, a square or a ratio.

Thank You

Some slides are based on the videos of 3Blue1Brown.

Slides created with Quarto and reveal.js.

R runs in the browser via webR and the quarto-webr extension; interactive plots use Observable JS.

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. To view a copy of this license, visit:

https://creativecommons.org/licenses/by-sa/4.0/