Skip to content
Statistics LabMAST20005 · 2020

Confidence intervals

Assignment 2 asked for intervals in four flavours: a t interval for one mean, the sample size for a given precision, a pooled interval for a difference of means with an F interval for the variance ratio, and a two-proportion test. This chapter rebuilds each one and starts with the idea that makes them all meaningful — coverage.

§3.1Coursework A2 Q2(a), Q4

Ninety-five out of a hundred

With σ unknown, the interval for a normal mean replaces σ by the sample standard deviation and pays for it with a heavier-tailed critical value:

xˉ±tn−1,  1−α/2 sn\bar{x} \pm t_{n-1,\;1-\alpha/2}\,\frac{s}{\sqrt{n}}

It works because T=Xˉ−μS/n∼tn−1T=\frac{\bar X-\mu}{S/\sqrt n}\sim t_{n-1} whatever the values of μ and σ — the pivot identified in Q4. Draw many samples and the intervals scatter around the truth; about 95% of them catch it.

10
100
95%

Interval

Each sample is rnorm(10, 15, 3). With seed 1, n = 10 and 100 t intervals, R’s own loop over t.test() also gives 99 hits.

Intervals covering μ

99 / 100

Intervals missing μ

1

Observed coverage

99.0%

Wilson 95% 94.6% to 99.8%

Nominal coverage

95%

expect ≈ 5 misses

  • Interval covers μ = 15
  • Interval misses μ (marked ×)
  • True mean μ
Figure 3.1.100 independent 95% t intervals, each from a fresh sample of size 10. The procedure, not any single interval, carries the 95% guarantee: in the long run that share of intervals catches the fixed but unknown μ.
Show the numbers as a table
Simulated intervals
#x̄loweruppercovers μ
115.39713.72117.072yes
215.74713.45118.042yes
314.59912.54816.650yes
415.36213.62717.097yes
515.40214.12416.681yes
615.43013.25317.608yes
716.35413.58819.119yes
814.25712.90615.608yes
915.38213.66217.102yes
1015.33713.07817.596yes
1116.04214.03118.054yes
1214.64113.28315.998yes
1314.98113.69516.268yes
1413.71512.22215.208yes
1513.33310.52116.146yes
1614.24611.90516.586yes
1715.63113.47117.790yes
1817.25715.30519.209no
1914.46312.38516.541yes
2014.55712.64016.474yes
2116.58113.78119.382yes
2215.00913.03616.982yes
2313.94011.71816.163yes
2413.90911.46516.353yes
2515.09112.92517.257yes
2615.05612.97417.138yes
2715.07313.11617.029yes
2815.38212.39218.373yes
2915.22413.41617.033yes
3015.62413.76917.480yes
3116.36514.06218.668yes
3214.89813.29316.504yes
3315.69613.50617.886yes
3414.87013.04716.694yes
3514.81212.13817.487yes
3615.32112.98517.657yes
3714.22311.77216.674yes
3814.82113.15716.486yes
3914.31411.79016.838yes
4016.22714.41618.039yes
4115.38013.33117.429yes
4215.50313.38717.620yes
4314.20311.72016.686yes
4414.45112.37516.527yes
4513.64210.55016.735yes
4613.37811.18215.575yes
4716.89714.98618.808yes
4814.90912.80417.013yes
4914.92311.83718.010yes
5015.53911.97419.104yes
5114.88512.67617.094yes
5213.86611.73615.996yes
5314.70112.93616.467yes
5415.70713.69317.721yes
5514.92512.89116.958yes
5614.11011.80516.416yes
5714.77013.15216.388yes
5815.45012.80418.097yes
5916.00813.97318.043yes
6014.24212.00216.481yes
6115.43913.13317.744yes
6214.51411.96017.067yes
6315.78213.19318.370yes
6413.81212.10515.519yes
6513.22110.92415.518yes
6614.12511.27616.974yes
6714.70312.13117.275yes
6813.38111.44815.314yes
6914.86912.74416.994yes
7014.16811.73216.605yes
7114.95912.19817.719yes
7215.37112.55118.191yes
7314.98912.93117.047yes
7415.23713.47017.004yes
7514.96311.77918.146yes
7614.09111.23616.947yes
7715.60713.60417.610yes
7815.37813.21117.545yes
7914.90013.06616.735yes
8014.51211.92517.099yes
8114.05911.84316.276yes
8214.69511.71017.680yes
8315.34913.07517.622yes
8416.07613.58418.568yes
8515.62012.23219.008yes
8613.20311.38115.024yes
8715.23913.30417.174yes
8815.53413.94717.121yes
8915.82813.64818.008yes
9014.69912.45716.941yes
9114.42712.56916.285yes
9215.57313.32317.824yes
9315.55513.28517.825yes
9414.85712.04317.670yes
9516.55914.60518.512yes
9615.15513.11817.193yes
9713.76810.46617.071yes
9814.60711.68817.525yes
9915.59914.44816.750yes
10014.02112.46615.577yes

With 100 intervals the observed coverage is itself an estimate. The Wilson interval above shows how far it could plausibly sit from the true coverage. Appendix B runs this experiment 5,000 times per setting, for skewed data too.

Explain this resultoptional · your own key

Add your own Anthropic or OpenAI key in AI settings to enable. Nothing is sent without one.

§3.2Coursework A2 Q2(b)

How many walks are enough?

When σ is known the half-width of a z interval is E=z1−α/2 σ/nE = z_{1-\alpha/2}\,\sigma/\sqrt n. Solving for n gives the planning formula

n=⌈(z1−α/2 σE)2⌉.n = \left\lceil \left(\frac{z_{1-\alpha/2}\,\sigma}{E}\right)^{2} \right\rceil .

With σ = 3 minutes and a target of ±1 minute, (1.96 × 3)² ≈ 34.57, so 35 walks are needed.

3.0
1.00
95%

Critical value z

1.960

(zσ / E)²

34.57

Walks needed

35

round up to a whole sample

Figure 3.2.Required sample size against the target half-width for σ = 3.0. Halving the width quadruples the number of observations — the 1/√n law behind every interval in this chapter.

§3.3Coursework A2 Q2(c)–(e)

Two routes, two samples

For independent normal samples with a common variance, pool the two sample variances,

sp2=(nX−1)sX2+(nY−1)sY2nX+nY−2,s_p^2=\frac{(n_X-1)s_X^2+(n_Y-1)s_Y^2}{n_X+n_Y-2},

and the interval for the difference in means is

xˉ−yˉ±tnX+nY−2,  1−α/2  sp1nX+1nY.\bar x-\bar y \pm t_{n_X+n_Y-2,\;1-\alpha/2}\; s_p\sqrt{\tfrac{1}{n_X}+\tfrac{1}{n_Y}} .

The equal-variance assumption can itself be examined: SX2/σX2SY2/σY2∼FnX−1, nY−1\frac{S_X^2/\sigma_X^2}{S_Y^2/\sigma_Y^2}\sim F_{n_X-1,\,n_Y-1}, which inverts to an interval for σX2/σY2\sigma_X^2/\sigma_Y^2 — what var.test() reports.

10 values

8 values

95%

Defaults are simulated (X ~ N(15, 3²), Y ~ N(20.3, 0.8²)) to mimic the shape of the assignment data. Paste your own values to analyse them.

95% CI for μX − μY

(-6.84, -1.85)

pooled sp = 2.486, df = 16

F = s²X / s²Y

9.881

df = (9, 7)

95% CI for σ²X / σ²Y

(2.049, 41.47)

F test p-value

0.00638

variances differ

  • Sample X
  • Sample Y
  • Mean with its t interval
X (route A)Y (route B)
Figure 3.3.Each dot is one walk. The bar under each row is that sample’s own 95% t interval for its mean. The pooled interval for the difference assumes equal variances — the F test on the right checks how plausible that is, and here it is doubtful.
Show the numbers as a table
samplenmeansdt interval
X (route A)1016.2803.191(13.997, 18.563)
Y (route B)820.6251.015(19.776, 21.474)

§3.4Coursework A2 Q1

Do two councils differ?

For large samples the difference of sample proportions is approximately normal. The test of  H0:p1=p2  \;H_0: p_1 = p_2\; uses the pooled proportion in its standard error, and its square is the chi-squared statistic that prop.test() prints; the interval uses each group’s own proportion:

p^1−p^2±z1−α/2 se^,\hat p_1-\hat p_2 \pm z_{1-\alpha/2}\,\widehat{\mathrm{se}},
se^=p^1(1−p^1)n1+p^2(1−p^2)n2.\widehat{\mathrm{se}}=\sqrt{\frac{\hat p_1(1-\hat p_1)}{n_1}+\frac{\hat p_2(1-\hat p_2)}{n_2}} .
95%

Defaults are the survey counts from the assignment (support for more bicycle lanes in two inner-Melbourne councils). Equivalent to prop.test(…, correct = FALSE).

p̂₁ − p̂₂

0.0500

0.650 vs 0.600

X² (= z²)

4.7269

z = 2.174

p-value

0.02969

reject H₀ at α = 0.05

95% CI for p₁ − p₂

(0.0051, 0.0949)

  • Estimate and interval for p₁ − p₂
  • No difference (H₀)
Figure 3.4.Here the test and the interval tell the same story: the interval excludes zero and the two-sided test rejects at the matching level. (The test pools the proportions for its standard error; the interval does not, so the two can disagree near the boundary.)