How to use this sheet

Full worked answers are in the solutions sheet.

Five exercises on Lectures 4 to 7. For the pen-and-paper ones, work them out with a calculator first, as you would in a quiz or the midterm, and only then check your numbers in R.

Exercise 1. Old Faithful: reading R output

The dataset faithful, which ships with R, records 272 eruptions of the Old Faithful geyser in Yellowstone: the duration of each eruption, in minutes, and the waiting time until the next eruption, also in minutes. Durations range from 1.6 to 5.1 minutes. We regress waiting time on duration. Two numbers in the output have been replaced by xxxx.

Call:
lm(formula = waiting ~ eruptions, data = faithful)

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept)  33.4744     1.1549   28.98   <2e-16 ***
eruptions    10.7296     0.3148    xxxx   <2e-16 ***

Residual standard error: 5.914 on 270 degrees of freedom
Multiple R-squared:  0.8115,    Adjusted R-squared:  0.8108 
F-statistic:  xxxx on 1 and 270 DF,  p-value: < 2.2e-16

You may use \(t(0.975;\,270) = 1.969\) and \(t(0.995;\,270) = 2.594\).

Questions.

  1. Find the missing \(t\) value for eruptions, and the missing \(F\) statistic.
  2. Interpret the estimated slope in the context of these data.
  3. A park ranger claims that each additional minute of eruption adds 10 minutes to the mean waiting time. Test this claim at the 5% level. Would your conclusion change at the 1% level?
  4. Give a 95% confidence interval for \(\beta_1\), and explain how it agrees with your answer to (c).
  5. What is the correlation between eruption duration and waiting time? What share of the variability in waiting times does the regression account for?
  6. Using the residual standard error and the standard error of the slope, find \(S_{xx}\), and then the sample variance of the eruption durations. Check your answer with var(faithful$eruptions).

To see the full output once you are done:

summary(lm(waiting ~ eruptions, data = faithful))

Exercise 2. Iced drinks: intervals by hand

A campus coffee shop records, over 20 days in May and June, the day’s maximum temperature \(x\) (in °C) and the number of iced drinks sold \(Y\). The temperatures range from 12.0 to 28.4 °C. Assume the simple linear regression model with normal errors.

\(n\) \(\sum x_i\) \(\sum Y_i\) \(S_{xx}\) \(S_{YY}\) \(S_{xY}\)
20 384.3 2726 538.5 50082.2 4769.9

You may use \(t(0.975;\,18) = 2.101\).

Questions.

  1. Find the fitted regression line, and interpret \(\hat\beta_1\).
  2. Interpret \(\hat\beta_0\) literally, then explain why these data cannot support that reading.
  3. Compute \(SSE\), using \(SSE = S_{YY} - S_{xY}^2/S_{xx}\), and then \(\hat\sigma\).
  4. Give a 95% confidence interval for the mean number of iced drinks sold on days with a maximum temperature of 25 °C.
  5. Tomorrow’s forecast is 25 °C. Give a 95% prediction interval for tomorrow’s sales. Why is it so much wider than the interval in (d)?
  6. A heat wave of 38 °C is forecast next week, and the manager asks how many drinks to prepare. Compute the point prediction, and explain why you would not trust it, even with a prediction interval around it.

Exercise 3. Tree volumes: an ANOVA table with gaps

The dataset trees records the girth (in inches) and the volume (in cubic feet) of black cherry trees. A regression of volume on girth gives an estimated slope of \(\hat\beta_1 = 5.066\) and the following ANOVA table, with some entries missing.

Source Df Sum of squares Mean square \(F\)
Regression ? 7581.8 ? ?
Error 29 ? ?
Total ? 8106.1

You may use \(F(0.99;\,1,\,29) = 7.60\).

Questions.

  1. Complete the table. How many trees are in the dataset?
  2. Compute \(R^2\) and the correlation \(r\) between girth and volume.
  3. Test \(H_0: \beta_1 = 0\) against \(H_a: \beta_1 \neq 0\) at the 1% level, using the \(F\) test.
  4. Find \(|t^*|\), the absolute value of the \(t\) statistic for the same test, and deduce the standard error of \(\hat\beta_1\).
  5. Find \(\hat\sigma\).

To check:

fit <- lm(Volume ~ Girth, data = trees)
anova(fit)
summary(fit)

Exercise 4. Why the ANOVA works

Consider the simple linear regression model under the Gauss-Markov conditions, with residuals \(e_i = Y_i - \hat Y_i\).

Questions.

  1. Using the normal equations \(\sum_i e_i = 0\) and \(\sum_i x_i e_i = 0\), show that \(\sum_i (\hat Y_i - \bar Y)\, e_i = 0\). Deduce that \(SST = SSR + SSE\).
  2. Recall from Lecture 6 that \(SSR = \hat\beta_1^2 S_{xx}\). Using \(E(\hat\beta_1) = \beta_1\) and \(\mathrm{Var}(\hat\beta_1) = \sigma^2 / S_{xx}\), show that \[E(SSR) = \sigma^2 + \beta_1^2 S_{xx}.\] Hint: for any random variable \(Z\), \(E(Z^2) = \mathrm{Var}(Z) + E(Z)^2\).
  3. We know that \(E(MSE) = \sigma^2\). Compare \(E(MSR)\) with \(E(MSE)\) when \(\beta_1 = 0\) and when \(\beta_1 \neq 0\). Why are large values of \(F^* = MSR/MSE\) evidence against \(H_0: \beta_1 = 0\)? What role does \(S_{xx}\) play?
  4. At \(x_{n+1} = \bar x\), write down the half-widths of the 95% confidence interval for \(E(Y \mid x_{n+1})\) and of the 95% prediction interval for \(Y_{n+1}\). What happens to each as \(n \to \infty\)? Explain in one sentence why they behave differently.

Exercise 5. What “95%” means, and when it stops meaning it

The code below simulates 2000 datasets from a known model with \(\beta_1 = 0.5\), fits a regression to each, and records how often the 95% confidence interval for \(\beta_1\) contains the true value. The design, the \(x\) values, stays fixed. Run it yourself: the answers depend on it.

set.seed(67)
n <- 20
x <- runif(n, 1, 10)                 # the same x values in every dataset
beta0 <- 2; beta1 <- 0.5

covers <- function(make_errors, copies = 1, reps = 2000) {
  mean(replicate(reps, {
    y  <- beta0 + beta1 * x + make_errors(x)
    xx <- rep(x, copies); yy <- rep(y, copies)   # copies > 1: the same units measured again
    ci <- confint(lm(yy ~ xx), "xx", level = 0.95)
    ci[1] <= beta1 && beta1 <= ci[2]
  }))
}

normal <- function(x) rnorm(length(x), 0, 1)             # the model is true
funnel <- function(x) rnorm(length(x), 0, 0.25 * x)      # the spread grows with x
skewed <- function(x) rexp(length(x), 1) - 1             # mean 0, sd 1, but skewed

covers(normal)

Questions.

  1. Report the coverage from covers(normal). What exactly is this number an estimate of, and why is it not exactly 0.95?
  2. Run covers(normal, copies = 3): each of the 20 units is now measured three times, as in the xkcd cartoon of Lecture 5. What happens to the coverage? Which assumption fails? Try copies = 10.
  3. Run covers(funnel). What happens, and which assumption fails?
  4. Run covers(skewed). What happens, and which assumption fails? Comparing (b), (c) and (d), which assumption does the confidence interval for \(\beta_1\) depend on most, and which one least?