It’s Just a Linear Model

Background Reading: Lindeløv, J. K. “Common statistical tests are linear models”
URL: https://lindeloev.github.io/tests-as-linear/

Instructions

Read through Lindeløv’s resource to understand how common statistical tests are special cases of linear models. Then complete the matrix-based exercises below using the provided data. I’ll do the first one (one-sample t-test) so you can see what I mean.

Question 1: t-tests

a) One-sample t-test

Data: \(\mathbf{y} = \begin{bmatrix} 2.1 \\ 1.8 \\ 2.3 \\ 1.9 \\ 2.0 \end{bmatrix}\)

  • Construct design matrix \(\mathbf{X}\) and \(\mathbf{y}\) such that \(\boldsymbol{\beta}\) gives the one-sample t-test result
  • Compare your matrix setup to t.test(y) in R
X <- c(1, 1, 1, 1, 1)
y <- c(2.1, 1.8, 2.3, 1.9, 2.0)

## these should give me the same answer
solve(crossprod(X)) %*% crossprod(X, y)
     [,1]
[1,] 2.02

    One Sample t-test

data:  y
t = 23.482, df = 4, p-value = 1.95e-05
alternative hypothesis: true mean is not equal to 0
95 percent confidence interval:
 1.781161 2.258839
sample estimates:
mean of x 
     2.02

b) Independent two-sample t-test

  • Group 1: \(\begin{bmatrix} 1.2 \\ 1.5 \\ 1.1 \\ 1.4 \end{bmatrix}\)
  • Group 2: \(\begin{bmatrix} 2.1 \\ 2.3 \\ 2.0 \end{bmatrix}\)

Combined: \(\mathbf{y} = \begin{bmatrix} 1.2 \\ 1.5 \\ 1.1 \\ 1.4 \\ 2.1 \\ 2.3 \\ 2.0 \end{bmatrix}\)

  • Construct design matrix \(\mathbf{X}\) (using dummy coding) and \(\mathbf{y}\) such that \(\boldsymbol{\beta}\) gives the two-sample t-test result
  • Compare to t.test(group1, group2, var.equal = TRUE) in R

c) Paired t-test

  • Before: \(\begin{bmatrix} 3.2 \\ 2.8 \\ 3.1 \\ 2.9 \end{bmatrix}\), After: \(\begin{bmatrix} 3.0 \\ 2.5 \\ 2.8 \\ 2.6 \end{bmatrix}\)

Differences: \(\mathbf{y} = \begin{bmatrix} -0.2 \\ -0.3 \\ -0.3 \\ -0.3 \end{bmatrix}\)

  • Construct design matrix \(\mathbf{X}\) and \(\mathbf{y}\) such that \(\boldsymbol{\beta}\) gives the paired t-test result
  • Compare to t.test(before, after, paired = TRUE) in R

Question 2: ANOVA

a) One-way ANOVA

  • Group A: \(\begin{bmatrix} 1.1 \\ 1.3 \\ 1.2 \end{bmatrix}\)
  • Group B: \(\begin{bmatrix} 2.0 \\ 2.2 \\ 1.9 \\ 2.1 \end{bmatrix}\)
  • Group C: \(\begin{bmatrix} 2.8 \\ 2.9 \end{bmatrix}\)

Combined: \(\mathbf{y} = \begin{bmatrix} 1.1 \\ 1.3 \\ 1.2 \\ 2.0 \\ 2.2 \\ 1.9 \\ 2.1 \\ 2.8 \\ 2.9 \end{bmatrix}\)

  • Construct design matrix \(\mathbf{X}\) (using dummy coding) and \(\mathbf{y}\) such that \(\boldsymbol{\beta}\) gives the one-way ANOVA result
  • Compare to aov(y ~ group) in R

b) Two-way ANOVA (2×2 design)

Factors: A (Low/High), B (Control/Treatment)

  • Low-Control: \(\begin{bmatrix} 1.0 \\ 1.2 \end{bmatrix}\)
  • Low-Treatment: \(\begin{bmatrix} 1.5 \\ 1.7 \end{bmatrix}\)
  • High-Control: \(\begin{bmatrix} 2.0 \\ 2.1 \end{bmatrix}\)
  • High-Treatment: \(\begin{bmatrix} 3.0 \\ 3.2 \end{bmatrix}\)

Combined: \(\mathbf{y} = \begin{bmatrix} 1.0 \\ 1.2 \\ 1.5 \\ 1.7 \\ 2.0 \\ 2.1 \\ 3.0 \\ 3.2 \end{bmatrix}\)

  • Construct \(\mathbf{X}\) with main effects and interaction and \(\mathbf{y}\) such that \(\boldsymbol{\beta}\) gives the two-way ANOVA result
  • Compare to aov(y ~ A * B) in R

Question 3: Correlation/Regression

a) Simple linear regression

\(\mathbf{x} = \begin{bmatrix} 1 \\ 2 \\ 3 \\ 4 \end{bmatrix}\), \(\mathbf{y} = \begin{bmatrix} 2.1 \\ 3.9 \\ 6.1 \\ 7.8 \end{bmatrix}\)

  • Construct design matrix \(\mathbf{X}\) and \(\mathbf{y}\) to calculate Pearson Correlation
  • Compare to cor.test(x, y) in R

b) Spearman correlation - Using same data as (a), construct \(\mathbf{X}\) and \(\mathbf{y}\) for rank-based analysis - Compare to cor.test(x, y, method = "spearman") in R