Matrix Problem Set

Instructions

Complete all problems showing your work. For matrix calculations, show each step clearly. You may use R to verify your answers, but show the mathematical steps first.

Problem 1

Given the following matrices:

\[\mathbf{A} = \begin{bmatrix} 1 & 3 & -2 \\ 4 & 0 & 5 \end{bmatrix}, \quad \mathbf{B} = \begin{bmatrix} 2 & 1 \\ -1 & 3 \\ 0 & 4 \end{bmatrix}, \quad \mathbf{C} = \begin{bmatrix} 3 & -1 \\ 2 & 6 \end{bmatrix}\]

a) What are the dimensions of each matrix?

b) Which of the following operations are possible? If possible, state the dimensions of the result:

• \(\mathbf{A} + \mathbf{B}\)

• \(\mathbf{A} \times \mathbf{B}\)

• \(\mathbf{B} \times \mathbf{C}\)

• \(\mathbf{C} \times \mathbf{B}\)

c) Calculate \(\mathbf{A} \times \mathbf{B}\) (show all steps).

Problem 2

Given \(\mathbf{P} = \begin{bmatrix} 2 & -1 & 4 \\ 3 & 0 & 1 \end{bmatrix}\) and \(\mathbf{Q} = \begin{bmatrix} 1 & 2 \\ -1 & 3 \\ 0 & 1 \end{bmatrix}\)

a) Find \(\mathbf{P}^T\) and \(\mathbf{Q}^T\).

b) Calculate \(\mathbf{P} \times \mathbf{Q}\).

c) Calculate \((\mathbf{P} \times \mathbf{Q})^T\).

d) Calculate \(\mathbf{Q}^T \times \mathbf{P}^T\) and verify that \((\mathbf{P} \times \mathbf{Q})^T = \mathbf{Q}^T \times \mathbf{P}^T\).

Problem 3

a) Find the inverse of \(\mathbf{R} = \begin{bmatrix} 3 & 1 \\ 2 & 1 \end{bmatrix}\).

b) Verify your answer by showing that \(\mathbf{R} \times \mathbf{R}^{-1} = \mathbf{I}\).

c) Explain why the matrix \(\mathbf{S} = \begin{bmatrix} 2 & 4 \\ 1 & 2 \end{bmatrix}\) does not have an inverse.

Problem 4

a) Find \(\frac{\partial}{\partial \mathbf{x}}(\mathbf{a}^T\mathbf{x})\) where \(\mathbf{a} = \begin{bmatrix} 3 \\ -2 \\ 1 \\ 4 \end{bmatrix}\) and \(\mathbf{x} = \begin{bmatrix} x_1 \\ x_2 \\ x_3 \\ x_4 \end{bmatrix}\).

b) If \(f(\mathbf{x}) = 2x_1 - 5x_2 + 3x_3\), write this in the form \(\mathbf{c}^T\mathbf{x}\) and find \(\frac{\partial f}{\partial \mathbf{x}}\).

Problem 5

Consider the quadratic form \(\mathbf{x}^T\mathbf{A}\mathbf{x}\) where \(\mathbf{A} = \begin{bmatrix} 2 & 3 \\ 1 & 4 \end{bmatrix}\) and \(\mathbf{x} = \begin{bmatrix} x_1 \\ x_2 \end{bmatrix}\).

a) Expand \(\mathbf{x}^T\mathbf{A}\mathbf{x}\) into scalar form.

b) Find \(\mathbf{A}^T\) and calculate \(\mathbf{A} + \mathbf{A}^T\).

c) Use the derivative rule to find \(\frac{\partial}{\partial \mathbf{x}}(\mathbf{x}^T\mathbf{A}\mathbf{x})\).

Problem 6

Let \(\mathbf{B} = \begin{bmatrix} 4 & -1 & 2 \\ -1 & 3 & 0 \\ 2 & 0 & 5 \end{bmatrix}\) and \(\mathbf{x} = \begin{bmatrix} x_1 \\ x_2 \\ x_3 \end{bmatrix}\).

a) Verify that \(\mathbf{B}\) is symmetric.

b) Find \(\frac{\partial}{\partial \mathbf{x}}(\mathbf{x}^T\mathbf{B}\mathbf{x})\) using the symmetric matrix rule.

c) What would be the result if we used the general rule \((\mathbf{A} + \mathbf{A}^T)\mathbf{x}\) instead? Show that both methods give the same answer.

Problem 7

Consider the expression \(g(\mathbf{x}) = \mathbf{b}^T\mathbf{A}\mathbf{x}\) where: \[\mathbf{b} = \begin{bmatrix} 2 \\ -1 \\ 3 \end{bmatrix}, \quad \mathbf{A} = \begin{bmatrix} 1 & 0 & 2 \\ 3 & -1 & 1 \\ 0 & 2 & 4 \end{bmatrix}, \quad \mathbf{x} = \begin{bmatrix} x_1 \\ x_2 \\ x_3 \end{bmatrix}\]

a) What is the dimension of \(g(\mathbf{x})\)? (Is it a scalar, vector, or matrix?)

b) Use the rule \(\frac{\partial}{\partial \mathbf{x}}(\mathbf{b}^T\mathbf{A}\mathbf{x}) = \mathbf{A}^T\mathbf{b}\) to find \(\frac{\partial g}{\partial \mathbf{x}}\).

c) Calculate \(\mathbf{A}\mathbf{x}\) first, then compute \(\mathbf{b}^T(\mathbf{A}\mathbf{x})\) to expand \(g(\mathbf{x})\) into scalar form.

d) Verify your answer from part (b) by taking partial derivatives of the scalar form from part (c).

Bonus Challenge

Hat Matrix Connection: In linear regression, we minimize the sum of squared errors: \[SSE = (\mathbf{y} - \mathbf{X}\boldsymbol{\beta})^T(\mathbf{y} - \mathbf{X}\boldsymbol{\beta})\]

Expand this expression and identify which derivative rules from this problem set you would need to find \(\frac{\partial SSE}{\partial \boldsymbol{\beta}}\). You don’t need to solve it completely, just identify the relevant derivative rules and explain how they would be applied.