Orthogonal Projection onto a Subspace

The orthogonal projection of a vector y onto a subspace W is the vector ŷ in W closest to y. With an orthogonal basis u₁, …, uₚ of W, ŷ = Σ (y · uᵢ)/(uᵢ · uᵢ) uᵢ. The leftover z = y − ŷ is orthogonal to W, and the distance from y to W is ‖y − ŷ‖.

Before this lesson: Orthogonal Sets and Orthonormal Bases, Vector Projections and Subspaces

What an orthogonal projection does

Picture a plane W through the origin and a point y above it. Drop a perpendicular from y to the plane. Where it lands is ŷ, the orthogonal projection, and the perpendicular piece z = y − ŷ is the part of y that W cannot reach.

Projecting onto a line spanned by one vector u is the vector projection you already know. Projecting onto a bigger subspace just adds one such piece for each vector in an orthogonal basis.

A plane W drawn as a slanted parallelogram through the origin O. A vector y rises from O above the plane. Its projection y-hat lies in the plane directly below the tip of y. A dashed segment z = y − y-hat runs straight down from the tip of y to the tip of y-hat and meets the plane at a right angle. The length of z is labeled as the distance from y to W. — y splits into ŷ, which lies in W, plus z = y − ŷ, which is perpendicular to W. The length of z is the distance from y to W.

The orthogonal decomposition theorem

Let W be a subspace of ℝⁿ with an orthogonal basis {u₁, …, uₚ}. Every y in ℝⁿ can be written in exactly one way as y = ŷ + z, with ŷ in W and z orthogonal to every vector in W, where

\hat{y} = \operatorname{proj}_W y = \frac{y \cdot u_1}{u_1 \cdot u_1}\,u_1 + \frac{y \cdot u_2}{u_2 \cdot u_2}\,u_2 + \cdots + \frac{y \cdot u_p}{u_p \cdot u_p}\,u_p

Best approximation: ŷ is the closest point in W to y. For every other w in W, ‖y − ŷ‖ is less than ‖y − w‖.
The distance from y to W is ‖z‖ = ‖y − ŷ‖.
If y is already in W, then ŷ = y and the distance is 0.
The basis must be orthogonal. With a non-orthogonal basis, the formula gives the wrong vector.
Projection matrix: if the columns of U are an orthonormal basis of W, then proj_W y = UUᵀy. The matrix P = UUᵀ satisfies P² = P.

How to find an orthogonal projection

Get an orthogonal basis for W. Check that every pair of basis vectors has dot product 0.
For each basis vector, compute the weight (y · uᵢ)/(uᵢ · uᵢ).
Add the weighted basis vectors to get ŷ.
Find z = y − ŷ, and check that z · uᵢ = 0 for every basis vector.
If you need the distance, compute ‖z‖.

Orthogonal projection examples

Example 1: Projection onto a line

Find the projection of $y = \begin{bmatrix} 2 \\ 3 \\ 7 \end{bmatrix}$ onto the line W spanned by $u = \begin{bmatrix} 1 \\ 1 \\ 1 \end{bmatrix}$ , and the distance from y to W.

y \cdot u = 2 + 3 + 7 = 12,\quad u \cdot u = 1 + 1 + 1 = 3

Dot products (see dot product).

\hat{y} = \frac{12}{3}\,u = 4\begin{bmatrix} 1 \\ 1 \\ 1 \end{bmatrix} = \begin{bmatrix} 4 \\ 4 \\ 4 \end{bmatrix}

Weight times u.

z = y - \hat{y} = \begin{bmatrix} 2 - 4 \\ 3 - 4 \\ 7 - 4 \end{bmatrix} = \begin{bmatrix} -2 \\ -1 \\ 3 \end{bmatrix}

Subtract to get the perpendicular part.

z \cdot u = -2 - 1 + 3 = 0

Check: z is orthogonal to the line.

\|z\| = \sqrt{4 + 1 + 9} = \sqrt{14}

Length of z.

Answer: ŷ = (4, 4, 4), and the distance from y to the line is √14 ≈ 3.74.

Example 2: Projection onto a plane

Let W be the plane spanned by $u_1 = \begin{bmatrix} 1 \\ 1 \\ 0 \end{bmatrix}$ and $u_2 = \begin{bmatrix} 1 \\ -1 \\ 2 \end{bmatrix}$ . Write $y = \begin{bmatrix} 6 \\ 0 \\ 3 \end{bmatrix}$ as a vector in W plus a vector orthogonal to W.

u_1 \cdot u_2 = 1 - 1 + 0 = 0

The basis is orthogonal, so the formula applies.

\frac{y \cdot u_1}{u_1 \cdot u_1} = \frac{6 + 0 + 0}{1 + 1 + 0} = \frac{6}{2} = 3

Weight on u₁.

\frac{y \cdot u_2}{u_2 \cdot u_2} = \frac{6 - 0 + 6}{1 + 1 + 4} = \frac{12}{6} = 2

Weight on u₂.

\hat{y} = 3\begin{bmatrix} 1 \\ 1 \\ 0 \end{bmatrix} + 2\begin{bmatrix} 1 \\ -1 \\ 2 \end{bmatrix} = \begin{bmatrix} 3 + 2 \\ 3 - 2 \\ 0 + 4 \end{bmatrix} = \begin{bmatrix} 5 \\ 1 \\ 4 \end{bmatrix}

Add the weighted basis vectors.

z = y - \hat{y} = \begin{bmatrix} 6 - 5 \\ 0 - 1 \\ 3 - 4 \end{bmatrix} = \begin{bmatrix} 1 \\ -1 \\ -1 \end{bmatrix}

Subtract.

z \cdot u_1 = 1 - 1 + 0 = 0,\quad z \cdot u_2 = 1 + 1 - 2 = 0

Check: z is orthogonal to both basis vectors, so to all of W.

Answer: y = (5, 1, 4) + (1, −1, −1), with (5, 1, 4) in W and (1, −1, −1) orthogonal to W.

Example 3: Distance from a vector to a subspace in ℝ⁴

Find the closest point to $y = \begin{bmatrix} 3 \\ 1 \\ 5 \\ -1 \end{bmatrix}$ in W = span{u₁, u₂}, where $u_1 = \begin{bmatrix} 1 \\ 1 \\ 1 \\ 1 \end{bmatrix}$ and $u_2 = \begin{bmatrix} 1 \\ -1 \\ 1 \\ -1 \end{bmatrix}$ , and the distance from y to W.

u_1 \cdot u_2 = 1 - 1 + 1 - 1 = 0

The basis is orthogonal.

\frac{y \cdot u_1}{u_1 \cdot u_1} = \frac{3 + 1 + 5 - 1}{4} = \frac{8}{4} = 2

Weight on u₁.

\frac{y \cdot u_2}{u_2 \cdot u_2} = \frac{3 - 1 + 5 + 1}{4} = \frac{8}{4} = 2

Weight on u₂.

\hat{y} = 2u_1 + 2u_2 = \begin{bmatrix} 2 + 2 \\ 2 - 2 \\ 2 + 2 \\ 2 - 2 \end{bmatrix} = \begin{bmatrix} 4 \\ 0 \\ 4 \\ 0 \end{bmatrix}

Closest point in W.

z = y - \hat{y} = \begin{bmatrix} -1 \\ 1 \\ 1 \\ -1 \end{bmatrix}

Subtract entry by entry.

z \cdot u_1 = -1 + 1 + 1 - 1 = 0,\quad z \cdot u_2 = -1 - 1 + 1 + 1 = 0

Check orthogonality.

\|z\| = \sqrt{1 + 1 + 1 + 1} = \sqrt{4} = 2

Distance.

Answer: the closest point is (4, 0, 4, 0), and the distance from y to W is 2.

Example 4: A projection matrix

The columns of $U = \begin{bmatrix} 1/\sqrt{2} & 0 \\ 1/\sqrt{2} & 0 \\ 0 & 1 \end{bmatrix}$ are an orthonormal basis for a plane W. Find the projection matrix P = UUᵀ and use it to project $y = \begin{bmatrix} 4 \\ 2 \\ 5 \end{bmatrix}$ .

P = UU^T = \begin{bmatrix} \tfrac{1}{2} + 0 & \tfrac{1}{2} + 0 & 0 + 0 \\ \tfrac{1}{2} + 0 & \tfrac{1}{2} + 0 & 0 + 0 \\ 0 + 0 & 0 + 0 & 0 + 1 \end{bmatrix}

Multiply U by its transpose (see transpose of a matrix).

= \begin{bmatrix} 0.5 & 0.5 & 0 \\ 0.5 & 0.5 & 0 \\ 0 & 0 & 1 \end{bmatrix}

Simplify.

Py = \begin{bmatrix} 2 + 1 + 0 \\ 2 + 1 + 0 \\ 0 + 0 + 5 \end{bmatrix} = \begin{bmatrix} 3 \\ 3 \\ 5 \end{bmatrix}

Multiply P by y (see matrix multiplication).

y - Py = \begin{bmatrix} 1 \\ -1 \\ 0 \end{bmatrix}

The leftover part.

\begin{bmatrix} 1 \\ -1 \\ 0 \end{bmatrix} \cdot \begin{bmatrix} 1/\sqrt{2} \\ 1/\sqrt{2} \\ 0 \end{bmatrix} = 0,\quad \begin{bmatrix} 1 \\ -1 \\ 0 \end{bmatrix} \cdot \begin{bmatrix} 0 \\ 0 \\ 1 \end{bmatrix} = 0

Check: the leftover is orthogonal to both columns of U.

Answer: P = [0.5 0.5 0; 0.5 0.5 0; 0 0 1] and proj_W y = (3, 3, 5). P averages the first two entries and keeps the third, which is projection onto the plane x₁ = x₂.

Example 5: Best straight-line trend in weekly sales

A shop sold y = (10, 14, 15, 21) units over four weeks. The analyst wants the closest sales pattern of the form “steady level plus straight-line trend.” Those patterns form the subspace W spanned by the orthogonal vectors $u_1 = (1, 1, 1, 1)$ (level) and $u_2 = (-3, -1, 1, 3)$ (trend). Find the best pattern ŷ and how far y is from it.

u_1 \cdot u_2 = -3 - 1 + 1 + 3 = 0

The two patterns are orthogonal.

\frac{y \cdot u_1}{u_1 \cdot u_1} = \frac{10 + 14 + 15 + 21}{4} = \frac{60}{4} = 15

Weight on the level.

y \cdot u_2 = -30 - 14 + 15 + 63 = 34

Dot product with the trend.

u_2 \cdot u_2 = 9 + 1 + 1 + 9 = 20

Squared length of the trend.

\frac{34}{20} = 1.7

Weight on the trend.

\hat{y} = 15\begin{bmatrix} 1 \\ 1 \\ 1 \\ 1 \end{bmatrix} + 1.7\begin{bmatrix} -3 \\ -1 \\ 1 \\ 3 \end{bmatrix} = \begin{bmatrix} 15 - 5.1 \\ 15 - 1.7 \\ 15 + 1.7 \\ 15 + 5.1 \end{bmatrix} = \begin{bmatrix} 9.9 \\ 13.3 \\ 16.7 \\ 20.1 \end{bmatrix}

Add the weighted patterns.

z = y - \hat{y} = \begin{bmatrix} 0.1 \\ 0.7 \\ -1.7 \\ 0.9 \end{bmatrix}

Subtract.

z \cdot u_1 = 0.1 + 0.7 - 1.7 + 0.9 = 0

Check against the level.

z \cdot u_2 = -0.3 - 0.7 - 1.7 + 2.7 = 0

Check against the trend.

\|z\| = \sqrt{0.01 + 0.49 + 2.89 + 0.81} = \sqrt{4.2} \approx 2.05

Distance from y to W.

Answer: the best pattern is (9.9, 13.3, 16.7, 20.1): sales grow by about 3.4 units a week (2 × 1.7, since u₂ steps by 2) around an average of 15. The actual sales are about 2.05 units away from this pattern. This is the same line that the least-squares regression line gives.

Common mistakes

Mistake	Example of the error	Fix
Using a basis that is not orthogonal	Projecting y = (2, 3, 4) onto span{(1, 1, 0), (1, 0, 0)} with the formula and getting (4.5, 2.5, 0)	First get an orthogonal basis of W. Here W is the x₁x₂-plane, so ŷ = (2, 3, 0).
Forgetting to divide by uᵢ · uᵢ	Writing ŷ = (y · u)u = 12(1, 1, 1) in Example 1	The weight is (y · u)/(u · u) = 12/3 = 4. Only unit vectors skip the division.
Giving ŷ as the distance	Saying the distance in Example 3 is ‖ŷ‖ = √32	The distance is the length of the leftover part, ‖y − ŷ‖ = 2.
Skipping the check	Reporting ŷ without testing z · uᵢ = 0	If some z · uᵢ is not 0, a weight or a subtraction is wrong.
Using UUᵀ with orthogonal but not unit columns	Using U with columns (1, 1, 0) and (0, 0, 1) and getting P = [1 1 0; 1 1 0; 0 0 1]	The formula P = UUᵀ needs orthonormal columns. Normalize first.

Related ideas

Orthogonal complement. The set of all vectors orthogonal to W is a subspace called W⊥. The theorem above says ℝⁿ splits as W plus W⊥: ŷ is in W and z is in W⊥.

Least squares. When Ax = b has no solution, the best you can do is solve Ax = proj b, projecting b onto the column space of A. Least-Squares Solutions does this with the normal equations, and Example 5 is a small preview.

Getting an orthogonal basis. The formula needs an orthogonal basis. The next lesson, Gram-Schmidt Process, builds one from any basis using these same projections.

Practice problems

Project y = (4, 7) onto the line spanned by u = (2, 1).
Project y = (1, 5, 3) onto the line spanned by u = (1, 1, 1).
Find the distance from y = (1, 5, 3) to the line in Problem 2.
Project y = (2, 5, 4) onto W = span{(1, 0, 1), (1, 0, −1)}.
Project y = (3, 3, 6) onto the plane W from Example 2, and find the distance from y to W.
Check that z = y − ŷ from Problem 5 is orthogonal to u₁ and u₂.
Find the projection matrix onto the line spanned by the unit vector u = (0.6, 0.8), and use it to project y = (25, 0).
Project y = (3, −1, 4) onto the plane W from Example 2. What does the answer tell you?
Find the projection of y = (2, 3, 4) onto W = span{(1, 1, 0), (1, 0, 0)}.
Weekly sales were y = (8, 9, 13, 14). Use the level and trend patterns from Example 5 to find the best straight-line pattern ŷ.

Show answers

\frac{y \cdot u}{u \cdot u} = \frac{8 + 7}{4 + 1} = \frac{15}{5} = 3

Weight on u.

\hat{y} = 3(2, 1) = (6, 3)

Multiply.

Answer: ŷ = (6, 3). The leftover (−2, 4) is orthogonal to u.

\frac{y \cdot u}{u \cdot u} = \frac{1 + 5 + 3}{3} = \frac{9}{3} = 3

Weight on u.

\hat{y} = 3(1, 1, 1) = (3, 3, 3)

Multiply.

Answer: ŷ = (3, 3, 3).

z = (1 - 3, 5 - 3, 3 - 3) = (-2, 2, 0)

Subtract ŷ from y.

\|z\| = \sqrt{4 + 4 + 0} = \sqrt{8} = 2\sqrt{2}

Length of z (see simplifying square roots).

Answer: the distance is 2√2 ≈ 2.83.

(1, 0, 1) \cdot (1, 0, -1) = 1 + 0 - 1 = 0

The basis is orthogonal.

\frac{2 + 0 + 4}{2} = 3,\quad \frac{2 + 0 - 4}{2} = -1

The two weights.

\hat{y} = 3(1, 0, 1) - (1, 0, -1) = (2, 0, 4)

Add the weighted basis vectors.

Answer: ŷ = (2, 0, 4). W is the x₁x₃-plane, so the projection just zeroes the middle entry.

\frac{3 + 3 + 0}{2} = 3,\quad \frac{3 - 3 + 12}{6} = 2

Weights on u₁ and u₂.

\hat{y} = 3(1, 1, 0) + 2(1, -1, 2) = (5, 1, 4)

Add the weighted basis vectors.

z = (3 - 5, 3 - 1, 6 - 4) = (-2, 2, 2)

Subtract.

\|z\| = \sqrt{4 + 4 + 4} = \sqrt{12} = 2\sqrt{3}

Length of z.

Answer: ŷ = (5, 1, 4), and the distance from y to W is 2√3 ≈ 3.46.

z \cdot u_1 = -2 + 2 + 0 = 0

Dot with u₁ = (1, 1, 0).

z \cdot u_2 = -2 - 2 + 4 = 0

Dot with u₂ = (1, −1, 2).

Answer: both dot products are 0, so z is orthogonal to W.

P = uu^T = \begin{bmatrix} 0.36 & 0.48 \\ 0.48 & 0.64 \end{bmatrix}

Entry (i, j) is uᵢuⱼ.

Py = \begin{bmatrix} 0.36(25) + 0.48(0) \\ 0.48(25) + 0.64(0) \end{bmatrix} = \begin{bmatrix} 9 \\ 12 \end{bmatrix}

Multiply P by y.

Answer: P = [0.36 0.48; 0.48 0.64], and Py = (9, 12).

\frac{3 - 1 + 0}{2} = 1,\quad \frac{3 + 1 + 8}{6} = 2

Weights on u₁ and u₂.

\hat{y} = (1, 1, 0) + 2(1, -1, 2) = (3, -1, 4)

Add the weighted basis vectors.

Answer: ŷ = y = (3, −1, 4). The vector is already in W, so its distance to W is 0.

(1, 1, 0) \cdot (1, 0, 0) = 1 \ne 0

This basis is not orthogonal, so the formula does not apply to it.

W = \text{span}\{(1, 0, 0), (0, 1, 0)\}

The same plane has the orthogonal basis e₁, e₂.

\hat{y} = 2(1, 0, 0) + 3(0, 1, 0) = (2, 3, 0)

Weights 2/1 and 3/1.

Answer: ŷ = (2, 3, 0).

10.

\frac{8 + 9 + 13 + 14}{4} = \frac{44}{4} = 11

Weight on the level.

\frac{-24 - 9 + 13 + 42}{20} = \frac{22}{20} = 1.1

Weight on the trend.

\hat{y} = (11 - 3.3, 11 - 1.1, 11 + 1.1, 11 + 3.3)

11u₁ + 1.1u₂.

Answer: ŷ = (7.7, 9.9, 12.1, 14.3).

Orthogonal projection FAQ

What is an orthogonal projection onto a subspace?

It is the vector ŷ in the subspace W that is closest to y. The difference y − ŷ is perpendicular to every vector in W.

How do you find the distance from a vector to a subspace?

Project the vector onto the subspace, subtract to get z = y − ŷ, and find the length ‖z‖.

Why does the formula need an orthogonal basis?

With an orthogonal basis, the pieces along different basis vectors do not overlap, so they can be computed one at a time and added. With a non-orthogonal basis, adding the pieces counts some directions twice.

What is a projection matrix?

It is the matrix P with Py = proj_W y for every y. If the columns of U are an orthonormal basis of W, then P = UUᵀ, and P² = P because projecting twice changes nothing.

Summary

proj_W y = Σ (y · uᵢ)/(uᵢ · uᵢ) uᵢ for an orthogonal basis u₁, …, uₚ of W.
y = ŷ + z with ŷ in W and z = y − ŷ orthogonal to W, and this split is unique.
ŷ is the closest point in W to y, and the distance from y to W is ‖y − ŷ‖.
Check your answer with z · uᵢ = 0 for every basis vector.
With orthonormal columns in U, the projection matrix is P = UUᵀ.

Previous lesson: Orthogonal Sets and Orthonormal Bases

Next lesson: Gram-Schmidt Process

Back to the Linear Algebra Index