Orthogonal Sets and Orthonormal Bases

An orthogonal set is a set of vectors in which every pair has dot product 0. An orthonormal basis is an orthogonal basis made of unit vectors. In an orthogonal basis, each coordinate comes from one dot product, with no row reduction. A square matrix with orthonormal columns is an orthogonal matrix, and its inverse is its transpose.

Before this lesson: Dot Product, Basis and Dimension and Transpose of a Matrix

Why orthogonal bases are easier

For most bases, finding the coordinates of a vector means solving a system of equations. When the basis vectors are perpendicular, each coordinate can be found on its own, as if the other basis vectors were not there.

In the picture, u₁ = (2, 1) and u₂ = (−1, 2) are perpendicular because u₁ · u₂ = −2 + 2 = 0. The vector x = (4, 7) splits into 3u₁ + 2u₂, and each weight comes from a single dot product: 3 = (x · u₁)/(u₁ · u₁) = 15/5 and 2 = (x · u₂)/(u₂ · u₂) = 10/5.

Definitions and formulas

For vectors in ℝⁿ, the dot product is u · v = u₁v₁ + u₂v₂ + ⋯ + uₙvₙ, and the length (norm) of v is ‖v‖ = √(v · v).

• Two vectors are orthogonal if u · v = 0.
• A set of vectors is an orthogonal set if every pair of different vectors in it is orthogonal.
• A unit vector has length 1. To normalize a nonzero vector, divide it by its length.
• An orthonormal set is an orthogonal set of unit vectors. An orthogonal (or orthonormal) set that is also a basis is an orthogonal basis (or orthonormal basis).
• An orthogonal set of nonzero vectors is always linearly independent. So n nonzero orthogonal vectors in ℝⁿ form a basis.

Coordinates in an orthogonal basis. If {u₁, …, uₙ} is an orthogonal basis and x = c₁u₁ + ⋯ + cₙuₙ, then

c_i = \dfrac{x \cdot u_i}{u_i \cdot u_i} \qquad \text{(for an orthonormal basis, } c_i = x \cdot u_i\text{)}

This works because dotting both sides with uᵢ wipes out every other term: uⱼ · uᵢ = 0 for j ≠ i.

Orthogonal matrices. A square matrix Q is an orthogonal matrix if its columns form an orthonormal set. Then

Q^TQ = I \qquad Q^{-1} = Q^T \qquad \|Qx\| = \|x\|

So the inverse matrix costs nothing to find: Qx = b is solved by x = Qᵀb, and multiplying by Q never changes lengths or angles. Rotations and reflections, like the ones in matrix transformations, are orthogonal matrices.

How to work with an orthogonal basis

1. Check that the set is orthogonal: compute the dot product of every pair and confirm each is 0.
2. To make it orthonormal, divide each vector by its length.
3. To find coordinates, compute cᵢ = (x · uᵢ)/(uᵢ · uᵢ) for each basis vector.
4. Check by rebuilding x = c₁u₁ + ⋯ + cₙuₙ.

Orthogonal set and orthonormal basis examples

Example 1: Check an orthogonal set

Show that u_1 = \begin{bmatrix} 1 \\ 1 \\ 1 \end{bmatrix}, u_2 = \begin{bmatrix} 1 \\ -2 \\ 1 \end{bmatrix} and u_3 = \begin{bmatrix} 1 \\ 0 \\ -1 \end{bmatrix} form an orthogonal basis for ℝ³.

u_1 \cdot u_2 = 1 - 2 + 1 = 0

u_1 \cdot u_3 = 1 + 0 - 1 = 0

u_2 \cdot u_3 = 1 + 0 - 1 = 0

Answer: the three vectors are nonzero and orthogonal, so they are linearly independent. Three independent vectors in ℝ³ form a basis, so {u₁, u₂, u₃} is an orthogonal basis.

Example 2: Coordinates without row reduction

Write x = \begin{bmatrix} 4 \\ 4 \\ -2 \end{bmatrix} in terms of the orthogonal basis from Example 1.

x \cdot u_1 = 4 + 4 - 2 = 6,\quad u_1 \cdot u_1 = 1 + 1 + 1 = 3

c_1 = \dfrac{6}{3} = 2

x \cdot u_2 = 4 - 8 - 2 = -6,\quad u_2 \cdot u_2 = 1 + 4 + 1 = 6

c_2 = \dfrac{-6}{6} = -1

x \cdot u_3 = 4 + 0 + 2 = 6,\quad u_3 \cdot u_3 = 1 + 0 + 1 = 2

c_3 = \dfrac{6}{2} = 3

2u_1 - u_2 + 3u_3 = \begin{bmatrix} 2 - 1 + 3 \\ 2 + 2 + 0 \\ 2 - 1 - 3 \end{bmatrix} = \begin{bmatrix} 4 \\ 4 \\ -2 \end{bmatrix}

Answer: x = 2u₁ − u₂ + 3u₃.

Example 3: Build an orthonormal basis

Normalize the basis from Example 1 to get an orthonormal basis for ℝ³.

\|u_1\| = \sqrt{u_1 \cdot u_1} = \sqrt{3}

\|u_2\| = \sqrt{6},\quad \|u_3\| = \sqrt{2}

q_1 = \dfrac{1}{\sqrt{3}}\begin{bmatrix} 1 \\ 1 \\ 1 \end{bmatrix},\quad q_2 = \dfrac{1}{\sqrt{6}}\begin{bmatrix} 1 \\ -2 \\ 1 \end{bmatrix},\quad q_3 = \dfrac{1}{\sqrt{2}}\begin{bmatrix} 1 \\ 0 \\ -1 \end{bmatrix}

q_2 \cdot q_2 = \dfrac{1 + 4 + 1}{6} = 1

Answer: {q₁, q₂, q₃} above is an orthonormal basis. Scaling does not change the zero dot products, so the vectors stay orthogonal.

Example 4: An orthogonal matrix

Let Q = \begin{bmatrix} 0.6 & -0.8 \\ 0.8 & 0.6 \end{bmatrix}. Show that Q is an orthogonal matrix, then solve Qx = \begin{bmatrix} 5 \\ 10 \end{bmatrix}.

q_1 \cdot q_2 = (0.6)(-0.8) + (0.8)(0.6) = -0.48 + 0.48 = 0

q_1 \cdot q_1 = 0.36 + 0.64 = 1,\quad q_2 \cdot q_2 = 0.64 + 0.36 = 1

Q^{-1} = Q^T = \begin{bmatrix} 0.6 & 0.8 \\ -0.8 & 0.6 \end{bmatrix}

x = Q^Tb = \begin{bmatrix} 0.6(5) + 0.8(10) \\ -0.8(5) + 0.6(10) \end{bmatrix}

= \begin{bmatrix} 3 + 8 \\ -4 + 6 \end{bmatrix} = \begin{bmatrix} 11 \\ 2 \end{bmatrix}

Qx = \begin{bmatrix} 6.6 - 1.6 \\ 8.8 + 1.2 \end{bmatrix} = \begin{bmatrix} 5 \\ 10 \end{bmatrix}

\|x\| = \sqrt{121 + 4} = \sqrt{125},\quad \|b\| = \sqrt{25 + 100} = \sqrt{125}

Answer: Q is orthogonal (it rotates the plane), and x = (11, 2).

Example 5: Splitting website traffic into patterns

A site’s visits over four weeks were x = (120, 100, 90, 70). An analyst splits this into four patterns: u_1 = (1, 1, 1, 1) (overall level), u_2 = (1, 1, -1, -1) (first half vs. second half), u_3 = (1, -1, 0, 0) (week 1 vs. week 2) and u_4 = (0, 0, 1, -1) (week 3 vs. week 4). These four vectors form an orthogonal basis of ℝ⁴. Find the weight on each pattern.

u_1 \cdot u_1 = 4,\quad u_2 \cdot u_2 = 4,\quad u_3 \cdot u_3 = 2,\quad u_4 \cdot u_4 = 2

x \cdot u_1 = 120 + 100 + 90 + 70 = 380

c_1 = \dfrac{380}{4} = 95

x \cdot u_2 = 120 + 100 - 90 - 70 = 60

c_2 = \dfrac{60}{4} = 15

x \cdot u_3 = 120 - 100 = 20

c_3 = \dfrac{20}{2} = 10

x \cdot u_4 = 90 - 70 = 20

c_4 = \dfrac{20}{2} = 10

95u_1 + 15u_2 + 10u_3 + 10u_4 = \begin{bmatrix} 95 + 15 + 10 + 0 \\ 95 + 15 - 10 + 0 \\ 95 - 15 + 0 + 10 \\ 95 - 15 + 0 - 10 \end{bmatrix} = \begin{bmatrix} 120 \\ 100 \\ 90 \\ 70 \end{bmatrix}

Answer: x = 95u₁ + 15u₂ + 10u₃ + 10u₄. Visits averaged 95 a week, the first half ran 15 above that average and the second half 15 below, and each half dropped by 20 visits from its first week to its second (10 above and 10 below the half’s average).

Common mistakes

Related ideas

Projections. The weight cᵢ = (x · uᵢ)/(uᵢ · uᵢ) is the same scalar as in vector projections: cᵢuᵢ is the projection of x onto uᵢ. The next lesson, Orthogonal Projections, adds these pieces to project onto a whole subspace.

Getting an orthogonal basis. Not every basis is orthogonal. The Gram-Schmidt Process turns any basis into an orthogonal one, and QR Factorization records the result as A = QR with orthonormal columns in Q.

Wavelets. The patterns in Example 5 are a small Haar wavelet basis. Image and audio compression keep only the largest weights in bases like this one.

Practice problems

1. Is {(1, 2), (−4, 2)} an orthogonal set?
2. Is {(1, 0, 1), (1, 1, −1), (0, 1, 1)} an orthogonal set?
3. Normalize v = (2, −1, 2).
4. The vectors u₁ = (2, 1) and u₂ = (−1, 2) are an orthogonal basis of ℝ². Write x = (1, 8) in terms of them.
5. Write x = (3, 3, 1) in terms of the orthogonal basis u₁ = (1, 1, 1), u₂ = (1, −2, 1), u₃ = (1, 0, −1).
6. Find a so that (1, a, 3) is orthogonal to (2, 4, −2).
7. Show that Q = \begin{bmatrix} 1/\sqrt{2} & -1/\sqrt{2} \\ 1/\sqrt{2} & 1/\sqrt{2} \end{bmatrix} is an orthogonal matrix and find Q⁻¹.
8. Is \begin{bmatrix} 1 & 2 \\ 2 & -1 \end{bmatrix} an orthogonal matrix?
9. Solve Qx = b for Q = \begin{bmatrix} 0.6 & 0.8 \\ -0.8 & 0.6 \end{bmatrix} and b = (10, 5).
10. Using the four patterns from Example 5, find the weights for visits x = (50, 30, 40, 40).

(1)(-4) + (2)(2) = -4 + 4 = 0

(1, 0, 1) \cdot (1, 1, -1) = 1 + 0 - 1 = 0

(1, 0, 1) \cdot (0, 1, 1) = 0 + 0 + 1 = 1

\|v\| = \sqrt{4 + 1 + 4} = \sqrt{9} = 3

\dfrac{v}{\|v\|} = \left(\tfrac{2}{3}, -\tfrac{1}{3}, \tfrac{2}{3}\right)

c_1 = \dfrac{2 + 8}{4 + 1} = \dfrac{10}{5} = 2

c_2 = \dfrac{-1 + 16}{1 + 4} = \dfrac{15}{5} = 3

2u_1 + 3u_2 = (4 - 3, 2 + 6) = (1, 8)

c_1 = \dfrac{3 + 3 + 1}{3} = \dfrac{7}{3}

c_2 = \dfrac{3 - 6 + 1}{6} = \dfrac{-2}{6} = -\dfrac{1}{3}

c_3 = \dfrac{3 + 0 - 1}{2} = \dfrac{2}{2} = 1

\tfrac{7}{3}u_1 - \tfrac{1}{3}u_2 + u_3 = \left(\tfrac{7}{3} - \tfrac{1}{3} + 1, \tfrac{7}{3} + \tfrac{2}{3} + 0, \tfrac{7}{3} - \tfrac{1}{3} - 1\right) = (3, 3, 1)

2 + 4a - 6 = 0

4a - 4 = 0

a = 1

q_1 \cdot q_2 = -\tfrac{1}{2} + \tfrac{1}{2} = 0

q_1 \cdot q_1 = \tfrac{1}{2} + \tfrac{1}{2} = 1,\quad q_2 \cdot q_2 = \tfrac{1}{2} + \tfrac{1}{2} = 1

(1)(2) + (2)(-1) = 0

1^2 + 2^2 = 5 \ne 1

x = Q^Tb = \begin{bmatrix} 0.6 & -0.8 \\ 0.8 & 0.6 \end{bmatrix}\begin{bmatrix} 10 \\ 5 \end{bmatrix}

= \begin{bmatrix} 6 - 4 \\ 8 + 3 \end{bmatrix} = \begin{bmatrix} 2 \\ 11 \end{bmatrix}

Qx = \begin{bmatrix} 1.2 + 8.8 \\ -1.6 + 6.6 \end{bmatrix} = \begin{bmatrix} 10 \\ 5 \end{bmatrix}

c_1 = \dfrac{50 + 30 + 40 + 40}{4} = \dfrac{160}{4} = 40

c_2 = \dfrac{50 + 30 - 40 - 40}{4} = \dfrac{0}{4} = 0

c_3 = \dfrac{50 - 30}{2} = 10,\quad c_4 = \dfrac{40 - 40}{2} = 0

Orthonormal basis FAQ

What is an orthonormal basis?

It is a basis whose vectors are all unit vectors and are pairwise orthogonal. The standard basis e₁, …, eₙ of ℝⁿ is the most familiar example.

What is the difference between orthogonal and orthonormal?

Orthogonal vectors have dot product 0. Orthonormal vectors are orthogonal and also have length 1.

Is an orthogonal set always linearly independent?

Yes, as long as none of its vectors is the zero vector. Dotting a dependence equation with each vector forces every weight to be 0.

What is an orthogonal matrix?

It is a square matrix whose columns are orthonormal. Its inverse equals its transpose, QᵀQ = I, and it preserves lengths and angles.

Summary

• An orthogonal set has every pair of dot products equal to 0; an orthonormal set also has unit vectors.
• Nonzero orthogonal vectors are linearly independent, so n of them form a basis of ℝⁿ.
• In an orthogonal basis, cᵢ = (x · uᵢ)/(uᵢ · uᵢ); in an orthonormal basis, cᵢ = x · uᵢ.
• Normalize a vector by dividing by its length ‖v‖ = √(v · v).
• An orthogonal matrix has orthonormal columns, Q⁻¹ = Qᵀ, and it preserves lengths.

Previous lesson: Diagonalization

Next lesson: Orthogonal Projections

Back to the Linear Algebra Index