Linear Independence
Linear independence means no vector in a set can be built from the others. Vectors v₁, …, vₚ are linearly independent when the only solution of c₁v₁ + ⋯ + cₚvₚ = 0 is c₁ = ⋯ = cₚ = 0. To test, put the vectors in the columns of a matrix and row reduce: they are independent exactly when every column has a pivot.
Before this lesson: Homogeneous Systems, Span of Vectors and Parametric Vector Form
What linear independence means
A set of vectors is linearly dependent if one of the vectors is a linear combination of the others. That vector adds nothing new: removing it leaves the span the same.
A set is linearly independent if no vector is redundant. Each one points in a direction the others cannot reach. In the plane, two vectors that point in different directions are independent and span the whole plane; two vectors on the same line are dependent.
Definition of linear independence
The formal test uses the vector equation below. Setting every weight to zero always works; this is the trivial solution. The question is whether any other choice of weights also works.
c_1\mathbf{v}_1 + c_2\mathbf{v}_2 + \cdots + c_p\mathbf{v}_p = \mathbf{0}- Independent: the trivial solution c₁ = c₂ = ⋯ = cₚ = 0 is the only solution.
- Dependent: some weights, not all zero, also solve it. That equation is called a linear dependence relation.
If A = [v₁ ⋯ vₚ], the vector equation is the homogeneous system Ax = 0. So the columns of A are independent exactly when Ax = 0 has only the trivial solution, which happens exactly when every column of A is a pivot column (no free variables).
How to test vectors for linear independence
- Write the vectors as the columns of a matrix A.
- Row reduce A to echelon form. You do not need an augmented column of zeros, because row operations never change it.
- Count pivots. If every column has a pivot, the vectors are linearly independent.
- If some column has no pivot, the vectors are dependent. To find a dependence relation, reduce to reduced row echelon form, set a free variable to 1 and solve for the weights.
Shortcuts that skip the row reduction:
- One vector: {v} is independent exactly when v ≠ 0.
- Two vectors: {u, v} is dependent exactly when one is a scalar multiple of the other.
- The zero vector: any set that contains 0 is dependent, since 1·0 = 0 is a nontrivial relation.
- Too many vectors: more than n vectors in ℝⁿ are always dependent, because a matrix with more columns than rows must have a free variable.
The pair test does not work for three or more vectors. A set can be dependent even when no two of its vectors are multiples of each other, as Example 3 shows.
Linear independence examples
Example 1: Pairs of vectors
Decide whether each pair is linearly independent: (a) u = (2, −1, 4) and w = (−6, 3, −12); (b) p = (1, 2, 0) and q = (2, 1, 0).
\tfrac{-6}{2} = -3,\; \tfrac{3}{-1} = -3,\; \tfrac{-12}{4} = -3(a) Divide each entry of w by the matching entry of u.
\mathbf{w} = -3\mathbf{u}Every ratio is −3, so w is a multiple of u.
3\mathbf{u} + \mathbf{w} = \mathbf{0}Add 3u to both sides: a dependence relation with weights 3 and 1.
\tfrac{q_1}{p_1} = \tfrac{2}{1} = 2,\; \tfrac{q_2}{p_2} = \tfrac{1}{2}(b) Compare the first two entries of q and p.
Answer: (a) dependent, since w = −3u. (b) independent, since the ratios 2 and 1/2 differ, so neither vector is a multiple of the other.
Example 2: Three independent vectors in ℝ³
Are v₁ = (1, 0, 2), v₂ = (2, 1, 5) and v₃ = (0, 3, 1) linearly independent?
A = \begin{bmatrix} 1 & 2 & 0 \\ 0 & 1 & 3 \\ 2 & 5 & 1 \end{bmatrix}Put the vectors in the columns of A.
\begin{bmatrix} 1 & 2 & 0 \\ 0 & 1 & 3 \\ 0 & 1 & 1 \end{bmatrix}R₃ − 2R₁ → R₃: 5 − 4 = 1 and 1 − 0 = 1 (see Gaussian elimination).
\begin{bmatrix} 1 & 2 & 0 \\ 0 & 1 & 3 \\ 0 & 0 & -2 \end{bmatrix}R₃ − R₂ → R₃: 1 − 1 = 0 and 1 − 3 = −2.
Answer: all three columns have a pivot, so Ax = 0 has only the trivial solution and v₁, v₂, v₃ are linearly independent.
Example 3: Find a linear dependence relation
Show that v₁ = (1, 2, −1), v₂ = (2, 5, 1) and v₃ = (0, 1, 3) are linearly dependent, and find a dependence relation. No two of these vectors are multiples of each other.
\begin{bmatrix} 1 & 2 & 0 \\ 2 & 5 & 1 \\ -1 & 1 & 3 \end{bmatrix}Put the vectors in the columns.
\begin{bmatrix} 1 & 2 & 0 \\ 0 & 1 & 1 \\ -1 & 1 & 3 \end{bmatrix}R₂ − 2R₁ → R₂: 5 − 4 = 1 and 1 − 0 = 1.
\begin{bmatrix} 1 & 2 & 0 \\ 0 & 1 & 1 \\ 0 & 3 & 3 \end{bmatrix}R₃ + R₁ → R₃: 1 + 2 = 3 and 3 + 0 = 3.
\begin{bmatrix} 1 & 2 & 0 \\ 0 & 1 & 1 \\ 0 & 0 & 0 \end{bmatrix}R₃ − 3R₂ → R₃. Column 3 has no pivot, so the set is dependent.
\begin{bmatrix} 1 & 0 & -2 \\ 0 & 1 & 1 \\ 0 & 0 & 0 \end{bmatrix}R₁ − 2R₂ → R₁: 2 − 2 = 0 and 0 − 2 = −2.
c_1 = 2c_3
Row 1: c₁ − 2c₃ = 0, so add 2c₃ to both sides.
c_2 = -c_3
Row 2: c₂ + c₃ = 0, so subtract c₃ from both sides.
c_1 = 2,\; c_2 = -1,\; c_3 = 1
Choose the free variable c₃ = 1.
2\mathbf{v}_1 - \mathbf{v}_2 + \mathbf{v}_3 = (2 - 2 + 0,\; 4 - 5 + 1,\; -2 - 1 + 3)Check by substituting the weights.
2\mathbf{v}_1 - \mathbf{v}_2 + \mathbf{v}_3 = (0, 0, 0)Simplify each entry.
Answer: dependent, with 2v₁ − v₂ + v₃ = 0. Solving for v₃ gives v₃ = v₂ − 2v₁, so v₃ lies in the span of v₁ and v₂.
Example 4: Shortcuts for special sets
(a) Are (1, 4), (2, −3) and (5, 0) independent? Find a dependence relation. (b) Are (3, 1, 2) and (0, 0, 0) independent?
\begin{bmatrix} 1 & 2 & 5 \\ 4 & -3 & 0 \end{bmatrix}(a) Three vectors in ℝ² must be dependent; row reduce to find the relation.
\begin{bmatrix} 1 & 2 & 5 \\ 0 & -11 & -20 \end{bmatrix}R₂ − 4R₁ → R₂: −3 − 8 = −11 and 0 − 20 = −20.
\begin{bmatrix} 1 & 2 & 5 \\ 0 & 1 & \tfrac{20}{11} \end{bmatrix}Divide R₂ by −11.
\begin{bmatrix} 1 & 0 & \tfrac{15}{11} \\ 0 & 1 & \tfrac{20}{11} \end{bmatrix}R₁ − 2R₂ → R₁: 5 − 40/11 = 55/11 − 40/11 = 15/11 (see subtracting fractions).
c_1 = -\tfrac{15}{11}c_3,\; c_2 = -\tfrac{20}{11}c_3Solve each row for its pivot variable.
c_1 = -15,\; c_2 = -20,\; c_3 = 11
Choose c₃ = 11 to clear the fractions.
-15(1, 4) - 20(2, -3) + 11(5, 0) = (-15 - 40 + 55,\; -60 + 60 + 0)
Check.
-15(1, 4) - 20(2, -3) + 11(5, 0) = (0, 0)
Simplify each entry.
0(3, 1, 2) + 1(0, 0, 0) = (0, 0, 0)
(b) A weight of 1 on the zero vector gives a nontrivial relation.
Answer: (a) dependent, with −15(1, 4) − 20(2, −3) + 11(5, 0) = 0. (b) dependent, because the set contains the zero vector.
Example 5: Can one feed mix replace another?
A farm buys three feed mixes. Per bag, Mix A has 2 kg protein, 1 kg fat and 3 kg fiber; Mix B has 1, 1 and 1 kg; Mix C has 4, 3 and 5 kg. Are the nutrient vectors A = (2, 1, 3), B = (1, 1, 1) and C = (4, 3, 5) independent? If not, how can Mix C be made from the other two?
\begin{bmatrix} 2 & 1 & 4 \\ 1 & 1 & 3 \\ 3 & 1 & 5 \end{bmatrix}Put A, B and C in the columns.
\begin{bmatrix} 1 & 1 & 3 \\ 2 & 1 & 4 \\ 3 & 1 & 5 \end{bmatrix}R₁ ↔ R₂, so the first pivot is 1.
\begin{bmatrix} 1 & 1 & 3 \\ 0 & -1 & -2 \\ 3 & 1 & 5 \end{bmatrix}R₂ − 2R₁ → R₂: 1 − 2 = −1 and 4 − 6 = −2.
\begin{bmatrix} 1 & 1 & 3 \\ 0 & -1 & -2 \\ 0 & -2 & -4 \end{bmatrix}R₃ − 3R₁ → R₃: 1 − 3 = −2 and 5 − 9 = −4.
\begin{bmatrix} 1 & 1 & 3 \\ 0 & 1 & 2 \\ 0 & -2 & -4 \end{bmatrix}−R₂ → R₂.
\begin{bmatrix} 1 & 1 & 3 \\ 0 & 1 & 2 \\ 0 & 0 & 0 \end{bmatrix}R₃ + 2R₂ → R₃. Column 3 has no pivot, so the set is dependent.
\begin{bmatrix} 1 & 0 & 1 \\ 0 & 1 & 2 \\ 0 & 0 & 0 \end{bmatrix}R₁ − R₂ → R₁.
c_1 = -c_3,\; c_2 = -2c_3
Solve rows 1 and 2 for the pivot variables.
-\mathbf{A} - 2\mathbf{B} + \mathbf{C} = \mathbf{0}Choose c₃ = 1.
\mathbf{C} = \mathbf{A} + 2\mathbf{B}Add A + 2B to both sides.
\mathbf{A} + 2\mathbf{B} = (2 + 2,\; 1 + 2,\; 3 + 2) = (4, 3, 5)Check.
Answer: the mixes are dependent. One bag of Mix A plus two bags of Mix B give the same nutrients as one bag of Mix C, so Mix C adds no new combination of nutrients.
Common mistakes
| Mistake | Example of the error | Fix |
|---|---|---|
| Using the pair test on three or more vectors | Calling the vectors in Example 3 independent because no two are multiples | Row reduce: a set can be dependent with no pair of multiples. |
| Treating the trivial solution as evidence of dependence | “c₁ = c₂ = c₃ = 0 works, so the set is dependent” | The trivial solution always works. Dependence needs a solution with some weight not zero. |
| Checking pivots in rows instead of columns | Calling three vectors in ℝ² independent because both rows have pivots | Independence needs a pivot in every column. A pivot in every row is the test for spanning ℝⁿ. |
| Ignoring the zero vector | Calling {(3, 1, 2), (0, 0, 0)} independent because (3, 1, 2) ≠ 0 | Any set containing 0 is dependent. |
| Using a determinant on a non-square matrix | Trying det for three vectors in ℝ⁴ | The determinant test only applies to n vectors in ℝⁿ; otherwise count pivots. |
Related methods
Determinant test. For exactly n vectors in ℝⁿ, the matrix A = [v₁ ⋯ vₙ] is square, and the vectors are independent exactly when det A ≠ 0. In Example 2, det A = 1(1 − 15) − 2(0 − 6) + 0 = −14 + 12 = −2, which is not 0, so the vectors are independent (see the determinant of a matrix).
Independence and span. Independence asks for a pivot in every column; spanning ℝⁿ asks for a pivot in every row. A set of n vectors in ℝⁿ has both or neither. A set that is independent and spans a space is called a basis, the topic of Basis and Dimension (coming soon).
Independence and solutions. If the columns of A are independent, then Ax = b has at most one solution for each b, because Ax = 0 has no free variables.
Practice problems
- Are (4, −2) and (−6, 3) linearly independent?
- Are (1, 3, −2) and (2, 6, −3) linearly independent?
- Are (1, 2), (3, 4) and (5, 6) linearly independent?
- Are (0, 0, 0) and (1, 2, 3) linearly independent?
- Test (1, 1, 0), (0, 1, 1) and (1, 0, 1) for linear independence.
- Show that (1, −1, 2), (3, −2, 5) and (1, 0, 1) are dependent and find a dependence relation.
- For what value of h are (1, −2, 3), (−2, 5, h) and (0, 1, 4) linearly dependent?
- Are the columns of the 3 × 2 matrix with rows [1 4], [2 5] and [3 6] linearly independent?
- Use a determinant to decide whether (2, 1) and (5, 3) are linearly independent.
- A paint store mixes colors from red, yellow and blue parts. Color A uses (1, 2, 0) parts, Color B uses (0, 1, 1) and Color C uses (2, 5, 1). Are the three colors independent? If not, write C in terms of A and B.
Show answers
1.
\tfrac{-6}{4} = -\tfrac{3}{2},\; \tfrac{3}{-2} = -\tfrac{3}{2}Divide each entry of the second vector by the matching entry of the first.
(-6, 3) = -\tfrac{3}{2}(4, -2)Both ratios are −3/2.
Answer: dependent, since (−6, 3) is a multiple of (4, −2).
2.
\tfrac{2}{1} = 2,\; \tfrac{6}{3} = 2,\; \tfrac{-3}{-2} = \tfrac{3}{2}Compare the ratios of matching entries.
Answer: independent. The ratios are not all equal, so neither vector is a multiple of the other.
3.
3 \gt 2
There are 3 vectors in ℝ², more than the 2 entries in each.
Answer: dependent. A 2 × 3 matrix has at most 2 pivots, so one column has no pivot.
4.
0(1, 2, 3) + 1(0, 0, 0) = (0, 0, 0)
A weight of 1 on the zero vector is a nontrivial relation.
Answer: dependent, because the set contains the zero vector.
5.
\begin{bmatrix} 1 & 0 & 1 \\ 1 & 1 & 0 \\ 0 & 1 & 1 \end{bmatrix}Put the vectors in the columns.
\begin{bmatrix} 1 & 0 & 1 \\ 0 & 1 & -1 \\ 0 & 1 & 1 \end{bmatrix}R₂ − R₁ → R₂: 0 − 1 = −1.
\begin{bmatrix} 1 & 0 & 1 \\ 0 & 1 & -1 \\ 0 & 0 & 2 \end{bmatrix}R₃ − R₂ → R₃: 1 − (−1) = 2.
Answer: three pivots in three columns, so the vectors are linearly independent.
6.
\begin{bmatrix} 1 & 3 & 1 \\ -1 & -2 & 0 \\ 2 & 5 & 1 \end{bmatrix}Put the vectors in the columns.
\begin{bmatrix} 1 & 3 & 1 \\ 0 & 1 & 1 \\ 2 & 5 & 1 \end{bmatrix}R₂ + R₁ → R₂: −2 + 3 = 1 and 0 + 1 = 1.
\begin{bmatrix} 1 & 3 & 1 \\ 0 & 1 & 1 \\ 0 & -1 & -1 \end{bmatrix}R₃ − 2R₁ → R₃: 5 − 6 = −1 and 1 − 2 = −1.
\begin{bmatrix} 1 & 3 & 1 \\ 0 & 1 & 1 \\ 0 & 0 & 0 \end{bmatrix}R₃ + R₂ → R₃. Column 3 has no pivot.
\begin{bmatrix} 1 & 0 & -2 \\ 0 & 1 & 1 \\ 0 & 0 & 0 \end{bmatrix}R₁ − 3R₂ → R₁: 3 − 3 = 0 and 1 − 3 = −2.
c_1 = 2c_3,\; c_2 = -c_3
Solve rows 1 and 2 for the pivot variables.
c_1 = 2,\; c_2 = -1,\; c_3 = 1
Choose c₃ = 1.
Answer: 2(1, −1, 2) − (3, −2, 5) + (1, 0, 1) = 0. Check: 2 − 3 + 1 = 0, −2 + 2 + 0 = 0 and 4 − 5 + 1 = 0.
7.
\begin{bmatrix} 1 & -2 & 0 \\ -2 & 5 & 1 \\ 3 & h & 4 \end{bmatrix}Put the vectors in the columns.
\begin{bmatrix} 1 & -2 & 0 \\ 0 & 1 & 1 \\ 3 & h & 4 \end{bmatrix}R₂ + 2R₁ → R₂: 5 − 4 = 1 and 1 + 0 = 1.
\begin{bmatrix} 1 & -2 & 0 \\ 0 & 1 & 1 \\ 0 & h + 6 & 4 \end{bmatrix}R₃ − 3R₁ → R₃: h − 3(−2) = h + 6 and 4 − 0 = 4.
\begin{bmatrix} 1 & -2 & 0 \\ 0 & 1 & 1 \\ 0 & 0 & -2 - h \end{bmatrix}R₃ − (h + 6)R₂ → R₃: 4 − (h + 6) = −2 − h.
-2 - h = 0
The set is dependent exactly when column 3 has no pivot.
h = -2
Add h to both sides.
Answer: h = −2. For every other h the vectors are independent.
8.
\begin{bmatrix} 1 & 4 \\ 2 & 5 \\ 3 & 6 \end{bmatrix}The matrix.
\begin{bmatrix} 1 & 4 \\ 0 & -3 \\ 3 & 6 \end{bmatrix}R₂ − 2R₁ → R₂: 5 − 8 = −3.
\begin{bmatrix} 1 & 4 \\ 0 & -3 \\ 0 & -6 \end{bmatrix}R₃ − 3R₁ → R₃: 6 − 12 = −6.
\begin{bmatrix} 1 & 4 \\ 0 & -3 \\ 0 & 0 \end{bmatrix}R₃ − 2R₂ → R₃: −6 − 2(−3) = 0.
Answer: both columns have a pivot, so the columns are linearly independent.
9.
\det\begin{bmatrix} 2 & 5 \\ 1 & 3 \end{bmatrix} = 2(3) - 5(1)Put the vectors in the columns and use ad − bc.
\det\begin{bmatrix} 2 & 5 \\ 1 & 3 \end{bmatrix} = 1Simplify: 6 − 5 = 1.
Answer: det A = 1 ≠ 0, so the vectors are linearly independent.
10.
\begin{bmatrix} 1 & 0 & 2 \\ 2 & 1 & 5 \\ 0 & 1 & 1 \end{bmatrix}Put A, B and C in the columns.
\begin{bmatrix} 1 & 0 & 2 \\ 0 & 1 & 1 \\ 0 & 1 & 1 \end{bmatrix}R₂ − 2R₁ → R₂: 5 − 4 = 1.
\begin{bmatrix} 1 & 0 & 2 \\ 0 & 1 & 1 \\ 0 & 0 & 0 \end{bmatrix}R₃ − R₂ → R₃. Column 3 has no pivot.
c_1 = -2c_3,\; c_2 = -c_3
Solve rows 1 and 2 for the pivot variables.
-2\mathbf{A} - \mathbf{B} + \mathbf{C} = \mathbf{0}Choose c₃ = 1.
\mathbf{C} = 2\mathbf{A} + \mathbf{B}Add 2A + B to both sides.
Answer: dependent; C = 2A + B. Check: 2(1, 2, 0) + (0, 1, 1) = (2, 5, 1).
Linear independence FAQ
What does it mean for vectors to be linearly independent?
No vector in the set is a linear combination of the others. Equivalently, the only way to combine them to get the zero vector is with all weights equal to zero.
How do you check if vectors are linearly independent?
Put the vectors in the columns of a matrix and row reduce. If every column has a pivot, they are independent; if any column lacks a pivot, they are dependent. For n vectors in ℝⁿ, you can also check that the determinant is not zero.
Can 3 vectors in ℝ² be linearly independent?
No. Any set of more than n vectors in ℝⁿ is linearly dependent, because the matrix has more columns than rows and must have a free variable.
What is the difference between linear independence and span?
Span asks which vectors you can reach with combinations; independence asks whether any vector in the set is redundant. Spanning ℝⁿ needs a pivot in every row, and independence needs a pivot in every column.
Summary
- Vectors are linearly independent when c₁v₁ + ⋯ + cₚvₚ = 0 has only the trivial solution.
- Test: put the vectors in the columns of A and row reduce. A pivot in every column means independent.
- A free variable gives a dependence relation, which writes one vector as a combination of the others.
- Shortcuts: two vectors are dependent exactly when one is a multiple of the other; any set with 0 is dependent; more than n vectors in ℝⁿ are dependent.
- For n vectors in ℝⁿ, independent exactly when det A ≠ 0.
Previous lesson: Parametric Vector Form
Next lesson: Linear Transformations
Back to the Linear Algebra Index