The standard method for finding an orthogonal basis is the Gram-Schmidt process, which takes any set of linearly independent vectors and systematically converts them into a set of vectors that are all perpendicular to each other. The core idea is simple: take each vector, subtract off the parts that point in the same direction as the vectors you’ve already processed, and what remains is perpendicular to all of them.
What Makes a Basis Orthogonal
A set of vectors forms a basis for a space if the vectors are linearly independent (none can be written as a combination of the others) and they span the entire space. An orthogonal basis adds one requirement: every pair of vectors in the set must be perpendicular, meaning their dot product equals zero.
You’ll also see the term “orthonormal basis,” which goes one step further. An orthonormal basis is orthogonal, and every vector in it has a length of exactly 1. Converting from orthogonal to orthonormal is straightforward: just divide each vector by its length. The hard part is getting orthogonality in the first place.
Orthogonal bases are useful because they make projections, decompositions, and coordinate calculations dramatically simpler. When your basis vectors are perpendicular, finding a vector’s coordinates doesn’t require solving a system of equations. You just compute dot products.
The Gram-Schmidt Process, Step by Step
Suppose you have a set of linearly independent vectors {u₁, u₂, …, uₙ} that form a basis for some space. The Gram-Schmidt process builds a new orthogonal basis {v₁, v₂, …, vₙ} for the same space, one vector at a time.
Step 1: Keep the first vector as-is. Set v₁ = u₁. This is your starting direction.
Step 2: Take the second vector u₂ and subtract its projection onto v₁. The projection formula is:
v₂ = u₂ − [(u₂ · v₁) / (v₁ · v₁)] × v₁
What you’re doing here is removing the component of u₂ that points along v₁. The remainder, v₂, is perpendicular to v₁.
Step 3: Take the third vector u₃ and subtract its projections onto both v₁ and v₂:
v₃ = u₃ − [(u₃ · v₁) / (v₁ · v₁)] × v₁ − [(u₃ · v₂) / (v₂ · v₂)] × v₂
Now v₃ is perpendicular to both v₁ and v₂.
General pattern: For each new vector, subtract its projections onto all the orthogonal vectors you’ve already computed. Each step peels away the components that overlap with previous directions, leaving only the genuinely new direction.
Continue until you’ve processed all n vectors. The resulting set {v₁, v₂, …, vₙ} is your orthogonal basis.
A Concrete Example
Start with two vectors in R³: u₁ = (1, 1, 0) and u₂ = (1, 0, 1).
Set v₁ = u₁ = (1, 1, 0).
Now compute v₂. First, find the dot products: u₂ · v₁ = (1)(1) + (0)(1) + (1)(0) = 1, and v₁ · v₁ = 1 + 1 + 0 = 2. The projection coefficient is 1/2, so:
v₂ = (1, 0, 1) − (1/2)(1, 1, 0) = (1/2, −1/2, 1)
To verify, check that v₁ · v₂ = (1)(1/2) + (1)(−1/2) + (0)(1) = 0. The dot product is zero, so the vectors are orthogonal. The set {(1, 1, 0), (1/2, −1/2, 1)} is an orthogonal basis for the subspace spanned by the original vectors.
If you want an orthonormal basis, divide each vector by its length. The length of v₁ is √2, and the length of v₂ is √(1/4 + 1/4 + 1) = √(3/2). Dividing gives you unit vectors that are still perpendicular to each other.
How to Verify Your Result
After running Gram-Schmidt, always check your work. The verification is simple: compute the dot product of every pair of vectors in your new basis. Every single pair should give you exactly zero. For an orthogonal basis with n vectors, you need to check n(n−1)/2 pairs.
If you’ve arranged your orthogonal vectors as columns of a matrix Q, there’s a shortcut. Multiply Qᵀ by Q. If the columns are orthogonal, the off-diagonal entries of QᵀQ will all be zero. If the columns are orthonormal, QᵀQ will be the identity matrix (zeros off the diagonal, ones on the diagonal).
The Projection Formula Is the Key
Everything in the Gram-Schmidt process hinges on one formula: projecting a vector x onto a line spanned by a vector u.
proj = [(x · u) / (u · u)] × u
The numerator x · u measures how much x points in the direction of u. The denominator u · u normalizes for the length of u. The result is the component of x that lies along u. When you subtract this projection from x, you get the component of x that is perpendicular to u.
When projecting onto a subspace spanned by multiple orthogonal vectors, you simply add up the individual projections onto each vector and subtract the total. This works precisely because the basis vectors are orthogonal, so you can handle them independently. That’s the beauty of the process building on itself: each new orthogonal vector lets you cleanly separate the next one.
Numerical Stability and the Modified Version
The version described above is called “classical” Gram-Schmidt. It works perfectly in exact arithmetic (pencil and paper, or symbolic computation), but it can behave poorly on a computer. Rounding errors accumulate, and the computed vectors may not actually be very orthogonal. In one illustrative example from MIT course materials, the classical version produced vectors with a pairwise dot product of 1/2 where the answer should have been 0. The modified version got exactly 0.
The fix is called Modified Gram-Schmidt. Instead of computing all projections using the original vector uⱼ, you update the vector after each individual projection is subtracted. So when finding v₃, you first subtract the v₁ projection from u₃ to get an intermediate result, then subtract the v₂ projection from that intermediate result rather than from the original u₃. Mathematically the two versions are equivalent, but the modified version is significantly less sensitive to rounding errors because each subtraction works with the most up-to-date vector.
For homework problems and small examples, classical Gram-Schmidt is fine. For programming or working with large matrices, use the modified version.
QR Decomposition as an Alternative
QR decomposition is closely related to Gram-Schmidt and provides another way to think about orthogonal bases. Any matrix A with linearly independent columns can be factored as A = QR, where Q is a matrix with orthonormal columns and R is an upper triangular matrix. The columns of Q are an orthonormal basis for the column space of A.
In practice, the Gram-Schmidt process is one way to compute this QR decomposition. You orthogonalize the columns of A to get Q, and the projection coefficients you calculated along the way fill in the entries of R. But for computational work on large matrices, Householder reflections are generally preferred over Gram-Schmidt. Householder reflections use a geometric trick: they reflect vectors across carefully chosen planes to zero out specific components, building up the orthogonal matrix Q through a sequence of these reflections. The result is more numerically stable than even Modified Gram-Schmidt.
If you’re using software like MATLAB, Python (NumPy), or similar tools, calling the built-in QR decomposition function and extracting the Q matrix is often the most practical way to get an orthogonal basis for a column space.
Why Orthogonal Vectors Are Automatically Independent
One useful property worth noting: a set of nonzero orthogonal vectors is always linearly independent. You never need to check independence separately. If every pair has a dot product of zero and no vector is the zero vector, the set is guaranteed to be independent. This means that if you produce n orthogonal vectors in an n-dimensional space, you automatically have a basis. The Gram-Schmidt process exploits this: as long as your input vectors are independent (so no step produces the zero vector), the output is guaranteed to be both orthogonal and a basis.