Understanding Eigenvalues and Eigenvectors
In linear algebra, eigenvalues and eigenvectors play a crucial role in decomposing matrices, solving systems of equations, and even powering algorithms in machine learning and computer graphics.
If you're new to this concept, think of eigenvectors as special directions that remain unchanged (except for scaling) when a transformation (matrix) is applied. The eigenvalue tells us how much the vector gets scaled.
Why Do Eigenvalues and Eigenvectors Matter?
These concepts are foundational in:
- Principal Component Analysis (PCA) for dimensionality reduction
- Stability analysis in systems
- Quantum mechanics and vibration modes
And in this tutorial, you’ll learn to compute and interpret them using NumPy
.
Step 1: Import NumPy
import numpy as np
Step 2: Define a Square Matrix
A = np.array([[4, 2],
[1, 3]])
We're working with a 2x2 matrix here. Eigen decomposition is only defined for square matrices, so always verify the matrix shape using:
print("Matrix shape:", A.shape)
Step 3: Compute Eigenvalues and Eigenvectors
eigenvalues, eigenvectors = np.linalg.eig(A)
Explanation: np.linalg.eig()
returns two results:
eigenvalues
: A 1D array of eigenvalues.eigenvectors
: A 2D array where each column is an eigenvector corresponding to an eigenvalue.
Step 4: Print and Understand the Output
print("Eigenvalues:", eigenvalues)
print("Eigenvectors:
", eigenvectors)
Sample Output:
Eigenvalues: [5. 2.]
Eigenvectors:
[[ 0.89442719 -0.70710678]
[ 0.4472136 0.70710678]]
What Does This Output Tell Us?
There are two eigenvalues: 5 and 2. Each one has a corresponding eigenvector:
- The first eigenvector
[0.894, 0.447]
is scaled by 5 when multiplied by the matrix A. - The second eigenvector
[-0.707, 0.707]
is scaled by 2.
Step 5: Verify the Eigenvalue Equation
The property of eigenvectors is:
A * v = λ * v
Let’s verify this manually for one eigenvalue-eigenvector pair:
# Pick the first eigenvector and eigenvalue
v = eigenvectors[:, 0]
λ = eigenvalues[0]
Av = A @ v
λv = λ * v
print("A @ v:", Av)
print("λ * v:", λv)
Expected Output: The two results should be approximately equal (allowing for minor floating-point differences).
Step 6: Diagonalization
A matrix A
is diagonalizable if:
A = P * D * P-1
Where:
P
is the matrix of eigenvectorsD
is the diagonal matrix of eigenvalues
Let's Perform Diagonalization
P = eigenvectors
D = np.diag(eigenvalues)
P_inv = np.linalg.inv(P)
A_reconstructed = P @ D @ P_inv
print("Reconstructed A:
", A_reconstructed)
print("Original A:
", A)
Explanation: The reconstructed matrix A_reconstructed
should be nearly identical to the original matrix A
. This confirms diagonalizability.
Numerical Check: Are They Equal?
print("Is reconstructed A close to original A?", np.allclose(A, A_reconstructed))
Output: True
Key Considerations
- Only square matrices can be diagonalized.
- Some matrices are not diagonalizable (e.g., defective matrices).
- Use
np.allclose()
instead of==
for floating-point equality checks.
Conclusion
Eigenvalues and eigenvectors offer a powerful lens to analyze linear transformations. With NumPy, computing and verifying these is just a few lines of code away. By understanding diagonalization, you gain the tools to simplify matrix operations and peek deeper into the nature of systems and data structures.
What’s Next?
In the next part of this module, we’ll explore how to use these techniques to solve systems of equations and power machine learning algorithms like PCA.