Eigenvalues, Eigenvectors, and Diagonalization in NumPy
Linear Algebra Tutorial

Understanding Eigenvalues and Eigenvectors

In linear algebra, eigenvalues and eigenvectors play a crucial role in decomposing matrices, solving systems of equations, and even powering algorithms in machine learning and computer graphics.

If you're new to this concept, think of eigenvectors as special directions that remain unchanged (except for scaling) when a transformation (matrix) is applied. The eigenvalue tells us how much the vector gets scaled.

Why Do Eigenvalues and Eigenvectors Matter?

These concepts are foundational in:

• Principal Component Analysis (PCA) for dimensionality reduction
• Stability analysis in systems
• Quantum mechanics and vibration modes

And in this tutorial, you’ll learn to compute and interpret them using NumPy.

Step 1: Import NumPy

import numpy as np

Step 2: Define a Square Matrix

A = np.array([[4, 2],
            [1, 3]])

We're working with a 2x2 matrix here. Eigen decomposition is only defined for square matrices, so always verify the matrix shape using:

print("Matrix shape:", A.shape)

Step 3: Compute Eigenvalues and Eigenvectors

eigenvalues, eigenvectors = np.linalg.eig(A)

Explanation: np.linalg.eig() returns two results:

• eigenvalues: A 1D array of eigenvalues.
• eigenvectors: A 2D array where each column is an eigenvector corresponding to an eigenvalue.

Step 4: Print and Understand the Output

print("Eigenvalues:", eigenvalues)
print("Eigenvectors:
", eigenvectors)

Sample Output:

Eigenvalues: [5. 2.]
Eigenvectors:
[[ 0.89442719 -0.70710678]
[ 0.4472136   0.70710678]]

What Does This Output Tell Us?

There are two eigenvalues: 5 and 2. Each one has a corresponding eigenvector:

• The first eigenvector [0.894, 0.447] is scaled by 5 when multiplied by the matrix A.
• The second eigenvector [-0.707, 0.707] is scaled by 2.

Step 5: Verify the Eigenvalue Equation

The property of eigenvectors is:

A * v = λ * v

Let’s verify this manually for one eigenvalue-eigenvector pair:

# Pick the first eigenvector and eigenvalue
v = eigenvectors[:, 0]
λ = eigenvalues[0]

Av = A @ v
λv = λ * v

print("A @ v:", Av)
print("λ * v:", λv)

Expected Output: The two results should be approximately equal (allowing for minor floating-point differences).

Step 6: Diagonalization

A matrix A is diagonalizable if:

A = P * D * P-1

Where:

• P is the matrix of eigenvectors
• D is the diagonal matrix of eigenvalues

Let's Perform Diagonalization

P = eigenvectors
D = np.diag(eigenvalues)
P_inv = np.linalg.inv(P)

A_reconstructed = P @ D @ P_inv

print("Reconstructed A:
", A_reconstructed)
print("Original A:
", A)

Explanation: The reconstructed matrix A_reconstructed should be nearly identical to the original matrix A. This confirms diagonalizability.

Numerical Check: Are They Equal?

print("Is reconstructed A close to original A?", np.allclose(A, A_reconstructed))

Output: True

Key Considerations

• Only square matrices can be diagonalized.
• Some matrices are not diagonalizable (e.g., defective matrices).
• Use np.allclose() instead of == for floating-point equality checks.

Conclusion

Eigenvalues and eigenvectors offer a powerful lens to analyze linear transformations. With NumPy, computing and verifying these is just a few lines of code away. By understanding diagonalization, you gain the tools to simplify matrix operations and peek deeper into the nature of systems and data structures.

What’s Next?

In the next part of this module, we’ll explore how to use these techniques to solve systems of equations and power machine learning algorithms like PCA.