Yandex

Course IndexCourse Index0

    ProgramGuru

    Eigenvalues, Eigenvectors, and Diagonalization in NumPy
    Linear Algebra Tutorial


    Understanding Eigenvalues and Eigenvectors

    In linear algebra, eigenvalues and eigenvectors play a crucial role in decomposing matrices, solving systems of equations, and even powering algorithms in machine learning and computer graphics.

    If you're new to this concept, think of eigenvectors as special directions that remain unchanged (except for scaling) when a transformation (matrix) is applied. The eigenvalue tells us how much the vector gets scaled.

    Why Do Eigenvalues and Eigenvectors Matter?

    These concepts are foundational in:

    • Principal Component Analysis (PCA) for dimensionality reduction
    • Stability analysis in systems
    • Quantum mechanics and vibration modes

    And in this tutorial, you’ll learn to compute and interpret them using NumPy.

    Step 1: Import NumPy

    import numpy as np

    Step 2: Define a Square Matrix

    A = np.array([[4, 2],
                [1, 3]])

    We're working with a 2x2 matrix here. Eigen decomposition is only defined for square matrices, so always verify the matrix shape using:

    print("Matrix shape:", A.shape)

    Step 3: Compute Eigenvalues and Eigenvectors

    eigenvalues, eigenvectors = np.linalg.eig(A)

    Explanation: np.linalg.eig() returns two results:

    • eigenvalues: A 1D array of eigenvalues.
    • eigenvectors: A 2D array where each column is an eigenvector corresponding to an eigenvalue.

    Step 4: Print and Understand the Output

    print("Eigenvalues:", eigenvalues)
    print("Eigenvectors:
    ", eigenvectors)

    Sample Output:

    
    Eigenvalues: [5. 2.]
    Eigenvectors:
    [[ 0.89442719 -0.70710678]
    [ 0.4472136   0.70710678]]

    What Does This Output Tell Us?

    There are two eigenvalues: 5 and 2. Each one has a corresponding eigenvector:

    • The first eigenvector [0.894, 0.447] is scaled by 5 when multiplied by the matrix A.
    • The second eigenvector [-0.707, 0.707] is scaled by 2.

    Step 5: Verify the Eigenvalue Equation

    The property of eigenvectors is:

    A * v = λ * v

    Let’s verify this manually for one eigenvalue-eigenvector pair:

    # Pick the first eigenvector and eigenvalue
    v = eigenvectors[:, 0]
    λ = eigenvalues[0]
    
    Av = A @ v
    λv = λ * v
    
    print("A @ v:", Av)
    print("λ * v:", λv)

    Expected Output: The two results should be approximately equal (allowing for minor floating-point differences).

    Step 6: Diagonalization

    A matrix A is diagonalizable if:

    A = P * D * P-1

    Where:

    • P is the matrix of eigenvectors
    • D is the diagonal matrix of eigenvalues

    Let's Perform Diagonalization

    P = eigenvectors
    D = np.diag(eigenvalues)
    P_inv = np.linalg.inv(P)
    
    A_reconstructed = P @ D @ P_inv
    
    print("Reconstructed A:
    ", A_reconstructed)
    print("Original A:
    ", A)

    Explanation: The reconstructed matrix A_reconstructed should be nearly identical to the original matrix A. This confirms diagonalizability.

    Numerical Check: Are They Equal?

    print("Is reconstructed A close to original A?", np.allclose(A, A_reconstructed))

    Output: True

    Key Considerations

    • Only square matrices can be diagonalized.
    • Some matrices are not diagonalizable (e.g., defective matrices).
    • Use np.allclose() instead of == for floating-point equality checks.

    Conclusion

    Eigenvalues and eigenvectors offer a powerful lens to analyze linear transformations. With NumPy, computing and verifying these is just a few lines of code away. By understanding diagonalization, you gain the tools to simplify matrix operations and peek deeper into the nature of systems and data structures.

    What’s Next?

    In the next part of this module, we’ll explore how to use these techniques to solve systems of equations and power machine learning algorithms like PCA.



    Welcome to ProgramGuru

    Sign up to start your journey with us

    Support ProgramGuru.org

    You can support this website with a contribution of your choice.

    When making a contribution, mention your name, and programguru.org in the message. Your name shall be displayed in the sponsors list.

    PayPal

    UPI

    PhonePe QR

    MALLIKARJUNA M