Solving Ax = b using the Inverse

The matrix equation Ax = b represents a system of linear equations.

$$Ax = b$$

Here, $A$ is the matrix of coefficients (which we know), $x$ is the vector of unknown variables (which we want to find), and $b$ is the vector of constant terms (which we also know). Our goal is to isolate and find the vector $x$.

Think back to simple algebra. If you have an equation like:

$$5x = 10$$

How do you solve for $x$? You'd typically divide both sides by 5. Another way to think about this is multiplying both sides by the multiplicative inverse of 5, which is $\frac{1}{5}$ or $5^{-1}$:

$$5^{-1}(5x) = 5^{-1}(10)$$ $$(5^{-1} \cdot 5)x = \frac{10}{5}$$ $$1 \cdot x = 2$$ $$x = 2$$

The matrix inverse, $A^{-1}$, plays a similar role for the matrix equation $Ax = b$. If the matrix $A$ is invertible (meaning it's square and its inverse $A^{-1}$ exists, as discussed in the previous section), we can use the inverse to solve for $x$.

We start with our equation:

$$Ax = b$$

Now, just like we multiplied by the scalar inverse $5^{-1}$ in the simple example, we can multiply both sides of the matrix equation by the matrix inverse $A^{-1}$. Remember that matrix multiplication is not commutative, so the order matters. Since $x$ is multiplied by $A$ on the left, we need to pre-multiply both sides by $A^{-1}$ (multiply from the left):

$$A^{-1}(Ax) = A^{-1}b$$

Matrix multiplication is associative, meaning we can regroup the terms on the left side:

$$(A^{-1}A)x = A^{-1}b$$

Recall the definition of the matrix inverse: multiplying a matrix by its inverse gives the identity matrix $I$. So, $A^{-1}A = I$:

$$Ix = A^{-1}b$$

And finally, remember the property of the identity matrix: multiplying any matrix or vector by the identity matrix leaves it unchanged ($Ix = x$). Therefore:

$$x = A^{-1}b$$

This gives us a formula to find the solution vector $x$. If we can find the inverse of the coefficient matrix $A$, we can simply multiply it by the constant vector $b$ to get the vector of unknowns $x$.

This theoretical solution is quite elegant. It tells us that if $A$ is invertible, the system $Ax=b$ has a unique solution given by $x = A^{-1}b$. This directly connects the concept of the matrix inverse to solving systems of linear equations.

However, a word of caution: while this formula $x = A^{-1}b$ is fundamental for understanding the theory, explicitly calculating the inverse $A^{-1}$ and then performing the matrix-vector multiplication is often not the most computationally efficient or numerically stable way to solve systems of equations in practice, especially for large systems. Think of it like calculating $10/5$ directly instead of finding $5^{-1}=0.2$ and then computing $0.2 \times 10$. For simple numbers it doesn't matter much, but for matrices, the process of finding the inverse can be computationally expensive and prone to floating-point errors.

In the upcoming sections, we'll see how libraries like NumPy allow us to compute $A^{-1}$, but more importantly, we'll learn about functions specifically designed to solve $Ax = b$ directly and efficiently, often without explicitly forming the inverse matrix. These methods are generally preferred for practical applications.