Many problems in computation involve finding values for a set of unknown variables that simultaneously satisfy several linear relationships. Think about finding the right price for a product based on supply and demand equations, or determining the optimal allocation of resources under budget constraints. These situations often lead to systems of linear equations.

Let's consider a simple example system with two equations and two unknown variables, $x_1$ and $x_2$ :

$2x_1 + 3x_2 = 7$ $1x_1 - 1x_2 = 1$

Our goal is to find the values of $x_1$ and $x_2$ that make both equations true. While we could solve this using methods like substitution or elimination from basic algebra, linear algebra provides a more systematic and scalable approach, especially when dealing with many equations and variables. The first step is to represent this system using matrices and vectors.

We can separate the system into three distinct parts:

The coefficients of the variables (the numbers multiplying $x_1$ and $x_2$ ).
The variables themselves ( $x_1$ , $x_2$ ).
The constants on the right-hand side of the equations (7 and 1).

Let's arrange these parts into matrix and vector structures.

The Coefficient Matrix (A)

We collect the coefficients of the variables into a matrix, which we'll call $A$ . Each row in the matrix corresponds to one equation, and each column corresponds to one variable.

For our example system: $2x_1 + 3x_2 = 7$ $1x_1 - 1x_2 = 1$

The coefficients are 2, 3, 1, and -1. We arrange them as:

A = \begin{bmatrix} 2 & 3 \\ 1 & -1 \end{bmatrix}

The first row $[2 \quad 3]$ contains the coefficients from the first equation. The second row $[1 \quad -1]$ contains the coefficients from the second equation. The first column $\begin{bmatrix} 2 \\ 1 \end{bmatrix}$ contains the coefficients of $x_1$ , and the second column $\begin{bmatrix} 3 \\ -1 \end{bmatrix}$ contains the coefficients of $x_2$ .

The Variable Vector (x)

The unknown variables are arranged into a column vector, typically denoted as $x$ :

x = \begin{bmatrix} x_1 \\ x_2 \end{bmatrix}

The Constant Vector (b)

The constants on the right-hand side of the equations are also arranged into a column vector, often denoted as $b$ :

b = \begin{bmatrix} 7 \\ 1 \end{bmatrix}

Assembling the Matrix Equation: Ax = b

Now, how do these pieces fit together? Recall the definition of matrix multiplication from the previous chapter. Let's multiply our coefficient matrix $A$ by our variable vector $x$ :

Ax = \begin{bmatrix} 2 & 3 \\ 1 & -1 \end{bmatrix} \begin{bmatrix} x_1 \\ x_2 \end{bmatrix}

Performing the matrix-vector multiplication (dot product of each row of $A$ with the column vector $x$ ):

Ax = \begin{bmatrix} (2 \times x_1) + (3 \times x_2) \\ (1 \times x_1) + (-1 \times x_2) \end{bmatrix} = \begin{bmatrix} 2x_1 + 3x_2 \\ x_1 - x_2 \end{bmatrix}

Look closely at the resulting vector. Its first element, $2x_1 + 3x_2$ , is exactly the left-hand side of our first original equation. Its second element, $x_1 - x_2$ , is the left-hand side of our second original equation.

The original system stated that these expressions must equal the constants 7 and 1, respectively. We captured these constants in the vector $b$ . Therefore, we can write the entire system of equations as a single matrix equation:

Ax = b

Substituting our matrices and vectors:

\begin{bmatrix} 2 & 3 \\ 1 & -1 \end{bmatrix} \begin{bmatrix} x_1 \\ x_2 \end{bmatrix} = \begin{bmatrix} 7 \\ 1 \end{bmatrix}

This compact form, $Ax = b$ , perfectly represents the original system of linear equations.

General Form

This representation isn't limited to two equations. Any system of $m$ linear equations in $n$ unknowns:

$a_{11}x_1 + a_{12}x_2 + \dots + a_{1n}x_n = b_1$ $a_{21}x_1 + a_{22}x_2 + \dots + a_{2n}x_n = b_2$ $\vdots$ $a_{m1}x_1 + a_{m2}x_2 + \dots + a_{mn}x_n = b_m$

can be written in the matrix form $Ax = b$ , where:

$A$ is the $m \times n$ coefficient matrix: $A = \begin{bmatrix} a_{11} & a_{12} & \dots & a_{1n} \\ a_{21} & a_{22} & \dots & a_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ a_{m1} & a_{m2} & \dots & a_{mn} \end{bmatrix}$
$x$ is the $n \times 1$ column vector of variables: $x = \begin{bmatrix} x_1 \\ x_2 \\ \vdots \\ x_n \end{bmatrix}$
$b$ is the $m \times 1$ column vector of constants: $b = \begin{bmatrix} b_1 \\ b_2 \\ \vdots \\ b_m \end{bmatrix}$

This $Ax = b$ format is standard in linear algebra. It separates the known coefficients ( $A$ ), the unknown variables ( $x$ ), and the target outcomes ( $b$ ). This structure is advantageous because it allows us to apply the powerful operations and concepts of matrix algebra, such as matrix inversion (which we'll discuss shortly), to analyze and solve these systems efficiently. Representing data and relationships in this form is common in various fields, including machine learning, where you might encounter it in algorithms like linear regression.