What is a Matrix?

Vectors represent ordered lists of numbers. A matrix extends this idea by arranging numbers not just in a single list, but in a rectangular grid. Picture a spreadsheet or a checkerboard: this type of rectangular arrangement of numbers, symbols, or expressions is essentially what defines a matrix.

A matrix (plural: matrices) is a rectangular array or table of numbers, symbols, or expressions, arranged in rows (horizontal lines) and columns (vertical lines). They are fundamental tools in linear algebra and are used extensively in machine learning for organizing data, representing transformations, and much more.

Anatomy of a Matrix: Rows, Columns, and Elements

Think of a matrix as a collection of numbers organized neatly. For example, consider this matrix:

$$A = \begin{bmatrix} 1 & 0 & 7 \\ -3 & 4 & -2 \\ 5 & 2 & 0 \end{bmatrix}$$

This matrix has:

• Rows: Three horizontal lines of numbers.
  • Row 1: [1 0 7]
  • Row 2: [-3 4 -2]
  • Row 3: [5 2 0]
• Columns: Three vertical lines of numbers.
  • Column 1: [1 -3 5]ᵀ (The T means transpose, often used to write column vectors horizontally)
  • Column 2: [0 4 2]ᵀ
  • Column 3: [7 -2 0]ᵀ
• Elements (or Entries): The individual numbers within the matrix. This matrix has $3 \times 3 = 9$ elements.

The dimensions or size of a matrix are given by the number of rows and the number of columns, typically written as "rows $\times$ columns". The matrix $A$ above is a $3 \times 3$ matrix (read as "three by three").

Here's another example:

$$B = \begin{bmatrix} 2.1 & -0.5 \\ 1.0 & 3.7 \\ 0 & 1.1 \\ 4.2 & 0 \end{bmatrix}$$

This matrix $B$ has 4 rows and 2 columns, so it is a $4 \times 2$ matrix.

Connecting Matrices and Vectors

You can think of vectors (which we discussed in the previous chapter) as special cases of matrices:

• A column vector is a matrix with only one column (an $m \times 1$ matrix). $$\begin{bmatrix} 1 \\ -3 \\ 5 \end{bmatrix} \quad \text{(This is a } 3 \times 1 \text{ matrix)}$$
• A row vector is a matrix with only one row (a $1 \times n$ matrix). $$\begin{bmatrix} 1 & 0 & 7 \end{bmatrix} \quad \text{(This is a } 1 \times 3 \text{ matrix)}$$

This connection helps in understanding how operations involving vectors and matrices relate to each other.

Why Use Matrices?

Matrices provide a powerful way to represent and manipulate collections of numbers simultaneously. In machine learning, they are constantly used:

• Datasets: A typical dataset can be represented as a matrix where each row corresponds to a data sample (like a customer or an image) and each column corresponds to a feature (like age, price, or pixel intensity).
• Images: A grayscale image can be represented as a matrix where each element is the intensity of a pixel. A color image often uses multiple matrices (one for each color channel: Red, Green, Blue).
• Linear Transformations: Matrices can represent operations like rotations, scaling, and shearing of geometric objects (represented by vectors).
• Model Parameters: The weights and biases learned by many machine learning models (like linear regression or neural networks) are often stored and manipulated as matrices.

The structured nature of matrices allows us to define consistent operations (like addition and multiplication, which we'll cover soon) that apply to entire blocks of data efficiently.

Visualizing a Generic Matrix

We often use capital letters (like $A$, $B$, $C$) to denote matrices. To refer to a specific element within a matrix, we use lowercase letters with two subscripts: $a_{ij}$. The first subscript $i$ indicates the row number, and the second subscript $j$ indicates the column number.

So, for a general $m \times n$ matrix $A$ (meaning $m$ rows and $n$ columns), we can write it as:

$$A = \begin{bmatrix} a_{11} & a_{12} & \cdots & a_{1n} \\ a_{21} & a_{22} & \cdots & a_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ a_{m1} & a_{m2} & \cdots & a_{mn} \end{bmatrix}$$

Here, $a_{21}$ is the element in the 2nd row and 1st column. $a_{mn}$ is the element in the last row ($m$) and last column ($n$).

A general $m \times n$ matrix $A$, showing the element $a_{ij}$ located at the $i$-th row and $j$-th column.

Important Note on Indexing: In mathematics, matrix indices typically start from 1 (so the top-left element is $a_{11}$). However, in many programming languages, including Python (and its library NumPy which we'll use extensively), indexing starts from 0. So, the element $a_{11}$ in math corresponds to A[0, 0] in NumPy, and $a_{ij}$ corresponds to A[i-1, j-1]. This is a common point of confusion, so keep it in mind when translating mathematical concepts into code.

Now that we understand what a matrix is and how it's structured, we'll look at how to denote matrix dimensions more formally and explore some important special types of matrices.