Hands-On Practical: Matrix Operations in NumPy

Theory provides the rules, but computation is where the work gets done. Performing matrix operations in Python code using the NumPy library allows for practical application of the underlying mechanics. NumPy is the foundation of scientific computing in Python because it provides highly optimized tools for working with arrays, which are perfect for representing vectors and matrices.

In this section, you will learn to perform all the core matrix operations we have discussed. Mastering these NumPy functions is a necessary step for implementing machine learning models from scratch or even for properly preparing data for popular libraries like Scikit-learn and TensorFlow.

Setting Up the Environment

First, we need to import the NumPy library. The standard convention, which you should always follow, is to import it with the alias np.

import numpy as np

Creating NumPy Matrices

We create matrices by calling np.array() with a nested list, where each inner list represents a row. Let's define a few matrices and a vector to use in our examples.

# A 2x3 matrix
A = np.array([[1, 2, 3], 
              [4, 5, 6]])

# A 3x2 matrix
B = np.array([[7, 8], 
              [9, 10], 
              [11, 12]])

# A 2x3 matrix for element-wise operations
C = np.array([[2, 4, 6], 
              [8, 10, 12]])

# A vector (1D array) of size 2
v = np.array([100, 200])

print("Matrix A (2x3):\n", A)
print("\nMatrix B (3x2):\n", B)
print("\nMatrix C (2x3):\n", C)
print("\nVector v:\n", v)

Element-wise Operations

Element-wise operations are performed on corresponding elements between matrices. For these operations to work, the matrices involved must have the exact same dimensions.

Matrix Addition and Subtraction

You can add or subtract matrices of the same shape using the standard + and - operators. NumPy handles the element-by-element calculation automatically.

# Matrix Addition (A and C are both 2x3)
add_result = A + C
print("A + C:\n", add_result)

# Output:
# A + C:
#  [[ 3  6  9]
#  [12 15 18]]

# Matrix Subtraction
subtract_result = C - A
print("C - A:\n", subtract_result)

# Output:
# C - A:
#  [[1 2 3]
#  [4 5 6]]

Attempting to add A (a 2x3 matrix) and B (a 3x2 matrix) would result in a ValueError, as their shapes are not compatible.

Matrix-Scalar Multiplication

To multiply every element of a matrix by a single number (a scalar), you use the * operator.

# Multiply matrix A by the scalar 10
scalar_mult_result = A * 10
print("A * 10:\n", scalar_mult_result)

# Output:
# A * 10:
#  [[10 20 30]
#  [40 50 60]]

Matrix Multiplication

Matrix multiplication is one of the most important operations in linear algebra. It is not an element-wise operation and has special rules regarding matrix dimensions. In NumPy, you should use the @ operator for matrix multiplication.

Matrix-Vector Multiplication

Multiplying a matrix by a vector transforms the vector. Let's transform our vector v using matrix A. This is not possible because the inner dimensions do not match: A has shape (2, 3) and v has shape (2,). We need the number of columns in A to equal the number of elements in v.

Let's use a compatible matrix, D, of shape (2, 2) and multiply it by v.

# A 2x2 matrix
D = np.array([[1, 2],
              [3, 4]])

# Multiply D (2x2) by v (2,)
# Resulting shape will be (2,)
mv_result = D @ v
print("Matrix D:\n", D)
print("\nVector v:\n", v)
print("\nD @ v:\n", mv_result)

# Output:
# Matrix D:
#  [[1 2]
#  [3 4]]
#
# Vector v:
#  [100 200]
#
# D @ v:
#  [ 500 1100]

The calculation performed was $(1 \cdot 100 + 2 \cdot 200)$ for the first element and $(3 \cdot 100 + 4 \cdot 200)$ for the second.

Matrix-Matrix Multiplication

To multiply two matrices, the number of columns in the first matrix must equal the number of rows in the second matrix. For example, we can multiply A (2x3) by B (3x2). The resulting matrix will have a shape of (2x2).

# Multiply A (2x3) by B (3x2)
# Resulting shape will be (2x2)
mm_result = A @ B
print("A @ B:\n", mm_result)

# Output:
# A @ B:
#  [[ 58  64]
#  [139 154]]

Note: A common mistake for beginners is to use the * operator for matrix multiplication. In NumPy, * performs element-wise multiplication. For two matrices to be multiplied element-wise, they must have the same shape. The @ operator was introduced in Python 3.5 specifically for matrix multiplication and is the recommended method.

The Matrix Transpose

Transposing a matrix flips it over its main diagonal, turning rows into columns and vice-versa. In NumPy, you can get the transpose of a matrix using the .T attribute.

# Transpose of A (2x3) will be a 3x2 matrix
A_transpose = A.T
print("Original Matrix A (2x3):\n", A)
print("\nTranspose of A (3x2):\n", A_transpose)

# Output:
# Original Matrix A (2x3):
#  [[1 2 3]
#  [4 5 6]]
#
# Transpose of A (3x2):
#  [[1 4]
#  [2 5]
#  [3 6]]

Creating Special Types of Matrices

NumPy provides convenient functions for creating common matrices used in linear algebra.

Identity Matrix

An identity matrix is a square matrix with ones on the main diagonal and zeros elsewhere. You can create one with np.identity().

# Create a 3x3 identity matrix
I = np.identity(3)
print("3x3 Identity Matrix:\n", I)

# Output:
# 3x3 Identity Matrix:
#  [[1. 0. 0.]
#  [0. 1. 0.]
#  [0. 0. 1.]]

Multiplying any matrix by a compatible identity matrix leaves the original matrix unchanged.

# A is 2x3, I must be 3x3 for A @ I to work
I_3x3 = np.identity(3)
result = A @ I_3x3

print("A @ I:\n", result)
print("\nIs the result the same as A?", np.array_equal(result, A))

# Output:
# A @ I:
#  [[1. 2. 3.]
#  [4. 5. 6.]]
#
# Is the result the same as A? True

Diagonal Matrix

A diagonal matrix has non-zero values only on its main diagonal. You can create one from a list or 1D array of values using np.diag().

# Create a diagonal matrix from a list of values
diag_matrix = np.diag([2, 5, 8])
print("Diagonal Matrix:\n", diag_matrix)

# Output:
# Diagonal Matrix:
#  [[2 0 0]
#  [0 5 0]
#  [0 0 8]]

You have now practiced all the fundamental matrix operations in NumPy. These functions form the toolkit you will use repeatedly to manipulate data and build machine learning algorithms. In the next chapter, we will see how to use these operations to solve systems of linear equations.