Singular vs. Non-Singular Matrices

In the previous section, we learned that the determinant of a matrix is a single number that holds significant information. Now we will see how that number helps us classify matrices into two distinct and important categories: non-singular and singular. This classification directly tells us about the kinds of solutions we can expect from a system of linear equations.

Non-Singular Matrices: The Path to a Unique Solution

A square matrix that has an inverse is called a non-singular or invertible matrix. The most direct way to identify one is by its determinant.

A matrix $A$ is non-singular if its determinant is not zero.

$$det(A) \neq 0$$

When the matrix $A$ in our system $Ax = b$ is non-singular, it means a single, unique solution for the vector $x$ exists. This is the ideal scenario. Because the inverse $A^{-1}$ exists, we can use it to solve the system cleanly:

$$x = A^{-1}b$$

Geometrically, a non-singular matrix transforms a space without losing any of its dimensions. For example, a 2D non-singular matrix might rotate, stretch, or shear a square into a parallelogram, but it will always remain a 2D shape with a positive area. The transformation can be fully reversed, which is what the inverse matrix does.

A non-singular matrix transforms a square into a parallelogram. The area is changed, but it doesn't collapse into a line. The transformation is reversible.

Singular Matrices: The Case of No Unique Solution

On the other hand, a square matrix that does not have an inverse is called a singular or non-invertible matrix.

A matrix $A$ is singular if its determinant is exactly zero.

$$det(A) = 0$$

If the matrix $A$ in $Ax = b$ is singular, our method of using an inverse immediately fails because $A^{-1}$ does not exist. This indicates a problem with the system of equations itself. It means the system has either no solution or infinitely many solutions. It will never have one unique solution.

Geometrically, a singular matrix squashes space into a lower dimension. For example, a 2D singular matrix will collapse a 2D plane of vectors onto a single line or even a single point. This action is not reversible. Once you've flattened a 2D shape into a line, there's no information left to tell you how to "un-flatten" it back to its original shape.

A singular matrix collapses a 2D square into a 1D line. Information is lost, and the transformation cannot be reversed.

Why This Distinction Matters in Machine Learning

In a machine learning algorithm like linear regression, a singular matrix often points to a problem with the input data. It can occur if two features are redundant. For example, if you have one feature for temperature in Celsius and another for temperature in Fahrenheit, they are perfectly linearly dependent. The matrix representing your data would be singular, and the standard method for solving linear regression would fail because it relies on a matrix inverse. Recognizing a singular matrix allows you to diagnose such problems in your dataset.

Summary of Differences

This table provides a quick reference for the properties of non-singular and singular matrices.

Feature	Non-Singular Matrix	Singular Matrix
Invertible?	Yes	No
Determinant	Not zero ($det(A) \neq 0$)	Zero ($det(A) = 0$)
Solution to Ax = b	One unique solution	No solution or infinite solutions
Geometric Effect	Preserves dimensions	Collapses dimensions
Linearly Independent	Columns (and rows) are independent	Columns (and rows) are dependent

Checking for Singularity in Python

We can use NumPy to easily determine if a matrix is singular by calculating its determinant.

Let's define a non-singular matrix first.

import numpy as np

# A non-singular matrix
A_non_singular = np.array([
    [3, 1],
    [1, 2]
])

det_A = np.linalg.det(A_non_singular)
print(f"Matrix A:\n{A_non_singular}")
print(f"Determinant of A: {det_A:.2f}")

Output:

Matrix A:
[[3 1]
 [1 2]]
Determinant of A: 5.00

Since the determinant is 5.0 (not zero), this matrix is non-singular and invertible.

Now let's try a singular matrix.

# A singular matrix
B_singular = np.array([
    [2, 4],
    [1, 2]
])

det_B = np.linalg.det(B_singular)
print(f"Matrix B:\n{B_singular}")
print(f"Determinant of B: {det_B}")

Output:

Matrix B:
[[2 4]
 [1 2]]
Determinant of B: 0.0

The determinant is 0.0, confirming that this matrix is singular.

A Note on Floating-Point Precision

When using computers, calculations may not be perfectly precise. A matrix that is theoretically singular might have a determinant calculated as a very small number, like 1.4e-17, instead of exactly 0. For this reason, in practical code, you should not check if the determinant is exactly equal to zero. Instead, check if it is very close to zero.

# A near-singular matrix due to floating point representation
C = np.array([
    [1, 2],
    [1, 2.000000000000001] 
])

det_C = np.linalg.det(C)
print(f"Determinant of C: {det_C}")

# Check if the determinant is close to zero
is_singular = np.isclose(det_C, 0)
print(f"Is the matrix C effectively singular? {is_singular}")

Output:

Determinant of C: 8.881784197001252e-16
Is the matrix C effectively singular? True

Using np.isclose() is a much more reliable way to test for singularity in numerical computations.