The Dot Product

The dot product is a fundamental operation that takes two vectors and produces a single number, a scalar. Unlike operations that combine vectors to yield another vector, this scalar result is incredibly informative, telling us about the relationship between the directions of the two vectors. It is one of the most fundamental operations in machine learning, used for everything from measuring similarity to performing geometric transformations.

The Algebraic Calculation

At its core, the dot product is a straightforward calculation. You multiply the corresponding components of two vectors and then sum up the results. For two vectors $v$ and $w$ of the same dimension $n$, their dot product, denoted as $v \cdot w$, is calculated as follows:

$$v \cdot w = v_1 w_1 + v_2 w_2 + \dots + v_n w_n$$

Let's take two simple 2D vectors as an example: $v = [2, 3]$ and $w = [4, 1]$.

To find their dot product, we multiply the first components together and the second components together, and then add the products:

$v \cdot w = (2 \times 4) + (3 \times 1) = 8 + 3 = 11$

The result is 11, a scalar value. This calculation extends to vectors of any dimension, as long as both vectors have the same number of components.

The Geometric Meaning

The real power of the dot product comes from its geometric interpretation. The value of the dot product is directly related to the angle between the two vectors. The formula that connects them is:

$$v \cdot w = \|v\| \|w\| \cos(\theta)$$

Here, $\|v\|$ and $\|w\|$ are the magnitudes (or norms) of the vectors $v$ and $w$, and $\theta$ (theta) is the angle between them. This equation reveals that the dot product is essentially a measure of how much one vector "points" in the direction of the other.

This relationship allows us to interpret the sign of the dot product's result:

• Positive Dot Product ($v \cdot w > 0$): The angle $\theta$ between the vectors is less than 90°. The vectors are pointing in a generally similar direction.
• Zero Dot Product ($v \cdot w = 0$): The angle $\theta$ is exactly 90°. The vectors are perpendicular to each other. This is a special and very useful property called orthogonality.
• Negative Dot Product ($v \cdot w < 0$): The angle $\theta$ is greater than 90°. The vectors are pointing in generally opposite directions.

The following diagram illustrates the projection of vector $v$ onto vector $w$. The length of this projection is related to the dot product.

The dot product is related to the length of the projection of one vector onto another. A larger projection length in the same direction corresponds to a larger positive dot product.

Application: Measuring Similarity

This geometric meaning makes the dot product a foundation for measuring similarity, a common task in machine learning. By rearranging the geometric formula, we can isolate the angle:

$$\cos(\theta) = \frac{v \cdot w}{\|v\| \|w\|}$$

This formula calculates the cosine similarity between two vectors. It gives a value between -1 and 1, which provides a clean metric for comparison:

• 1: The vectors point in the exact same direction ($\theta = 0°$). They are considered perfectly similar.
• 0: The vectors are orthogonal ($\theta = 90°$). They have no similarity.
• -1: The vectors point in opposite directions ($\theta = 180°$). They are perfectly dissimilar.

For example, in natural language processing, documents can be converted into vectors where each component represents the frequency of a word. By calculating the cosine similarity between these vectors, we can determine how similar two documents are in their subject matter.

Calculating the Dot Product with NumPy

NumPy provides highly efficient functions for performing linear algebra operations. There are two common ways to compute the dot product.

The first is using the numpy.dot() function.

import numpy as np

# Define two vectors as NumPy arrays
v = np.array([2, 3])
w = np.array([4, 1])

# Calculate the dot product using np.dot()
dot_product = np.dot(v, w)

print(f"Vector v: {v}")
print(f"Vector w: {w}")
print(f"Dot Product: {dot_product}")

Running this code will produce the expected output:

Vector v: [2 3]
Vector w: [4 1]
Dot Product: 11

A more modern and Pythonic way to compute the dot product (and perform matrix multiplication, as we'll see later) is with the @ operator. This operator was introduced to make numerical code cleaner and more readable.

import numpy as np

v = np.array([2, 3])
w = np.array([4, 1])

# Calculate the dot product using the @ operator
dot_product_operator = v @ w

print(f"Dot Product with @: {dot_product_operator}")

This code yields the same result, 11. The @ operator is often preferred for its conciseness and clear mathematical intent.