Measuring Similarity with Dot Products

The dot product is a simple calculation that offers an effective way to measure how "alike" two vectors are. This task is fundamental to search engines, recommendation systems, and many other machine learning applications. When items like documents, movies, or customer preferences are represented as vectors, their orientation in space carries significant meaning.

From Algebra to Geometry

Recall the two ways we can define the dot product between two vectors, $a$ and $b$:

1. Algebraic Definition: The sum of the element-wise products. $$a \cdot b = \sum_{i=1}^{n} a_i b_i$$
2. Geometric Definition: The product of their lengths and the cosine of the angle between them. $$a \cdot b = \|a\| \|b\| \cos(\theta)$$

The geometric definition holds the secret to measuring similarity. The term $\cos(\theta)$ directly tells us about the angle, $\theta$, between the two vectors. This angle is a great indicator of their directional similarity, regardless of their lengths.

Let's look at what the value of $\cos(\theta)$ means:

• If $\cos(\theta) = 1$, the angle $\theta$ is $0^\circ$. The vectors point in the exact same direction. They are maximally similar.
• If $\cos(\theta) = 0$, the angle $\theta$ is $90^\circ$. The vectors are orthogonal (perpendicular). They share no directional similarity.
• If $\cos(\theta) = -1$, the angle $\theta$ is $180^\circ$. The vectors point in opposite directions. They are maximally dissimilar.

The values between these extremes represent varying degrees of similarity. A value of $0.8$ indicates a much higher similarity than a value of $0.2$.

The cosine of the angle between vectors provides a direct measure of their directional alignment.

Cosine Similarity

To isolate this directional measure, we can rearrange the geometric formula to solve for $\cos(\theta)$. This gives us the cosine similarity formula, which is one of the most widely used similarity metrics in machine learning.

$$\text{similarity} = \cos(\theta) = \frac{a \cdot b}{\|a\| \|b\|}$$

The formula computes the dot product and then divides it by the product of the two vectors' norms (their lengths). This division is a form of normalization. It ensures the resulting value is always between -1 and 1, regardless of how large the vector components are. It effectively asks: "Ignoring the magnitude, how much do these vectors point in the same direction?"

This is especially useful in fields like Natural Language Processing (NLP). Imagine you represent two documents as vectors where each element corresponds to a word count. One document might be very long and the other very short. Their vector magnitudes would be very different, but if they are about the same topic, their vectors will point in a similar direction in the high-dimensional word space. Cosine similarity will capture this topical similarity while ignoring the difference in document length.

Example: Finding Similar Users

Let's apply this to a simple recommendation system problem. Suppose we have rating data for three users on two movie genres: Sci-Fi and Comedy. The ratings are from 1 to 5.

• User A loves Sci-Fi, dislikes Comedy: [5, 1]
• User B likes Sci-Fi, dislikes Comedy: [4, 1]
• User C dislikes Sci-Fi, loves Comedy: [1, 5]

Intuitively, User A and User B have similar tastes, while User C has very different tastes from both A and B. Let's verify this with cosine similarity. We can create a simple function in Python using NumPy to perform the calculation.

import numpy as np

# User ratings for [Sci-Fi, Comedy]
user_a = np.array([5, 1])
user_b = np.array([4, 1])
user_c = np.array([1, 5])

# Function to calculate cosine similarity
def cosine_similarity(v1, v2):
    dot_product = np.dot(v1, v2)
    norm_v1 = np.linalg.norm(v1)
    norm_v2 = np.linalg.norm(v2)
    return dot_product / (norm_v1 * norm_v2)

# Calculate similarities
sim_ab = cosine_similarity(user_a, user_b)
sim_ac = cosine_similarity(user_a, user_c)

print(f"Similarity between User A and User B: {sim_ab:.4f}")
print(f"Similarity between User A and User C: {sim_ac:.4f}")

Running this code produces the following output:

Similarity between User A and User B: 0.9992
Similarity between User A and User C: 0.3846

The results match our intuition perfectly. The similarity score between User A and User B is very close to 1, indicating their preferences are strongly aligned. In contrast, the similarity between User A and User C is much lower, reflecting their different tastes. A recommendation engine could use this logic to suggest a movie liked by User A to User B, but not to User C.

This simple example demonstrates how a core linear algebra operation, the dot product, becomes a practical tool for comparing data points and making intelligent predictions in machine learning systems.