As mentioned in the chapter introduction, enabling computers to process and understand the meaning behind data like text, images, or audio requires translating this information into a format they can work with: numbers. This section revisits the fundamental concept of representing complex, often unstructured data as numerical vectors, commonly known as embeddings.

Think about how you might represent a single word numerically. You could assign a unique ID, but that wouldn't capture any relationship between words. "Dog" (ID 5) and "Cat" (ID 12) would seem just as unrelated as "Dog" (ID 5) and "Software" (ID 98). We need something richer.

Vector embeddings provide this richness. The core idea is to map each piece of data, whether it's a word, a sentence, an entire document, an image, or even a song clip, to a point in a multi-dimensional geometric space. This point is represented by a vector, which is essentially a list of numbers.

For example, a piece of text like "semantic search" might be transformed into a vector $\vec{v}$ like:

\vec{v} = [0.12, -0.45, 0.88, ..., -0.05]

This vector typically has many dimensions, often hundreds or even thousands. Each number in the vector represents a coordinate along one of the dimensions in this high-dimensional space.

Why go through this transformation? The power lies in the relationships between these vectors. Well-constructed embeddings place items with similar meanings closer together in this vector space.

Text: The vector for "dog" would likely be closer to the vector for "puppy" or "canine" than it would be to the vector for "computer" or "cloud". Similarly, sentences like "How old are you?" and "What is your age?" should result in vectors that are near each other.
Images: Embeddings for images of different breeds of cats would cluster together in the vector space, distinctly separate from embeddings of cars or landscapes.
Other Data: This concept extends to audio (similar sounds or spoken phrases), user profiles (similar interests or behaviors), and more.

The process of generating these meaningful vectors relies on sophisticated embedding models, often based on deep learning architectures like transformers, which we will explore in the next section. For now, the important concept to grasp is this transformation:

Data (Text, Image, Audio, etc.) → Embedding Model → Numerical Vector (Embedding)

These vectors capture latent features and semantic nuances of the original data. The geometry of the vector space reflects the underlying meaning. Points that are close together represent data items that are semantically similar. This property is fundamental to semantic search, recommendation systems, and many other AI applications, as it allows us to perform mathematical operations, like calculating distances or angles between vectors, to quantify similarity.

This refresher sets the stage for understanding how these vectors are created and utilized. We've established that we can represent diverse data numerically in a way that captures meaning. Next, we'll look at the specific models responsible for generating these powerful vector representations.