Vectors: Points in Space

A single piece of information is often called a scalar, but most data isn't that simple. Consider describing a house for a real estate website. You would not just list its price. Instead, you would list its size, the number of bedrooms, and the number of bathrooms. An ordered list of numbers like this is a vector.

A vector is a fundamental way to store data that has multiple attributes. If a house is 1,500 square feet, has 3 bedrooms, and 2 bathrooms, we can represent it with the vector $\mathbf{x}$:

$$\mathbf{x} = [1500, 3, 2]$$

Each number in the vector is an element or a component. The order matters. The vector $[3, 2, 1500]$ would represent a completely different house, one with 3 square feet, 2 bedrooms, and 1,500 bathrooms, which does not make much sense.

Geometric Interpretation

Being a simple container for numbers, vectors have a powerful geometric meaning. For simplicity, let's consider a vector with just two components, like $\mathbf{v} = [3, 4]$. We can visualize this in a two-dimensional (2D) coordinate system.

There are two common ways to think about this:

1. As a Point: The vector $\mathbf{v} = [3, 4]$ can represent the coordinates of a single point in space. You go 3 units along the x-axis and 4 units along the y-axis.
2. As an Arrow: The vector can also be seen as an arrow starting from the origin $(0, 0)$ and ending at the point $(3, 4)$.

This arrow interpretation is especially useful because it gives the vector two distinct properties: a magnitude (how long is the arrow?) and a direction (where is the arrow pointing?).

The vector $\mathbf{v} = [3, 4]$ visualized as both an arrow from the origin and a point in a 2D plane.

In machine learning, we often work with data in many dimensions, far more than the two or three we can easily visualize. A vector representing a 28x28 pixel grayscale image, for example, would have $28 \times 28 = 784$ components. Even though we cannot draw a 784-dimensional space, the geometric ideas of length and direction still apply, and they are central to many algorithms.

Notation

In textbooks and mathematical papers, you will see a few common notations for vectors.

• Lowercase, Bold Letters: Vectors are often written as lowercase, bold letters, like $\mathbf{v}$ or $\mathbf{x}$. This is the convention we will follow.
• Column vs. Row Vectors: By default, in linear algebra, a vector is often assumed to be a column vector.

A column vector arranges its components vertically:

$$\mathbf{v} = \begin{bmatrix} 3 \\ 4 \end{bmatrix}$$

A row vector arranges its components horizontally, as we have been writing them so far:

$$\mathbf{v} = [3, 4]$$

This distinction becomes important when we start multiplying vectors with matrices. For now, just recognize that both forms exist. The number of components in a vector determines its dimension. The vector $[3, 4]$ is two-dimensional, while $[1500, 3, 2]$ is three-dimensional.

Vectors provide a way to package the features of a single data point. But a typical dataset contains many data points. To organize an entire collection of vectors, we need our next building block: the matrix.