Definitions and Properties

Vectors are fundamental mathematical constructs with versatile applications in fields like physics, engineering, computer graphics, and machine learning. They possess both magnitude and direction, contrasting with scalars that have only magnitude. Geometrically, vectors are represented as arrows in a coordinate space, where the arrow's length denotes its magnitude, and the arrowhead indicates its direction.

Algebraically, vectors are ordered lists of numbers. A 2D vector is written as v = (v₁, v₂), with v₁ and v₂ as components along the x and y axes, respectively. A 3D vector is denoted as v = (v₁, v₂, v₃).

Properties of Vectors

Vectors have several properties that enable mathematical operations:

1. Magnitude (or Norm): The magnitude of a vector v = (v₁, v₂, ..., vₙ) in n-dimensional space is calculated using the Euclidean formula:

   $$||\mathbf{v}|| = \sqrt{v_1^2 + v_2^2 + \cdots + v_n^2}$$

   This property is crucial for determining distances between points in space, a common requirement in algorithms like k-nearest neighbors.

2. Direction: The direction of a vector is the line along which it acts. To obtain a unit vector u in the direction of v, divide each component of v by its magnitude:

   $$\mathbf{u} = \frac{\mathbf{v}}{||\mathbf{v}||}$$

3. Zero Vector: The zero vector 0 has all components as zero, no direction, and a magnitude of zero. It acts as the additive identity: v + 0 = v for any vector v.

4. Equality: Two vectors u and v are equal if and only if their corresponding components are identical: u₁ = v₁, u₂ = v₂, ..., uₙ = vₙ.

Vector Operations

Basic vector operations include:

• Addition and Subtraction: Vectors are added or subtracted component-wise. For u = (u₁, u₂, ..., uₙ) and v = (v₁, v₂, ..., vₙ), their sum and difference are:

  $$\mathbf{u} + \mathbf{v} = (u_1 + v_1, u_2 + v_2, \ldots, u_n + v_n)$$ $$\mathbf{u} - \mathbf{v} = (u_1 - v_1, u_2 - v_2, \ldots, u_n - v_n)$$

• Scalar Multiplication: A vector v = (v₁, v₂, ..., vₙ) is multiplied by a scalar c, scaling its magnitude without altering its direction:

  $$c\mathbf{v} = (cv_1, cv_2, \ldots, cv_n)$$

  This operation is frequently used for scaling features in machine learning.

Significance in Machine Learning

Vectors are indispensable in machine learning as they represent data points. Each feature of a data point is a component of a vector, and the entire dataset is a collection of vectors. This representation allows algorithms to perform operations like calculating distances, finding projections, and optimizing functions.

Mastering vector definitions and properties provides a robust foundation for exploring advanced linear algebra topics and appreciating their critical role in machine learning applications.