Vector Addition and Subtraction

Vectors are fundamental mathematical objects that can be represented using structures like NumPy arrays. Understanding basic operations on these vectors is essential for many applications. This content focuses on two primary operations: vector addition and subtraction. These operations work element-wise, meaning they operate on the corresponding components of the vectors involved.

Vector Addition

Adding two vectors is straightforward. You simply add the corresponding elements of each vector together. For this operation to be valid, the vectors must have the same number of elements, meaning they must have the same dimension or length.

Mathematically, if you have two $n$-dimensional vectors, $\vec{u}$ and $\vec{v}$:

$$\vec{u} = [u_1, u_2, \dots, u_n]$$ $$\vec{v} = [v_1, v_2, \dots, v_n]$$

Their sum, $\vec{w} = \vec{u} + \vec{v}$, is calculated as:

$$\vec{w} = [u_1 + v_1, u_2 + v_2, \dots, u_n + v_n]$$

For example, let's add two 3-dimensional vectors: $\vec{u} = [1, 2, 3]$ $\vec{v} = [4, 5, 6]$

The sum is: $\vec{u} + \vec{v} = [1+4, 2+5, 3+6] = [5, 7, 9]$

Geometric Interpretation

Geometrically, vector addition can be visualized by placing the tail of the second vector ($\vec{v}$) at the head of the first vector ($\vec{u}$). The resulting vector sum ($\vec{u} + \vec{v}$) starts at the origin (tail of $\vec{u}$) and ends at the head of the repositioned $\vec{v}$. This is often called the "head-to-tail" rule. Alternatively, it forms the diagonal of a parallelogram defined by the two original vectors when both start from the origin.

Vector addition using the head-to-tail method. Vector $\vec{u}$ (blue) is added to vector $\vec{v}$ (pink, shifted). The resultant vector $\vec{u}+\vec{v}$ (green) goes from the origin to the tip of the shifted $\vec{v}$. Dashed lines show the parallelogram completion.

Implementation in NumPy

NumPy makes vector addition very intuitive. You can use the standard + operator directly on NumPy arrays representing your vectors. NumPy automatically handles the element-wise addition.

import numpy as np

# Define two vectors as NumPy arrays
u = np.array([1, 2, 3])
v = np.array([4, 5, 6])

# Add the vectors
w = u + v

print(f"Vector u: {u}")
print(f"Vector v: {v}")
print(f"Sum u + v: {w}")

Output:

Vector u: [1 2 3]
Vector v: [4 5 6]
Sum u + v: [5 7 9]

Remember, if you try to add vectors of different lengths, NumPy will raise a ValueError because the element-wise operation cannot be performed.

# Example of incompatible shapes
a = np.array([1, 2])
b = np.array([3, 4, 5])

try:
    c = a + b
except ValueError as e:
    print(f"Error adding vectors a and b: {e}")

Output:

Error adding vectors a and b: operands could not be broadcast together with shapes (2,) (3,)

Vector Subtraction

Vector subtraction works similarly to addition. You subtract the corresponding elements of the second vector from the first vector. Again, both vectors must have the same dimension.

If you have two $n$-dimensional vectors, $\vec{u}$ and $\vec{v}$:

$$\vec{u} = [u_1, u_2, \dots, u_n]$$ $$\vec{v} = [v_1, v_2, \dots, v_n]$$

Their difference, $\vec{d} = \vec{u} - \vec{v}$, is calculated as:

$$\vec{d} = [u_1 - v_1, u_2 - v_2, \dots, u_n - v_n]$$

For example, using the same vectors $\vec{u}$ and $\vec{v}$ as before: $\vec{u} = [1, 2, 3]$ $\vec{v} = [4, 5, 6]$

The difference is: $\vec{u} - \vec{v} = [1-4, 2-5, 3-6] = [-3, -3, -3]$

Geometric Interpretation

Geometrically, subtracting $\vec{v}$ from $\vec{u}$ ($\vec{u} - \vec{v}$) is equivalent to adding the negative of $\vec{v}$ to $\vec{u}$ ($\vec{u} + (-\vec{v})$). The vector $-\vec{v}$ has the same magnitude as $\vec{v}$ but points in the opposite direction. Another way to visualize $\vec{u} - \vec{v}$ is as the vector that starts at the head of $\vec{v}$ and ends at the head of $\vec{u}$, when both $\vec{u}$ and $\vec{v}$ start from the origin.

Vector subtraction using the $\vec{u} + (-\vec{v})$ method. Vector $\vec{u}$ (blue) is added to vector $-\vec{v}$ (pink, shifted). The resultant vector $\vec{u}-\vec{v}$ (green) goes from the origin to the tip of the shifted $-\vec{v}$.

Implementation in NumPy

Similar to addition, vector subtraction in NumPy uses the standard - operator.

import numpy as np

# Define two vectors
u = np.array([1, 2, 3])
v = np.array([4, 5, 6])

# Subtract vector v from vector u
d = u - v

print(f"Vector u: {u}")
print(f"Vector v: {v}")
print(f"Difference u - v: {d}")

# Subtract vector u from vector v
d_reversed = v - u
print(f"Difference v - u: {d_reversed}")

Output:

Vector u: [1 2 3]
Vector v: [4 5 6]
Difference u - v: [-3 -3 -3]
Difference v - u: [3 3 3]

Notice that, unlike addition, the order matters in subtraction ($\vec{u} - \vec{v}$ is generally not the same as $\vec{v} - \vec{u}$). Vector addition is commutative ($\vec{u} + \vec{v} = \vec{v} + \vec{u}$), but subtraction is not.

Vector addition and subtraction are fundamental tools in linear algebra. They form the basis for many operations used in machine learning algorithms, such as calculating error vectors or updating feature weights. Understanding how they work, both mathematically and in code using libraries like NumPy, is an important step in building your linear algebra foundation.