Just as we can perform arithmetic with single numbers (scalars), we can also perform arithmetic with vectors. These operations form the foundation for many algorithms in machine learning, from calculating error to adjusting model parameters. We'll begin with the two most straightforward operations: addition and subtraction.The Algebra and Geometry of Vector AdditionAdding two vectors is an element-wise operation. This means you simply add the corresponding components of each vector to get the components of the new, resulting vector. For this to work, the vectors must have the same dimension, meaning they must have the same number of components.Let's say we have two two-dimensional vectors, $v$ and $w$:$$ v = \begin{bmatrix} v_1 \ v_2 \end{bmatrix}, \quad w = \begin{bmatrix} w_1 \ w_2 \end{bmatrix} $$Their sum, $v + w$, is calculated as:$$ v + w = \begin{bmatrix} v_1 + w_1 \ v_2 + w_2 \end{bmatrix} $$For a concrete example, if $v = \begin{bmatrix} 2 \ 1 \end{bmatrix}$ and $w = \begin{bmatrix} 1 \ 3 \end{bmatrix}$, their sum is:$$ v + w = \begin{bmatrix} 2 + 1 \ 1 + 3 \end{bmatrix} = \begin{bmatrix} 3 \ 4 \end{bmatrix} $$While the algebra is simple, the geometric interpretation provides a deeper understanding. To visualize vector addition, we use the tip-to-tail method. You draw the first vector starting from the origin. Then, you draw the second vector starting from the tip (or head) of the first vector. The sum of the two vectors is the new vector that starts at the origin and ends at the tip of the second vector.digraph G { rankdir=TB; splines=false; bgcolor="transparent"; node [shape=point, style=invis]; edge [arrowhead=vee]; 0 -> "v" [label=" v = [2, 1]", color="#4263eb", fontcolor="#4263eb", penwidth=2]; "v" -> "v+w" [label=" w = [1, 3]", color="#12b886", fontcolor="#12b886", penwidth=2, style=dashed]; 0 -> "v+w" [label=" v+w = [3, 4]", color="#ae3ec9", fontcolor="#ae3ec9", penwidth=2.5]; 0 [label=""]; {rank = same; 0;}}The tip-to-tail method for vector addition. The resulting vector, $v+w$, represents the combined movement of both vectors.Another way to see this is the parallelogram law. If you draw both vectors $v$ and $w$ starting from the origin, they form two adjacent sides of a parallelogram. Their sum, $v+w$, is the diagonal of that parallelogram that also starts at the origin. Notice that this gives the same result, as $v+w = w+v$. This is known as the commutative property of vector addition.Subtracting Vectors: Finding the DifferenceVector subtraction also works element-wise, just like addition. To subtract vector $w$ from vector $v$, you subtract the components of $w$ from the corresponding components of $v$. Again, the vectors must have the same dimension.$$ v - w = \begin{bmatrix} v_1 - w_1 \ v_2 - w_2 \end{bmatrix} $$Using our previous vectors, $v = \begin{bmatrix} 3 \ 4 \end{bmatrix}$ and $w = \begin{bmatrix} 1 \ 3 \end{bmatrix}$, the subtraction is:$$ v - w = \begin{bmatrix} 3 - 1 \ 4 - 3 \end{bmatrix} = \begin{bmatrix} 2 \ 1 \end{bmatrix} $$Geometrically, the easiest way to think about subtraction is as adding the opposite. The vector $-w$ is a vector with the same length as $w$ but pointing in the exact opposite direction. So, the operation $v - w$ is equivalent to $v + (-w)$. We can apply the same tip-to-tail rule: draw vector $v$, and from its tip, draw the vector $-w$.digraph G { rankdir=TB; splines=false; bgcolor="transparent"; node [shape=point, style=invis]; edge [arrowhead=vee]; 0 -> "v" [label=" v = [3, 4]", color="#4263eb", fontcolor="#4263eb", penwidth=2]; "v" -> "v-w" [label=" -w = [-1, -3]", color="#f03e3e", fontcolor="#f03e3e", penwidth=2, style=dashed]; 0 -> "v-w" [label=" v-w = [2, 1]", color="#ae3ec9", fontcolor="#ae3ec9", penwidth=2.5]; 0 [label=""]; {rank = same; 0;}}Vector subtraction as the addition of an opposite vector. The vector $v-w$ is the result of moving along $v$ and then along $-w$.This operation is useful for finding the vector that points from one point to another. The vector from the point defined by $w$ to the point defined by $v$ is exactly $v - w$.Vector Arithmetic in Python with NumPyPerforming these operations by hand is good for understanding, but in practice, we use libraries like NumPy to do the work. NumPy's arrays are a perfect match for representing vectors, and the standard arithmetic operators (+ and -) perform element-wise addition and subtraction automatically.First, let's import NumPy and create two vectors.import numpy as np v = np.array([2, 1]) w = np.array([1, 3]) print(f"Vector v: {v}") print(f"Vector w: {w}")Now, we can add them together with a simple + operation.# Vector addition sum_vw = v + w print(f"v + w = {sum_vw}")Output:Vector v: [2 1] Vector w: [1 3] v + w = [3 4]Subtraction is just as easy using the - operator.# Vector subtraction diff_vw = sum_vw - w print(f"[3, 4] - w = {diff_vw}")Output:[3, 4] - w = [2 1]As you can see, NumPy handles the component-wise arithmetic for you. This is not only convenient but also highly optimized for performance, especially when working with vectors that have thousands or even millions of dimensions, which is common in machine learning.