Hands-On Practical: Vector Operations in NumPy

Translating the fundamental operations of vectors into code is essential for practical applications. Performing these operations efficiently often involves using libraries like NumPy, the standard library for numerical computation in Python.

First, let's set up our environment by importing the NumPy library. We'll follow the common convention of importing it with the alias np.

import numpy as np

Creating Vectors in NumPy

In NumPy, vectors are represented as one-dimensional arrays. You can create one using the np.array() function, passing in a Python list.

# Create two vectors, v and w
v = np.array([2, 1])
w = np.array([1, 3])

print(f"Vector v: {v}")
print(f"Vector w: {w}")

Output:

Vector v: [2 1]
Vector w: [1 3]

Vector Addition and Subtraction

Performing addition and subtraction in NumPy is straightforward. The library overloads the standard + and - operators to perform element-wise operations on arrays of the same shape.

# Vector addition
v_plus_w = v + w
print(f"v + w = {v_plus_w}")

# Vector subtraction
v_minus_w = v - w
print(f"v - w = {v_minus_w}")

Output:

v + w = [3 4]
v - w = [ 1 -2]

This matches the algebraic rules we discussed. Adding v and w gives a new vector $[2+1, 1+3] = [3, 4]$. Geometrically, this operation can be visualized as placing the tail of one vector at the head of another. The resulting vector connects the starting point to the final point, forming a triangle.

The geometric interpretation of vector addition. The resultant vector v+w is the diagonal of the parallelogram formed by v and w.

Scalar Multiplication

To multiply a vector by a scalar, we use the * operator. This operation scales the length of the vector. Multiplying by a positive scalar changes its magnitude, while a negative scalar reverses its direction in addition to changing its magnitude.

# Define a vector and a scalar
u = np.array([3, 2])
s = 2.5

# Perform scalar multiplication
scaled_u = s * u
print(f"Vector u: {u}")
print(f"Scalar s: {s}")
print(f"s * u = {scaled_u}")

Output:

Vector u: [3 2]
Scalar s: 2.5
s * u = [7.5 5. ]

As you can see, each element of the vector u is multiplied by the scalar s. The resulting vector [7.5, 5] points in the same direction as u but is 2.5 times longer.

The Dot Product

The dot product is one of the most important operations in linear algebra, especially for machine learning. It gives us a single number that relates to the angle between two vectors. NumPy provides a few ways to compute it.

The np.dot() function is a common method:

v = np.array([2, 1])
w = np.array([1, 3])

# Calculate the dot product using np.dot()
dot_product = np.dot(v, w)
print(f"The dot product of v and w is: {dot_product}")

Output:

The dot product of v and w is: 5

The calculation is $(2 \times 1) + (1 \times 3) = 2 + 3 = 5$.

Alternatively, since Python 3.5, you can use the @ operator, which is designated for matrix multiplication and also works for the dot product of vectors. This operator often makes the code more readable.

# Calculate the dot product using the @ operator
dot_product_operator = v @ w
print(f"The dot product using @ is: {dot_product_operator}")

Output:

The dot product using @ is: 5

Vector Norms: Measuring Length

The norm of a vector is a measure of its length or magnitude. The np.linalg.norm() function is the tool for this job. By default, it calculates the $L_2$ norm (Euclidean norm).

u = np.array([3, 4])

# Calculate the L2 norm (default)
l2_norm = np.linalg.norm(u)
print(f"The L2 norm of u is: {l2_norm}")

Output:

The L2 norm of u is: 5.0

This is calculated as $\sqrt{3^2 + 4^2} = \sqrt{9 + 16} = \sqrt{25} = 5$.

To calculate other norms, like the $L_1$ norm (Manhattan distance), you can specify the ord parameter.

# Calculate the L1 norm
l1_norm = np.linalg.norm(u, ord=1)
print(f"The L1 norm of u is: {l1_norm}")

Output:

The L1 norm of u is: 7.0

This is the sum of the absolute values of the elements: $|3| + |4| = 7$.

Checking for Orthogonal Vectors

Two vectors are orthogonal if their dot product is zero. This is a simple and effective test. Let's define two vectors that should be orthogonal and check.

# Define two orthogonal vectors
vec1 = np.array([1, 0])
vec2 = np.array([0, 1])

# Calculate their dot product
dot_prod = vec1 @ vec2
print(f"The dot product of vec1 and vec2 is: {dot_prod}")

Output:

The dot product of vec1 and vec2 is: 0

A dot product of 0 confirms they are orthogonal. Now let's try with two non-orthogonal vectors.

# Define two non-orthogonal vectors
vec3 = np.array([1, 1])
vec4 = np.array([1, -0.5])

# Calculate their dot product
dot_prod_non_ortho = vec3 @ vec4
print(f"The dot product of vec3 and vec4 is: {dot_prod_non_ortho}")

Output:

The dot product of vec3 and vec4 is: 0.5

Since the result is not zero, the vectors vec3 and vec4 are not orthogonal.

Application: Finding the Angle Between Two Vectors

We can combine these operations to solve interesting problems. For example, we can find the angle $\theta$ between two vectors using the formula derived from the dot product definition:

$$\cos(\theta) = \frac{\mathbf{v} \cdot \mathbf{w}}{\|\mathbf{v}\| \|\mathbf{w}\|}$$

Which means:

$$\theta = \arccos\left(\frac{\mathbf{v} \cdot \mathbf{w}}{\|\mathbf{v}\| \|\mathbf{w}\|}\right)$$

Let's implement this in NumPy.

v = np.array([2, 1])
w = np.array([1, 3])

# Calculate the dot product
dot_vw = v @ w

# Calculate the norms
norm_v = np.linalg.norm(v)
norm_w = np.linalg.norm(w)

# Calculate the cosine of the angle
cos_theta = dot_vw / (norm_v * norm_w)

# Calculate the angle in radians
angle_rad = np.arccos(cos_theta)

# Convert to degrees for easier interpretation
angle_deg = np.degrees(angle_rad)

print(f"Dot product: {dot_vw:.2f}")
print(f"Norm of v: {norm_v:.2f}")
print(f"Norm of w: {norm_w:.2f}")
print(f"Angle between v and w (radians): {angle_rad:.2f}")
print(f"Angle between v and w (degrees): {angle_deg:.2f}")

Output:

Dot product: 5.00
Norm of v: 2.24
Norm of w: 3.16
Angle between v and w (radians): 0.64
Angle between v and w (degrees): 36.87

This practical example shows how the dot product and norms work together to provide meaningful geometric information. Mastering these NumPy operations is the first step toward implementing more advanced machine learning algorithms.