Creating Tensors

Tensors are fundamental data structures in PyTorch, representing the multi-dimensional arrays used in deep learning models. Practical methods for creating tensors within your code are available. PyTorch offers a flexible set of functions for creating tensors, whether using existing data or requiring specific shapes and initial values.

Creating Tensors from Existing Data

The most direct way to create a tensor is often from existing Python data structures like lists or NumPy arrays. The primary function for this is torch.tensor(). This function intelligently infers the data type (dtype) from the input data, but you can also specify it explicitly. Importantly, torch.tensor() always copies the input data.

Let's see it in action. First, ensure you have PyTorch imported:

import torch
import numpy as np

Now, create a tensor from a Python list:

# Create a tensor from a Python list
data_list = [[1, 2], [3, 4]]
tensor_from_list = torch.tensor(data_list)

print("Tensor from list:")
print(tensor_from_list)
print(f"Data type: {tensor_from_list.dtype}")
print(f"Shape: {tensor_from_list.shape}")

This will output:

Tensor from list:
tensor([[1, 2],
        [3, 4]])
Data type: torch.int64
Shape: torch.Size([2, 2])

Notice how PyTorch inferred the data type as torch.int64 (a 64-bit integer) based on the input integers.

You can achieve the same result starting with a NumPy array:

# Create a tensor from a NumPy array
data_numpy = np.array([[5.0, 6.0], [7.0, 8.0]])
tensor_from_numpy = torch.tensor(data_numpy)

print("\nTensor from NumPy array:")
print(tensor_from_numpy)
print(f"Data type: {tensor_from_numpy.dtype}")
print(f"Shape: {tensor_from_numpy.shape}")

Output:

Tensor from NumPy array:
tensor([[5., 6.],
        [7., 8.]], dtype=torch.float64)
Data type: torch.float64
Shape: torch.Size([2, 2])

Here, the data type was inferred as torch.float64 because the NumPy array contained floating-point numbers. We will discuss the relationship between PyTorch tensors and NumPy arrays, including more efficient conversion methods, later in this chapter.

Creating Tensors with Specific Shapes and Values

Often, you need to create tensors of a certain size without having the data beforehand, perhaps initializing them with zeros, ones, or random values. PyTorch provides dedicated functions for these common scenarios. The shape is usually passed as a tuple or a sequence of integers.

Tensors of Zeros, Ones, or Uninitialized Data:

• torch.zeros(*size, ...): Creates a tensor filled with zeros.
• torch.ones(*size, ...): Creates a tensor filled with ones.
• torch.empty(*size, ...): Creates a tensor with the specified size, but its elements are uninitialized. The values will be whatever was in that memory location at the time of allocation. Using torch.empty can be slightly faster if you plan to fill the tensor immediately afterward, but be cautious as the initial values are indeterminate.

# Define the desired shape
shape = (2, 3) # 2 rows, 3 columns

# Create tensors with specific values
zeros_tensor = torch.zeros(shape)
ones_tensor = torch.ones(shape)
empty_tensor = torch.empty(shape) # Values are arbitrary

print(f"\nZeros Tensor (shape {shape}):")
print(zeros_tensor)

print(f"\nOnes Tensor (shape {shape}):")
print(ones_tensor)

print(f"\nEmpty Tensor (shape {shape}):")
print(empty_tensor)

Output (note that the empty_tensor values will vary):

Zeros Tensor (shape (2, 3)):
tensor([[0., 0., 0.],
        [0., 0., 0.]])

Ones Tensor (shape (2, 3)):
tensor([[1., 1., 1.],
        [1., 1., 1.]])

Empty Tensor (shape (2, 3)):
tensor([[2.1321e-38, 5.0837e+24, 3.0763e-41],
        [3.0763e-41, 1.2326e-32, 1.8750e+00]])

By default, these functions create tensors with dtype=torch.float32. You can specify a different dtype using the dtype argument:

# Create a tensor of ones with integer type
ones_int_tensor = torch.ones(shape, dtype=torch.int32)
print(f"\nOnes Tensor (dtype=torch.int32):")
print(ones_int_tensor)

Output:

Ones Tensor (dtype=torch.int32):
tensor([[1, 1, 1],
        [1, 1, 1]], dtype=torch.int32)

Tensors with Random Values:

Initializing model parameters often involves starting with random values. PyTorch provides functions for this:

• torch.rand(*size, ...): Creates a tensor with elements sampled uniformly from the interval [0, 1).
• torch.randn(*size, ...): Creates a tensor with elements sampled from a standard normal distribution (mean 0, variance 1).

# Create tensors with random values
rand_tensor = torch.rand(shape) # Uniform distribution [0, 1)
randn_tensor = torch.randn(shape) # Standard normal distribution

print(f"\nRandom Tensor (Uniform [0, 1), shape {shape}):")
print(rand_tensor)

print(f"\nRandom Tensor (Standard Normal, shape {shape}):")
print(randn_tensor)

Output (values will differ on each run):

Random Tensor (Uniform [0, 1), shape (2, 3)):
tensor([[0.6580, 0.5089, 0.1642],
        [0.3742, 0.5989, 0.7775]])

Random Tensor (Standard Normal, shape (2, 3)):
tensor([[-0.2651, -0.3249, -1.0134],
        [ 1.1314,  1.1751, -0.1411]])

Creating Tensors Based on Other Tensors

Sometimes, you need to create a new tensor that has the same properties (like shape and dtype) as an existing tensor. This is useful for ensuring compatibility in operations. PyTorch offers _like variants for this purpose:

• torch.zeros_like(input_tensor, ...)
• torch.ones_like(input_tensor, ...)
• torch.rand_like(input_tensor, ...)
• torch.randn_like(input_tensor, ...)

These functions take an existing tensor as input and return a new tensor with the specified content (zeros, ones, random) but matching the input tensor's shape and dtype, unless explicitly overridden.

# Use an existing tensor as a template
base_tensor = torch.tensor([[1, 2], [3, 4]], dtype=torch.float32)
print(f"\nBase Tensor (shape {base_tensor.shape}, dtype {base_tensor.dtype}):")
print(base_tensor)

# Create tensors matching the base tensor's properties
zeros_like_base = torch.zeros_like(base_tensor)
rand_like_base = torch.rand_like(base_tensor)

print("\nZeros tensor like base:")
print(zeros_like_base)
print(f"Shape: {zeros_like_base.shape}, dtype: {zeros_like_base.dtype}")

print("\nRandom tensor like base:")
print(rand_like_base)
print(f"Shape: {rand_like_base.shape}, dtype: {rand_like_base.dtype}")

Output:

Base Tensor (shape torch.Size([2, 2]), dtype torch.float32):
tensor([[1., 2.],
        [3., 4.]])

Zeros tensor like base:
tensor([[0., 0.],
        [0., 0.]])
Shape: torch.Size([2, 2]), dtype: torch.float32

Random tensor like base:
tensor([[0.1216, 0.5908],
        [0.3264, 0.9272]])
Shape: torch.Size([2, 2]), dtype: torch.float32

As you can see, the new tensors zeros_like_base and rand_like_base automatically inherited the shape (torch.Size([2, 2])) and dtype (torch.float32) from base_tensor.

These methods provide a comprehensive toolkit for generating the tensors you need, forming the foundation for the computations performed in deep learning models. In the next sections, we will explore how to manipulate these tensors and perform essential operations.