Once your autoencoder has been successfully trained, its encoder part has learned a mapping from the input data to a compact, lower-dimensional representation residing in the bottleneck layer. This learned representation is what we're after. The next step is to use this trained encoder to transform your data, both existing and new, into these meaningful features. This process effectively turns your encoder into a dedicated feature extraction tool.
The complete autoencoder model, which includes both the encoder and the decoder, was necessary for the training process. The decoder's role was to reconstruct the input from the bottleneck's representation, thereby forcing the encoder to learn useful features. However, for feature extraction, we only need the encoder.
Deep learning frameworks like PyTorch allow you to easily define your autoencoder in such a way that the encoder component is a separate, callable part of the overall model. This makes extracting features straightforward once the full autoencoder is trained.
Imagine your autoencoder architecture as depicted below. For feature extraction, we essentially use only the encoder component.
The trained autoencoder (top) learns to compress data into a bottleneck representation. For feature extraction (bottom), only the encoder part is used to transform input data into these features.
In PyTorch, the typical way to handle this is to define your encoder as a distinct nn.Sequential
block or nn.Module
within your main Autoencoder
class. Then, after training the full Autoencoder
model, you can directly access its encoder
attribute to perform feature extraction.
Let's assume you have a trained autoencoder
object based on the previous example's Autoencoder
class.
import torch
import torch.nn as nn
import numpy as np
# Re-define the Autoencoder class to ensure encoder is a clear module
class Autoencoder(nn.Module):
def __init__(self, latent_dim=32):
super(Autoencoder, self).__init__()
self.latent_dim = latent_dim
# Encoder part
self.encoder = nn.Sequential(
nn.Linear(784, 128),
nn.ReLU(),
nn.Linear(128, 64),
nn.ReLU(),
nn.Linear(64, latent_dim),
nn.ReLU()
)
# Decoder part
self.decoder = nn.Sequential(
nn.Linear(latent_dim, 64),
nn.ReLU(),
nn.Linear(64, 128),
nn.ReLU(),
nn.Linear(128, 784),
nn.Tanh() # Or Sigmoid, based on your normalization
)
def forward(self, x):
encoded = self.encoder(x)
decoded = self.decoder(encoded)
return decoded
# Create an instance of the autoencoder (assuming latent_dim=32)
autoencoder = Autoencoder(latent_dim=32)
# --- IMPORTANT: Load trained weights here ---
# This is important. Without trained weights, the encoder won't extract meaningful features.
# For demonstration, let's assume you have saved your trained model:
# torch.save(autoencoder.state_dict(), 'autoencoder_mnist.pth')
# autoencoder.load_state_dict(torch.load('autoencoder_mnist.pth'))
# ---------------------------------------------
# Move model to device (CPU/GPU)
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
autoencoder.to(device)
autoencoder.eval() # Set the model to evaluation mode
print("Autoencoder architecture:")
print(autoencoder)
# Example: Prepare some dummy new data (replace with your actual data)
# This dummy data should mimic the shape and preprocessing of your original training data.
# For MNIST, that was flattened 784-dimensional vectors normalized to [-1, 1].
num_new_samples = 5
dummy_new_data = torch.randn(num_new_samples, 784).to(device) # Random data, replace with actual preprocessed data
# Generate the features by passing data through the encoder
with torch.no_grad(): # Disable gradient calculations, as we're not training
extracted_features = autoencoder.encoder(dummy_new_data)
# Move the features back to CPU and convert to NumPy array for further use
extracted_features_np = extracted_features.cpu().numpy()
print(f"\nShape of original (dummy) data: {dummy_new_data.shape}")
print(f"Shape of extracted features: {extracted_features_np.shape}")
The extracted_features_np
array will have N
rows (where N
is the number of input samples) and D_latent
columns (where D_latent
is the dimensionality of your bottleneck layer, in our case 32). Each row is a dense vector, representing the compressed form of the corresponding input sample.
The features you've just extracted have several notable characteristics:
A word of caution that cannot be overstressed: any new data for which you want to extract features must undergo the exact same preprocessing steps (e.g., normalization, scaling, reshaping) that were applied to the data used to train the autoencoder.
If your training data was scaled to a [0, 1] range, new data must also be scaled to [0, 1] using the same parameters (e.g., the min/max values from the training set). If it was standardized (zero mean, unit variance), new data must be standardized using the mean and standard deviation from the training set. Failure to do so will result in the encoder receiving data from a different distribution than it was trained on, leading to extracted features that are likely suboptimal or even meaningless.
Now that you have these compact, learned feature vectors, what's next? These features can be incredibly versatile:
The process of extracting features from the bottleneck is a fundamental step in leveraging autoencoders for feature engineering. With these techniques, you're now equipped to transform raw data into more potent representations for a variety of machine learning applications. In the upcoming hands-on section, you'll apply these steps to a practical example using tabular data.
Was this section helpful?
© 2025 ApX Machine Learning