Examining Encoded Data: Practice

An autoencoder reconstructs images from datasets like MNIST. While the reconstructed images demonstrate the autoencoder's capability, understanding the underlying compressed representation is equally important. This is where the "feature learning" aspect truly comes into play. The bottleneck layer of an autoencoder holds this compressed, or "encoded," version of the input data.

In this section, you'll practice accessing and examining these encoded representations. This will give you a more direct look at what the autoencoder has learned about the data.

Understanding the Encoded Data

Remember, an autoencoder has two main parts: an encoder and a decoder.

1. The encoder takes the input data (like an MNIST digit image) and compresses it down into a much smaller set of numbers. This compressed version lives in the "bottleneck" layer.
2. The decoder then takes this compressed representation and tries to reconstruct the original input from it.

The output of the encoder, right at the bottleneck, is what we call the "encoded data" or "latent representation." It's a compact summary of the input, capturing its most important features as determined by the network during training. For an MNIST image of $28 \times 28 = 784$ pixels, the bottleneck might only have, say, 32 numbers. These 32 numbers are the learned features.

Accessing the Encoded Representation: Creating an Encoder Model

The autoencoder model you built in the previous sections is designed to output the final, reconstructed image. To get the data from the bottleneck layer, we need to create a new, simpler model that consists only of the encoder part of our full autoencoder.

In Keras, this is straightforward. If you named your layers when building the autoencoder (which is a good practice!), you can easily grab any layer's output. Let's assume your full autoencoder model is named autoencoder, and the bottleneck layer (the final layer of your encoder) was named bottleneck_layer.

Here’s how you can define a new model, let's call it encoder_model, that takes the same input as your autoencoder but outputs the activations from the bottleneck_layer:

import os
os.environ["KERAS_BACKEND"] = "torch"
import keras
from keras.models import Model

# Assume 'autoencoder' is your already trained autoencoder model
# Assume 'bottleneck_layer_name' is the name you gave to your bottleneck layer
# For example, if your bottleneck layer was defined as:
# Dense(32, activation='relu', name='bottleneck_layer_name')

# Create the encoder model
encoder_model = Model(inputs=autoencoder.input,
                      outputs=autoencoder.get_layer('bottleneck_layer_name').output)

If you didn't name your bottleneck layer, you might need to find it by its index or by reconstructing the encoder part explicitly. Naming layers makes this process much cleaner. For instance, if your encoder part was Dense(784) -> Dense(128) -> Dense(64) -> Dense(32, name='bottleneck_layer_name'), then bottleneck_layer_name would be the string you use.

Generating and Inspecting Encoded Data

Once you have your encoder_model, you can use its predict method, just like with any other Keras model, to get the encoded representations for your input data. Let's try this with a few images from your test set (e.g., x_test if you're using MNIST).

# Let's take a few samples from x_test to encode
# Ensure x_test is preprocessed (flattened and normalized) as it was for training
num_samples_to_encode = 5
x_test_sample = x_test[:num_samples_to_encode]

# Get the encoded representations
encoded_imgs = encoder_model.predict(x_test_sample)

print("Shape of encoded images:", encoded_imgs.shape)
# This will print something like (5, 32) if your bottleneck has 32 units
# and you used 5 samples.

The encoded_imgs.shape will tell you two things: the number of samples you encoded and the dimensionality of your bottleneck (the number of learned features per sample).

Now, let's look at the actual values for one or two of these encoded images:

print("\nEncoded representation for the first image:")
print(encoded_imgs[0])

if num_samples_to_encode > 1:
    print("\nEncoded representation for the second image:")
    print(encoded_imgs[1])

You'll see an array of numbers for each image. For a bottleneck of 32 dimensions, each image from your input is now represented by just 32 floating-point numbers. These numbers are the "features" that the autoencoder has learned. Their exact meaning can be abstract, but they represent patterns and characteristics that the autoencoder found useful for reconstructing the original images. If your encoder used ReLU activation functions, these values will be non-negative.

Making Sense of Encoded Vectors

These arrays of numbers are the core of what the autoencoder learns. Each vector (e.g., encoded_imgs[0]) is a point in a lower-dimensional "latent space." The autoencoder's goal was to map similar inputs to nearby points in this latent space and different inputs to points that are further apart, in a way that allows the decoder to reconstruct them.

Visualizing a 32-dimensional space directly is impossible for us humans! However, we can still gain some insights:

1. Comparing Vectors: If you take two different input images (say, an MNIST digit '0' and a digit '7'), their encoded vectors should look different. The specific values in their 32-dimensional representations will vary. This difference is what allows the decoder to distinguish them and reconstruct them correctly.

   Let's imagine you have the encoded vector for a '0' and a '7'. You can plot these vectors as bar charts to see how their feature activations differ. For this example, we'll use some placeholder data for what these vectors might look like. Assume encoded_vector_digit_0 and encoded_vector_digit_7 are NumPy arrays, each with 32 values, obtained from encoder_model.predict().

   # Example:
   # encoded_vector_digit_0 = encoder_model.predict(image_of_a_zero)[0]
   # encoded_vector_digit_7 = encoder_model.predict(image_of_a_seven)[0]
   
   # For demonstration, let's use some made-up data:
   # (In your actual code, you'd use the real output from encoder_model.predict)
   example_encoded_0 = [0.0, 1.2, 0.0, 0.5, 2.1, 0.0, 0.0, 0.8, 1.5, 0.0, 0.2, 0.0, 2.5, 0.3, 0.0, 1.1,
                    0.0, 0.0, 1.9, 0.0, 0.6, 0.0, 1.3, 0.0, 0.0, 2.2, 0.0, 0.9, 0.0, 0.0, 1.7, 0.0]
   example_encoded_7 = [1.8, 0.0, 0.7, 0.0, 0.0, 1.2, 2.0, 0.0, 0.0, 0.5, 1.6, 0.0, 0.0, 2.3, 0.0, 0.1,
                    1.4, 0.0, 0.0, 2.6, 0.0, 0.9, 0.0, 1.1, 0.0, 0.0, 1.0, 0.0, 2.4, 0.0, 0.0, 0.4]
   feature_indices = list(range(len(example_encoded_0))) # Should be 32 in this example
   

   A comparison of encoded vectors for two different digits. Notice how the pattern of activations (the heights of the bars) differs, indicating that the autoencoder represents them distinctly in its learned feature space.

2. Visualizing Lower-Dimensional Embeddings: If you had designed your autoencoder with a very small bottleneck, say just 2 dimensions, you could plot these 2D points on a scatter plot. Each point would represent an input image, and you could color-code them by their actual class (e.g., by digit label for MNIST). Ideally, you'd see clusters of points corresponding to different digits.

   While your current autoencoder might have more than 2 bottleneck dimensions, here's what such a 2D plot might look like to give you the idea:

   A scatter plot of data points in a 2D latent space. Different colors/shapes could represent different classes of input data (e.g., different MNIST digits). Clear separation between clusters would indicate that the autoencoder has learned to distinguish these classes well.

   Even if your bottleneck has more than two dimensions (like 32), techniques like t-SNE or UMAP (which are more advanced topics) can be used to project these higher-dimensional encoded vectors down to 2D or 3D for visualization. The main idea is that the structure in the latent space (how points are arranged) reflects what the autoencoder has learned about the similarities and differences in your data.

What This Means for Feature Learning

By examining the encoded data, you're directly observing the features your autoencoder has learned.

• The autoencoder automatically discovers a way to represent complex input data (like an image) using a much smaller set of numbers.
• These numbers, or features, capture important characteristics of the data because they are optimized to allow the decoder to reconstruct the original input.
• The fact that different inputs lead to different encoded vectors (as seen in the bar chart comparison) is evidence of this learning.

This process of automatically deriving useful features from raw data is a foundation of deep learning. While we might not always be able to assign a simple human-understandable label to each learned feature (e.g., "this feature detects curves," "this one detects straight lines"), their collective pattern effectively encodes the input.

This hands-on experience of building an autoencoder, assessing its reconstructions, and now examining its internal encoded representations should solidify your understanding of how these networks operate and learn. In more advanced applications, these learned features can be extracted and used for other machine learning tasks, such as classification or anomaly detection, often leading to better performance than using raw data alone.