Executing the Training Process

Your autoencoder model, defined and configured with an optimizer and loss function, is now ready for its training phase. Training is the process where the autoencoder learns to perform its task, which, for a basic autoencoder, is to reconstruct the input data as accurately as possible. This learning happens by iteratively adjusting the model's internal parameters (weights) based on the reconstruction error it makes. In Keras, the primary tool for this is the fit method.

Starting the Training with model.fit()

The fit method is your main interface for instructing your Keras model to learn from the data. You'll provide your training data, specify a few training parameters, and Keras will handle the complex underlying process of iteration and weight adjustment.

Here’s a common way you’d call model.fit() to train an autoencoder:

# Assume 'autoencoder' is your compiled Keras model
# 'x_train_processed' is your preprocessed training data
# 'x_val_processed' is your preprocessed validation data (a subset of your data not used for direct training)

# Define some training settings
num_epochs = 50       # How many times to go through the entire training dataset
batch_size_val = 128  # How many samples to process before updating the model's weights

# Let's train the model!
history = autoencoder.fit(
    x_train_processed,  # Input data for training
    x_train_processed,  # Target data: for an autoencoder, this is the same as the input
    epochs=num_epochs,
    batch_size=batch_size_val,
    shuffle=True,       # It's good practice to shuffle training data at the start of each epoch
    validation_data=(x_val_processed, x_val_processed) # Data to evaluate loss on at the end of each epoch
)

Let's look at what these arguments mean:

• First argument (x_train_processed in our example): This is your input training data. The autoencoder will take this data, pass it through the encoder to get a compressed representation, and then through the decoder to try and reconstruct it.
• Second argument (also x_train_processed here): This is the "target" data. Since the goal of our autoencoder is to reconstruct its input, the input data itself serves as the desired output. The model's learning process aims to minimize the difference between its actual output and this target.
• epochs=num_epochs: An epoch represents one full pass through your entire training dataset. If you have 1,000 training samples and a batch_size of 100, then 10 batches would make up one epoch. Training for multiple epochs (e.g., 50 or 100) gives the model repeated opportunities to see the data and refine its weights. Think of it like re-reading a chapter in a textbook; each pass can help solidify your understanding.
• batch_size=batch_size_val: Instead of feeding the entire dataset to the model at once (which can be too large for computer memory), the data is divided into smaller segments called batches. The batch_size defines the number of samples in each batch. After processing each batch, the model calculates the error and updates its weights. For instance, with 1,000 training samples and a batch_size of 32, one epoch would involve approximately $1000 / 32 \\approx 32$ sets of weight updates.
• shuffle=True: This setting tells Keras to shuffle the order of the training samples before starting each epoch. Shuffling helps prevent the model from inadvertently learning patterns related to the order of data presentation, which can lead to a model that generalizes better to new, unseen data. It's generally a good idea to keep this enabled.
• validation_data=(x_val_processed, x_val_processed): This argument is optional but highly recommended for good practice. You provide a separate dataset (the validation set) that the model does not train on. After each epoch of training on the x_train_processed data, Keras will evaluate your model's performance (specifically, the loss) on this validation_data. This allows you to monitor how well your model is generalizing to data it hasn't seen during training. If the model performs well on training data but poorly on validation data, it might be overfitting. For an autoencoder, the validation input and target are also the same.

What Happens During fit?

When you call model.fit(), Keras initiates and manages the learning loop. For each epoch, and for each batch within that epoch:

1. The model takes a batch of data (e.g., from x_train_processed).
2. It performs a forward pass: The input data flows through the encoder network, is compressed into the bottleneck representation, and then flows through the decoder network to produce a reconstructed output.
3. The loss function (like the Mean Squared Error you specified when you compiled the model) calculates how different the reconstructed output is from the original input batch. This difference is often called the "reconstruction error."
4. The optimizer uses this error value to determine how to adjust the model's internal weights. This adjustment is done using an algorithm called backpropagation (which we touched upon in "Learning from Errors: Backpropagation (High-Level)" in Chapter 3), aiming to reduce the error in subsequent predictions.
5. This cycle of forward pass, loss calculation, and weight update repeats for all batches in the current epoch. Then, Keras moves to the next epoch, repeating the process until all specified epochs are completed.

Monitoring Training with the History Object

The model.fit() method doesn't just train your model; it also returns a History object. This object is quite valuable because it records what happened during training. Specifically, it contains the loss values (and any other metrics you asked Keras to track) at the end of each epoch, for both the training data and, if provided, the validation data.

You can access this information from the history.history attribute, which is a dictionary:

# 'history' is the object returned by autoencoder.fit()

training_loss_values = history.history['loss']
validation_loss_values = history.history['val_loss']

# If you had included other metrics like 'mean_absolute_error' during model.compile():
# training_mae = history.history['mean_absolute_error']
# validation_mae = history.history['val_mean_absolute_error']

# Let's print the loss after the first and last epochs
print(f"Loss on training data after first epoch: {training_loss_values[0]:.4f}")
print(f"Loss on validation data after first epoch: {validation_loss_values[0]:.4f}")
print(f"Loss on training data after last epoch: {training_loss_values[-1]:.4f}")
print(f"Loss on validation data after last epoch: {validation_loss_values[-1]:.4f}")

Visualizing Training Progress

While numbers are precise, a graph often tells a clearer story. Plotting the training and validation loss over epochs is a standard way to assess how the training is progressing.

Plot of training loss and validation loss over epochs. Observing these trends helps in understanding model performance.

Interpreting the Loss Curves:

• A Good Sign: You generally want to see both the training loss and the validation loss decrease over epochs and then stabilize or converge. It's common for the validation loss to be slightly higher than the training loss. This indicates your model is learning and generalizing.
• Overfitting: A common issue is overfitting. This occurs when your model learns the training data too well, including its noise and specific quirks, but fails to generalize to new, unseen data. On the plot, you'd see the training loss continuing to decrease, but the validation loss might flatten out or, worse, start to increase. This divergence is a red flag for overfitting. We briefly discussed this in "A Glimpse into Overfitting and Underfitting" in Chapter 3.
• Underfitting: If both training and validation loss remain high or decrease very slowly and then plateau at a high value, your model might be underfitting. This could suggest that the model is too simple to capture the underlying patterns in the data, it hasn't been trained for enough epochs, or perhaps the learning rate of your optimizer needs adjustment.

The training process can take a few moments or much longer, depending on the size of your dataset, the complexity of your model, and the number of epochs. Keras will typically print updates during training, showing the loss for each epoch. This allows you to monitor progress in real time.

Once model.fit() completes, your autoencoder model's weights have been updated through the learning process. It has now learned, to the best of its ability given the data and architecture, how to compress your input data into the bottleneck layer and then reconstruct it. The next steps, which we'll cover in "Assessing Reconstruction Quality" and "Visualizing Reconstructed Outputs: Hands-on Practical," involve evaluating how well it learned this task.