Hands-on Practical: Training a Simple Classifier

Building a simple neural network involves several steps: compiling it with appropriate settings, training it on synthetic data, monitoring its learning process, and evaluating its performance. This practical example demonstrates the complete workflow for defining, compiling, training, and evaluating a Keras model.

Objective

Our goal is to train a binary classifier. We will generate a synthetic dataset that isn't linearly separable, build a simple feed-forward neural network, train it using the fit() method, observe its training progress, and assess how well it learned using the evaluate() method.

Setting Up

First, ensure you have the necessary libraries installed. We'll use keras, numpy for numerical operations, scikit-learn to generate our synthetic dataset and split it, and plotly for visualization (if you want to run the plotting code).

import numpy as np
import keras
from keras import layers
from sklearn.datasets import make_moons
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
import plotly.graph_objects as go
import plotly.io as pio

# Optional: Set default theme for plotly plots
pio.templates.default = "plotly_white"

1. Data Generation and Preparation

We need data to train our classifier. The make_moons function from scikit-learn is excellent for this, as it creates two interleaving half-circles, a dataset that requires a non-linear decision boundary, making it suitable for a neural network.

# Generate synthetic data
X, y = make_moons(n_samples=1000, noise=0.2, random_state=42)

# Split data into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Scale the features (important for many neural network tasks)
scaler = StandardScaler()
X_train = scaler.fit_transform(X_train)
X_test = scaler.transform(X_test)

print(f"Shape of training data: {X_train.shape}")
print(f"Shape of training labels: {y_train.shape}")
print(f"Shape of test data: {X_test.shape}")
print(f"Shape of test labels: {y_test.shape}")
print(f"Sample data point (scaled): {X_train[0]}")
print(f"Sample label: {y_train[0]}")

This code generates 1000 data points, each with 2 features (like coordinates on a 2D plane). It adds some noise to make the classification slightly challenging. We then split it into 80% for training and 20% for testing. Finally, we scale the features using StandardScaler. Scaling ensures that all features contribute more evenly to the distance calculations within the network and can help optimization algorithms converge faster.

2. Building the Neural Network Model

Now, let's define a simple sequential model using the Keras API, similar to what we learned in Chapter 2. Our network will have:

• An input layer implicit in the first Dense layer, expecting 2 features.
• One hidden Dense layer with 8 neurons and the ReLU activation function. ReLU is a common choice for hidden layers.
• An output Dense layer with 1 neuron (for binary classification) and the Sigmoid activation function. Sigmoid outputs a value between 0 and 1, which can be interpreted as a probability for the positive class.

# Define the model architecture using the Sequential API
model = keras.Sequential(
    [
        layers.Input(shape=(2,)), # Specify input shape explicitly
        layers.Dense(8, activation="relu", name="hidden_layer"),
        layers.Dense(1, activation="sigmoid", name="output_layer")
    ]
)

# Display the model's architecture
model.summary()

The model.summary() gives a quick overview of the layers, their output shapes, and the number of trainable parameters (weights and biases) in each layer.

3. Compiling the Model

Before training, we must compile the model. This step configures the learning process. As discussed in this chapter, we need to specify:

• Optimizer: The algorithm used to update the network's weights. Adam is a popular and generally effective choice.
• Loss Function: Measures how well the model is doing on the training data. Since this is a binary classification problem and our output layer uses a sigmoid activation, binary_crossentropy is the appropriate loss function.
• Metrics: Used to monitor the training and testing steps. We'll use accuracy to see the percentage of correctly classified samples.

# Compile the model
model.compile(optimizer='adam',
              loss='binary_crossentropy',
              metrics=['accuracy'])

print("Model compiled successfully.")

4. Training the Model

Now we train the model using the fit() method. We provide the training data (X_train, y_train), specify the number of epochs (passes over the entire dataset), and the batch_size (number of samples processed before the model's weights are updated). We also provide the test set as validation_data to monitor the model's performance on unseen data during training. This helps us identify potential overfitting.

The fit() method returns a History object, which contains a record of training loss, validation loss, training accuracy, and validation accuracy for each epoch.

# Train the model
print("Starting training...")
history = model.fit(X_train,
                    y_train,
                    epochs=50,           # Number of passes through the entire dataset
                    batch_size=32,       # Number of samples per gradient update
                    validation_data=(X_test, y_test), # Data to evaluate loss and metrics on at the end of each epoch
                    verbose=1)           # Set verbose=1 to see progress per epoch, 0 for silent
print("Training finished.")

# Access training history
print("\nTraining History Keys:", history.history.keys())

Setting verbose=1 shows the progress for each epoch, including the loss and accuracy on both the training and validation sets.

5. Monitoring Training Performance

The history object is very useful. Let's plot the training and validation loss and accuracy over the epochs to visualize the learning process.

# Extract data from history object
history_dict = history.history
loss_values = history_dict['loss']
val_loss_values = history_dict['val_loss']
acc_values = history_dict['accuracy']
val_acc_values = history_dict['val_accuracy']
epochs_range = range(1, len(loss_values) + 1)

# Create plot for Loss
fig_loss = go.Figure()
fig_loss.add_trace(go.Scatter(x=list(epochs_range), y=loss_values,
                             mode='lines+markers', name='Training Loss',
                             line=dict(color='#4263eb'), marker=dict(color='#4263eb')))
fig_loss.add_trace(go.Scatter(x=list(epochs_range), y=val_loss_values,
                             mode='lines+markers', name='Validation Loss',
                             line=dict(color='#f76707'), marker=dict(color='#f76707')))
fig_loss.update_layout(title='Training and Validation Loss',
                       xaxis_title='Epochs',
                       yaxis_title='Loss',
                       width=700, height=400)

# Create plot for Accuracy
fig_acc = go.Figure()
fig_acc.add_trace(go.Scatter(x=list(epochs_range), y=acc_values,
                            mode='lines+markers', name='Training Accuracy',
                            line=dict(color='#1c7ed6'), marker=dict(color='#1c7ed6')))
fig_acc.add_trace(go.Scatter(x=list(epochs_range), y=val_acc_values,
                            mode='lines+markers', name='Validation Accuracy',
                            line=dict(color='#fd7e14'), marker=dict(color='#fd7e14')))
fig_acc.update_layout(title='Training and Validation Accuracy',
                      xaxis_title='Epochs',
                      yaxis_title='Accuracy',
                      width=700, height=400)

# Display plots (this requires plotly installed and configured for your environment)
# In a Jupyter notebook, these might display automatically. Otherwise use fig.show().
# For web rendering:
print("```plotly")
print(fig_loss.to_json(pretty=False))
print("```")
print("> Training and validation loss across epochs.")
print("```plotly")
print(fig_acc.to_json(pretty=False))
print("```")
print("> Training and validation accuracy across epochs.")

# If not running in an environment that automatically displays plotly:
# fig_loss.show()
# fig_acc.show()

By observing these plots:

• Loss Plot: We typically want to see both training and validation loss decrease and converge. If the validation loss starts increasing while the training loss continues to decrease, it's a sign of overfitting.
• Accuracy Plot: We want to see both training and validation accuracy increase and converge. A large gap between training and validation accuracy also suggests overfitting.

These plots provide visual feedback on how well the training parameters (like the number of epochs) were chosen and whether the model is generalizing well to unseen data.

6. Evaluating the Model

Finally, let's evaluate the trained model's performance on the test set using the evaluate() method. This gives us the final loss and accuracy on data the model has never seen during training updates.

# Evaluate the model on the test set
loss, accuracy = model.evaluate(X_test, y_test, verbose=0) # verbose=0 for silent evaluation

print(f"\nTest Loss: {loss:.4f}")
print(f"Test Accuracy: {accuracy:.4f}")

The output shows the final performance metrics on the test set. An accuracy score close to 1.0 indicates good performance on this dataset.

Summary

In this practical exercise, we walked through the complete process of training a simple neural network classifier using Keras:

1. We prepared a synthetic dataset suitable for binary classification.
2. We defined a neural network architecture with Dense layers.
3. We compiled the model, specifying the adam optimizer, binary_crossentropy loss, and accuracy metric.
4. We trained the model using the fit() method, providing training and validation data, and setting the number of epochs and batch size.
5. We monitored the training process by plotting the loss and accuracy curves, checking for convergence and overfitting.
6. We evaluated the final model performance on the unseen test set using the evaluate() method.

This workflow forms the foundation for training more complex deep learning models in Keras, incorporating the core concepts covered in this chapter. You now have hands-on experience with the essential steps involved in making a neural network learn from data.