Serving and Deployment with Pusher

The Pusher component within the TensorFlow Extended (TFX) framework plays a primary role in deploying machine learning models. Its function is to make trained and thoroughly validated models available for serving predictions in production pipelines. This component acts as a gatekeeper, ensuring that only approved models are deployed to the designated serving infrastructure.

The Pusher component is designed to conditionally push a validated model artifact to a predefined deployment target. Its operation relies on the outcome of the model validation steps.

Conditional Deployment Based on Validation

Recall that the Evaluator component compares the newly trained model against a baseline (often the currently deployed model) using evaluation metrics computed on a holdout dataset. Based on predefined thresholds and criteria, the Evaluator produces a ModelBlessing artifact. This artifact signals whether the new model meets the quality standards required for deployment.

The Pusher component consumes two primary inputs:

1. Model Artifact: The trained model produced by the Trainer component, typically in the SavedModel format.
2. ModelBlessing Artifact: The output from the Evaluator or ModelValidator.

The core logic of Pusher is straightforward: if the ModelBlessing artifact indicates that the model is "blessed" (meaning it passed validation), Pusher proceeds to copy the Model artifact to the specified push_destination. If the model is not blessed, Pusher takes no action, preventing a potentially underperforming model from reaching the production serving environment. This conditional step is fundamental for maintaining the reliability of your machine learning system.

Pusher consumes the Model from Trainer and the ModelBlessing from Evaluator, pushing the model to a deployment target only if it is blessed.

Deployment Targets and Configuration

The push_destination configured for the Pusher component determines where the blessed model is copied. While TFX is flexible, a common and well-supported target is a filesystem directory structure compatible with TensorFlow Serving.

TensorFlow Serving monitors specified directories for new model versions. It expects a specific structure, typically like /path/to/base/model_name/version_number/saved_model.pb|variables/. Pusher handles creating this structure automatically. When a new blessed model is pushed, Pusher creates a new numbered subdirectory (representing the model version) within the model's base path and copies the SavedModel contents into it. TensorFlow Serving can then detect and load this new version for serving requests.

Here's a simplified example of configuring the Pusher component in a TFX pipeline definition using Python:

# Assuming 'trainer' and 'evaluator' are previously defined TFX components
# producing 'model' and 'blessing' artifacts respectively.

from tfx.components import Pusher
from tfx.proto import pusher_pb2
import os

# Define the location for TF Serving to monitor
serving_model_dir = os.path.join('/path/to/serving/models/my_model')

pusher = Pusher(
    model=trainer.outputs['model'],
    model_blessing=evaluator.outputs['blessing'],
    push_destination=pusher_pb2.PushDestination(
        filesystem=pusher_pb2.PushDestination.Filesystem(
            base_directory=serving_model_dir
        )
    )
)

# This 'pusher' component would then be included in the pipeline's component list.

In this configuration:

• model=trainer.outputs['model'] links the Pusher to the output model artifact of the Trainer.
• model_blessing=evaluator.outputs['blessing'] links it to the validation outcome from the Evaluator.
• push_destination specifies the target. Here, pusher_pb2.PushDestination.Filesystem indicates a location on a filesystem accessible by the pipeline and the serving system. The base_directory is the root path where versioned model directories will be created (e.g., /path/to/serving/models/my_model/1/, /path/to/serving/models/my_model/2/, etc.).

While TensorFlow Serving is a primary target, Pusher can also be configured to push to other locations like Google Cloud Storage (GCS), AWS S3, or even trigger custom deployment logic by implementing custom executors, although filesystem-based deployment for TF Serving is the most common use case. If deploying to TF Lite, the Pusher might push the converted .tflite model file (often generated by a custom component or modified Trainer) to a designated location for mobile app bundling or edge device deployment.

Integration with ML Metadata (MLMD)

Like all standard TFX components, Pusher interacts with ML Metadata. When it successfully pushes a model, it records this event in the MLMD store. This creates a traceable record linking the specific model artifact, its validation status (ModelBlessing), and its deployment location and version. This metadata is invaluable for tracking model lineage, understanding which model version is serving in production, and potentially rolling back deployments if issues arise.

In summary, the Pusher component serves as a critical control point in a TFX pipeline. It bridges the gap between model development/validation and production serving by automating the conditional deployment of validated models. By leveraging the ModelBlessing artifact and integrating with deployment targets like TensorFlow Serving and MLMD, Pusher helps ensure that only high-quality, vetted models are exposed to end-users, contributing significantly to the reliability and maintainability of production machine learning systems.