Planning and Optimizing AI Infrastructure
Chapter 1: Foundations of AI Compute Infrastructure
Introduction to AI Workloads
The Role of CPUs in AI Systems
The Role of GPUs in Accelerating AI
Comparing CPU and GPU Architectures for ML
Introduction to TPUs and other ASICs
Memory and its Importance for Large Models
Storage Solutions for AI Datasets
Networking Considerations for Distributed Systems
Hands-on Practical: Benchmarking CPU vs GPU
Chapter 2: Designing On-Premise AI Infrastructure
Assessing Workload Requirements
Selecting Server Hardware for AI
GPU Interconnect Technologies
High-Speed Storage Configurations
Networking for Data and Model Transfer
Power and Cooling Requirements
Building a Bare-Metal AI Server
Practice: Creating a Hardware Specification Sheet
Chapter 3: Leveraging Cloud Platforms for AI
Overview of Major Cloud Providers for AI
Comparing Managed AI Services vs IaaS
Selecting Virtual Machine Instances for Training
Choosing Instances for Inference and Serving
Object Storage Services for Datasets
Understanding Cloud Networking and VPCs
Security Considerations in the Cloud
Hands-on Practical: Launching a GPU Cloud Instance
Chapter 4: Containerization and Orchestration for ML
Introduction to Docker for Reproducible Environments
Building a Docker Image with ML Libraries
Introduction to Kubernetes for Managing ML Workloads
Kubernetes Components: Pods, Services, Deployments
Managing GPU Resources in a Kubernetes Cluster
Using Kubeflow for ML Pipelines
Hands-on Practical: Deploying a Model on Kubernetes
Chapter 5: Strategies for Performance Optimization
Identifying Performance Bottlenecks
Techniques for Distributed Training
Using Mixed-Precision Training
Model Quantization for Efficient Inference
Optimizing Data Loading and Preprocessing Pipelines
Profiling GPU and CPU Usage
Hands-on Practical: Applying Mixed-Precision Training
Chapter 6: Cost Management and Optimization
Analyzing On-Premise Total Cost of Ownership
Understanding Cloud Pricing Models
Strategies for Reducing Cloud Compute Costs
Managing Data Storage and Transfer Costs
Implementing Cost Monitoring and Alerting
Right-Sizing Infrastructure for Workloads
Practice: Calculating and Comparing Job Costs
Introduction to TPUs and other ASICs
Was this section helpful?
- In-Datacenter Performance Analysis of a Tensor Processing Unit, Norman P. Jouppi, Cliff Young, Nishant Patil, David A. Patterson, et al., 2017 Proceedings of the 44th ACM/IEEE Annual International Symposium on Computer Architecture (ISCA '17) (IEEE and ACM) DOI: 10.1145/3079856.3080246 - This seminal paper introduces the Google Tensor Processing Unit (TPU), detailing its architecture, the systolic array, and its performance characteristics for machine learning inference workloads.
- Tensor Processing Units (TPUs) | Google Cloud, Google Cloud, 2023 (Google Cloud) - Official documentation providing current details on Google's TPUs, their capabilities, and how to utilize them within the Google Cloud ecosystem.
- The Deep Learning Hardware Landscape, Albert Reuther, Jeremy Kepner, Andrew P. Sage, Jeremy R. Holland, Robert T. Bond, Jeffrey K. Oddson, Roger Pearce, Charles M. Leiserson, William M. McMahon, and Peter A. Michaleas, 2021 MIT Lincoln Laboratory Journal, Vol. 22 (MIT Lincoln Laboratory) - This survey article presents an overview of hardware options for deep learning, discussing the strengths and weaknesses of CPUs, GPUs, and specialized ASICs, illustrating their respective trade-offs.
© 2025 ApX Machine LearningEngineered with