Learning path
MLOps learning path: everything to study, topic by topic
Comprehensive MLOps assessment covering experiment tracking, ML pipelines, model serving, monitoring, infrastructure, and governance for production machine learning systems.
Short answer
To learn MLOps, work through 7 modules and 30 topics in the order they build on each other. It starts with Experiment Tracking, then moves on to ML Pipelines and Model Serving. Use the map as a checklist, or measure your level and study only what is missing.
What you will study
Topics are grouped by module. Open one to see its objectives and key concepts.
1 · Experiment Tracking4 topics
Experiment Tracking Fundamentals
Logging metrics, parameters, artifacts, and comparing experiment runs
Objectives
- Set up experiment tracking with MLflow or W&B
- Compare and analyze experiment results
Key concepts
- MLflow tracking
- Weights & Biases
- experiment runs
- metric logging
- artifact storage
- run comparison
- experiment organization
- parameter logging
Hyperparameter Optimization
Systematic hyperparameter search strategies and automated tuning
Objectives
- Choose the right HPO strategy for a given problem
- Configure Bayesian optimization with Optuna
Key concepts
- grid search
- random search
- Bayesian optimization
- Optuna
- Ray Tune
- early stopping
- search space definition
- multi-objective optimization
Model Registry
Centralized model versioning, staging, and lifecycle management
Objectives
- Manage model lifecycle through registry stages
- Implement model promotion workflows
Key concepts
- model versioning
- staging environments
- model promotion
- model lineage
- MLflow Model Registry
- model metadata
- approval workflows
- model aliases
Reproducibility
Ensuring ML experiments are reproducible across environments and time
Objectives
- Identify and fix reproducibility issues in ML pipelines
- Set up DVC for data versioning
Key concepts
- random seeds
- environment pinning
- data versioning
- DVC
- deterministic training
- dependency management
- container snapshots
- code versioning
2 · ML Pipelines5 topics
Data Pipelines
Building reliable data ingestion, transformation, and feature engineering pipelines
Objectives
- Design idempotent data pipelines for ML
- Choose between batch and streaming approaches
Key concepts
- ETL vs ELT
- data ingestion
- data transformation
- Apache Airflow
- batch vs streaming
- data quality checks
- idempotent pipelines
- schema evolution
Feature Stores
Centralized feature management for training and serving consistency
Objectives
- Explain online vs offline feature stores
- Design feature pipelines with point-in-time correctness
Key concepts
- online store
- offline store
- feature freshness
- Feast
- Tecton
- feature versioning
- point-in-time joins
- feature sharing
Training Pipelines
Automated model training workflows with validation and artifact management
Objectives
- Design automated retraining pipelines
- Implement proper validation in training workflows
Key concepts
- training workflow
- data splitting
- cross-validation
- automated retraining
- training triggers
- artifact management
- compute allocation
- training checkpoints
Pipeline Orchestration
Orchestrating complex ML workflows with DAGs, retries, and scheduling
Objectives
- Choose the right orchestration tool for ML workflows
- Design fault-tolerant ML pipelines
Key concepts
- DAG design
- Kubeflow Pipelines
- Apache Airflow
- Prefect
- task dependencies
- retry policies
- scheduling
- pipeline caching
Data Validation
Automated data quality validation, schema enforcement, and anomaly detection
Objectives
- Implement data validation gates in ML pipelines
- Design expectation suites for production data
Key concepts
- Great Expectations
- TensorFlow Data Validation
- schema validation
- data profiling
- expectation suites
- data quality metrics
- anomaly detection
- validation gates
3 · Model Serving4 topics
Model Packaging
Packaging models for deployment with serialization, containerization, and format standards
Objectives
- Choose the right model serialization format
- Package models as containerized services
Key concepts
- ONNX
- TensorFlow SavedModel
- pickle serialization
- Docker containerization
- model signatures
- input/output schemas
- model optimization
- quantization
Inference Servers
Deploying and managing model inference servers for real-time and batch predictions
Objectives
- Configure inference servers for production workloads
- Optimize inference latency and throughput
Key concepts
- TensorFlow Serving
- Triton Inference Server
- TorchServe
- BentoML
- batching
- model ensembles
- gRPC vs REST
- latency optimization
ML Deployment Strategies
Progressive rollout strategies for ML models including canary, shadow, and blue-green
Objectives
- Design safe model deployment strategies
- Implement canary deployments with automatic rollback
Key concepts
- canary deployment
- blue-green deployment
- shadow deployment
- traffic splitting
- rollback procedures
- A/B testing deployment
- multi-armed bandit
- feature flags for ML
Edge & Mobile Serving
Deploying ML models to edge devices and mobile platforms
Objectives
- Optimize models for edge deployment
- Choose compression techniques for mobile inference
Key concepts
- TensorFlow Lite
- ONNX Runtime Mobile
- model compression
- knowledge distillation
- edge inference
- on-device ML
- model pruning
- hardware acceleration
4 · Monitoring & Observability4 topics
Model Monitoring
Monitoring model performance, accuracy degradation, and prediction quality in production
Objectives
- Design model monitoring strategies for production
- Handle delayed ground truth in monitoring
Key concepts
- prediction quality
- accuracy degradation
- ground truth collection
- delayed feedback
- proxy metrics
- Evidently AI
- Whylogs
- monitoring dashboards
Data & Concept Drift
Detecting and responding to data drift, concept drift, and distribution shifts
Objectives
- Distinguish between data drift and concept drift
- Implement automated drift detection pipelines
Key concepts
- data drift
- concept drift
- covariate shift
- prior probability shift
- statistical tests
- KL divergence
- PSI
- drift response strategies
Infrastructure Performance Monitoring
Monitoring latency, throughput, resource utilization, and SLA compliance for ML systems
Objectives
- Set up infrastructure monitoring for ML systems
- Diagnose latency bottlenecks in inference pipelines
Key concepts
- latency percentiles
- throughput metrics
- GPU utilization
- memory profiling
- SLA compliance
- Prometheus
- Grafana
- resource bottlenecks
Alerting & Observability
Building alerting systems and observability for end-to-end ML system health
Objectives
- Design alerting strategies that avoid alert fatigue
- Define SLOs for ML system reliability
Key concepts
- alert design
- alert fatigue
- runbooks
- distributed tracing
- structured logging
- incident response
- SLOs for ML
- on-call practices
5 · Infrastructure4 topics
GPU & Compute Management
Managing GPU resources, instance selection, and compute allocation for ML workloads
Objectives
- Select appropriate GPU instances for ML workloads
- Optimize GPU utilization and cost
Key concepts
- GPU instance types
- CUDA
- multi-GPU training
- GPU memory management
- spot instances
- preemptible VMs
- compute scheduling
- resource quotas
Containerized ML Workflows
Using containers and Kubernetes for ML training and serving infrastructure
Objectives
- Design Kubernetes-based ML infrastructure
- Configure GPU scheduling and resource management
Key concepts
- ML Docker images
- Kubernetes for ML
- Kubeflow
- GPU scheduling
- resource limits
- node selectors
- pod affinity
- ML-specific operators
Distributed Training
Scaling model training across multiple GPUs and nodes with data and model parallelism
Objectives
- Choose the right parallelism strategy for a workload
- Configure distributed training with PyTorch DDP
Key concepts
- data parallelism
- model parallelism
- pipeline parallelism
- Horovod
- PyTorch DDP
- communication backends
- gradient synchronization
- all-reduce
ML Cost Optimization
Optimizing cloud costs for ML workloads including training, serving, and storage
Objectives
- Design cost-effective ML infrastructure
- Implement auto-scaling for inference workloads
Key concepts
- spot instance strategies
- right-sizing
- auto-scaling inference
- model caching
- batch inference
- storage tiering
- reserved capacity
- cost attribution
6 · Governance & Strategy4 topics
Model Governance
Establishing governance frameworks for model development, deployment, and lifecycle management
Objectives
- Design model governance frameworks
- Implement model cards and documentation standards
Key concepts
- model cards
- model documentation
- approval workflows
- risk assessment
- bias detection
- fairness metrics
- explainability
- responsible AI
Compliance & Audit
Regulatory compliance, audit trails, and lineage tracking for ML systems
Objectives
- Design audit-compliant ML systems
- Implement data lineage tracking
Key concepts
- GDPR compliance
- model audit trails
- data lineage
- right to explanation
- data retention
- PII handling
- regulatory frameworks
- SOC 2 for ML
A/B Testing for ML
Designing and analyzing A/B tests for ML model evaluation in production
Objectives
- Design rigorous A/B tests for ML models
- Analyze experiment results with proper statistical methods
Key concepts
- statistical significance
- sample size calculation
- multi-armed bandits
- interleaving experiments
- guardrail metrics
- novelty effects
- long-term holdouts
- experiment platforms
MLOps Maturity Model
Assessing and advancing organizational MLOps maturity across people, process, and technology
Objectives
- Assess organizational MLOps maturity
- Design roadmaps for MLOps maturity advancement
Key concepts
- maturity levels
- Google MLOps maturity model
- automation levels
- CI/CD for ML
- CT (continuous training)
- platform engineering
- organizational readiness
- ML platform design
7 · Expert MLOps5 topics
ML Observability
Implementing ML observability: data drift detection, concept drift monitoring, model decay metrics, and alerting pipelines
Objectives
- Design a comprehensive ML observability system covering data, model, and infrastructure
- Configure drift detection with appropriate statistical tests and alert thresholds
Key concepts
- data drift
- concept drift
- PSI
- KL divergence
- model decay metrics
- statistical tests
- Evidently
- Arize
- alert thresholds
Feature Store Architecture
Designing and operating feature stores: online/offline split, backfill strategies, feature freshness, and serving latency
Objectives
- Design a feature store architecture that eliminates training-serving skew
- Implement backfill strategies for point-in-time correct feature retrieval
Key concepts
- online vs offline store
- point-in-time correctness
- backfill
- feature freshness
- serving latency
- Feast
- Tecton
- training-serving skew
Model Registry & Versioning at Scale
Operating model registries at scale: versioning strategies, stage transitions, artifact lineage, and multi-team governance
Objectives
- Design a multi-team model registry with governance controls and stage transitions
- Implement artifact lineage tracking from training data to production model
Key concepts
- model versioning
- stage transitions
- artifact lineage
- MLflow Registry
- SageMaker Model Registry
- multi-team access
- champion/challenger
A/B Testing & Shadow Deployment for ML
Deploying ML models safely using shadow mode, traffic splitting, bandit algorithms, and statistical significance testing
Objectives
- Design a shadow deployment pipeline that validates new models with zero production impact
- Select between A/B testing and bandit algorithms for online model evaluation
Key concepts
- shadow mode
- traffic splitting
- multi-armed bandit
- Thompson sampling
- statistical significance
- online evaluation
- delayed feedback
ML Incident Response
Responding to ML model incidents: degradation detection, rollback procedures, root cause analysis, and post-mortems
Objectives
- Execute a structured ML incident response from detection through post-mortem
- Design rollback procedures and champion model fallback for production ML systems
Key concepts
- model degradation
- rollback procedures
- champion model fallback
- root cause analysis
- incident classification
- runbooks
- postmortem templates
Levels: Junior, Mid, Senior, and Expert
Explain it in your own words
Graded by AI · no account · 1 questionNo options to guess from: you write, the AI grades against the topic's rubric and tells you what level your answer reads at.
Already studying MLOps?
The assessment measures topic by topic, and your plan skips what you already know.
Open to read. An account is only needed to be measured.
Frequently asked questions
Where should I start studying MLOps?
Start with Experiment Tracking: Experiment Tracking Fundamentals, Hyperparameter Optimization, and Model Registry. The map orders topics by what they depend on, so each one prepares the next.
How long does a full MLOps assessment take?
A full assessment of this path takes about 55 minutes. Study time depends on where you start; a diagnostic shows which topics you can skip.
Do I need an account to use this page?
No. The map, objectives and key concepts are open. You only need an account to be measured, get your gap by topic and receive a plan.