Machine Learning Products for PMs
Manage traditional ML products: problem types, data and labelling, metrics, working with data scientists, recommendation and personalisation, the model lifecycle and MLOps monitoring.
Machine Learning Products for PMs
Manage traditional ML products: problem types, data and labelling, metrics, working with data scientists, recommendation and personalisation, the model lifecycle and MLOps monitoring.
In one line
Before LLMs, most AI in products was machine learning that predicts, classifies, ranks or recommends (fraud scores, feeds, ETAs). PMs still own these products: the problem, the data, the metric trade-offs and the lifecycle.
The 6 classes
ML problem types
Recognise which ML problem you're solving.
- Name the problem type first.
- Each type has different metrics.
- Many products combine types.
Data and labelling
Get the data an ML product needs.
- Data quality decides model quality.
- Write labelling guidelines.
- Check leakage, imbalance and bias.
Metrics and trade-offs
Choose model metrics that match business costs.
- Precision vs recall is a business decision.
- Set thresholds by cost of errors.
- Validate with online experiments.
Recommendations and personalisation
Understand how recommenders work and are measured.
- Hybrid recommenders are common.
- Plan for cold start.
- Balance engagement with diversity and long-term value.
Model lifecycle and MLOps
Keep models working after launch.
- Launch is the start of the lifecycle.
- Monitor drift and business impact.
- Always have a rollback.
Working with data scientists and final project
Collaborate well and ship an ML feature.
- Start from a simple baseline.
- Agree metrics and costs of errors early.
- Document with a model card.
Interview questions
How is ML PM different from regular PM?
Probabilistic outputs, data dependency, metric trade-offs, experimentation and ongoing monitoring/retraining.
Precision vs recall trade-off?
Choose based on the cost of false positives vs false negatives; set thresholds accordingly and validate online.
How do you handle cold start?
Onboarding preferences, popular/trending items, content-based features, and exploration.
What is drift?
Change in input data or relationships over time that degrades model quality; detect via monitoring and retrain.
What goes in an ML PRD?
Problem type, labels/data, metrics and thresholds, baseline, experiment plan, monitoring, rollback, model card.
