Complete and Continue
Product Data Science Course
Product Sense via Machine Learning
Intro to Insights
Partial Dependence Plots - Theory
Partial Dependence Plots - Practice
Rulefit - Theory
Rulefit - Practice
Product and Metrics - Case Studies
How to improve a given metric
Marketplace - Focusing on supply or demand?
Actionable insights vs spurious relationships
Behavioral vs demographic variables
How to create key growth metrics
How to come up with new feature ideas?
How to estimate user LifeTime Value?
Design metrics for customer service
Product Development - Should we test this new feature?
Metrics - Understanding what drives them
Metrics - How to find bugs
How to predict user behavior?
Metrics - How to create an effective metric
Metrics - Self-selection bias
Metrics - Trade-off between short term vs long term effects
How to improve revenue from ads
Metrics - How to create an effective metric for gaming apps
Personalization - Full project
Personalization from a probabilistic standpoint
Cut-off point optimization
Changing class weights
Missing Data - Full Project
Missing Data - Uber example
Fraud and Machine Learning - Case Studies
Random Forest - Key Params
False Positives vs False Negatives
Training/Test set split
Fraud - Two step authentication
Fraud - Fake Profiles
Supervised vs Unsupervised ML
A/B Testing - Practice
Sample Size Estimation
Randomization - Exercise
A/B testing - Case Studies
Test by Market
Testing long term metrics
Test by market - Drawbacks
Multiple A/B tests at the same time
Test first version of a data product
Statistically sound A/B test
When to rerun the same test again
When not to run an A/B test
New Product Case Studies - Miscellaneous
INSIGHTS: Should users be asked for their CC info when starting a free trial?
INSIGHTS: How much would you charge for a new product?
INSIGHTS: What was the data-driven hypothesis behind testing FB stories?
INSIGHTS: Trade-off between revenue and user experience metrics
INSIGHTS: What did you do when you got unexpected results?
INSIGHTS: How would you place ads on a page?
INSIGHTS: Would you test a feature allowing users to easily switch account?
INSIGHTS: Would you merge two similar products or run them separately?
INSIGHTS: Categorical variables in regressions. Better one-hot-encoding or multiple regressions?
INSIGHTS: How would you make Airbnb booking process smoother?
INSIGHTS: Can a bug actually improve your metric? What to do if that happens?
INSIGHTS: How to improve a marketplace?
A/B TESTING: Can you do a t-test on a dummy variable like conversion (0/1/)?
A/B TESTING: How would you test the success of a new ad campaign?
A/B TESTING: Define test statistical significance in layman’s terms. Why 0.05?
A/B TESTING: What would you do if test p-value is slightly above signficance level?
A/B TESTING: What to do if after a test some metrics are up and some down?
A/B TESTING: How to test different prices?
A/B TESTING: When would you design an A/B test as a sequence of multiple tests?
A/B TESTING: How can you halve the width of a t-test confidence interval?
A/B TESTING: How to check if a metric is actually up after a successful test?
A/B TESTING: How to avoid the A/B test issue of over-optimizing for the current user base?
A/B TESTING: When would you increase a test significance level?
METRICS: Conversion rate down, but absolute number of conversions up. Is it good or bad?
METRICS: How choosing an average-based metric vs a percentile one would impact product development?
METRICS: When to run a test on multiple metrics?
METRICS: When is accuracy a bad metric for a model?
METRICS: What kind of analysis is required after a logging-related bug?
METRICS: What kind of analysis is required after a product-related bug?
METRICS: In most metrics, you expect the average to be larger or smaller than the median?
Projects with Solutions
Project: How to improve conversion rate
Project: Predicting fraud
Project: Predicting employee retention
Collection of tech company blog posts/case studies
A/B testing overview from Netflix, Scribd, MS
Theory behind A/B test from Google, StitchFix, Stackoverflow
A/B testing with connected users from Google, Instacart
Common Issues in A/B testing from Wallmart Labs, Google
Product Development from Airbnb, FB, Linkedin, Google, Netflix
Data Challenges with Solutions
INSIGHTS: Workplace Diversity Analysis
Solution: Workplace Diversity Analysis
INSIGHTS: Funnel Analysis
Solution: Funnel Analysis
INSIGHTS: Subscription Retention Rate
Solution: Subscription Retention Rate
INSIGHTS: Sessionize user activity
Solution: Sessionize user activity
INSIGHTS: Video Sharing Analysis
Solution: Video Sharing Analysis
METRICS: Ads Analysis
Solution: Ads Analysis
METRICS: Hotel Search Data
Solution: Hotel Search Data
A/B Testing: User Referral Program
Solution: User Referral Program
A/B TESTING: Pricing Test
Solution: Pricing Test
ML: Applying for a loan
Solution: Applying for a loan
ML: Optimization of Employee Shuttle Stops
Solution: Optimization of Employee Shuttle Stops
ML: Clustering Grocery Items
Solution: Clustering Grocery Items
ML: Credit Card Transactions
Solution: Credit Card Transactions
ML: Song Recommendation
Solution: Song Recommendation
Teach online with
Metrics - How to find bugs
Lecture content locked
If you're already enrolled,
you'll need to login
Enroll in Course to Unlock