Sample Size Estimation | Data Masked

Autoplay
Autocomplete

Previous Lesson Complete and Continue

Product Data Science Course

Product Sense via Machine Learning

Intro to Insights
Regressions
Decision Trees
Partial Dependence Plots - Theory
Partial Dependence Plots - Practice
Rulefit - Theory
Rulefit - Practice
Parting Notes
Exercise

Product and Metrics - Case Studies

How to improve a given metric
Marketplace - Focusing on supply or demand?
Actionable insights vs spurious relationships
Behavioral vs demographic variables
How to create key growth metrics
How to come up with new feature ideas?
How to estimate user LifeTime Value?
User Similarity
Design metrics for customer service
Product Development - Should we test this new feature?
Metrics - Understanding what drives them
Metrics - How to find bugs
How to predict user behavior?
Metrics - How to create an effective metric
Metrics - Self-selection bias
Metrics - Trade-off between short term vs long term effects
How to improve revenue from ads
Metrics - How to create an effective metric for gaming apps

Personalization

Personalization - Full project
Personalization from a probabilistic standpoint

Unbalanced Classes

Cut-off point optimization
Changing class weights
Exercise

Missing data

Missing Data - Full Project
Missing Data - Uber example

Fraud and Machine Learning - Case Studies

Fake Listings
Random Forest - Key Params
False Positives vs False Negatives
Training/Test set split
Fraud - Two step authentication
Fraud - Fake Profiles
Supervised vs Unsupervised ML

A/B Testing - Practice

Overview
Sample Size Estimation
Randomization
Randomization - Exercise
Novelty Effect
General Guidelines

A/B testing - Case Studies

Test by Market
Opportunity costs
Testing long term metrics
Novelty Effect
Test by market - Drawbacks
Multiple A/B tests at the same time
Network Effect
Sample Size
Test first version of a data product
Statistically sound A/B test
When to rerun the same test again
When not to run an A/B test

New Product Case Studies - Miscellaneous

INSIGHTS: Should users be asked for their CC info when starting a free trial?
INSIGHTS: How much would you charge for a new product?
INSIGHTS: What was the data-driven hypothesis behind testing FB stories?
INSIGHTS: Trade-off between revenue and user experience metrics
INSIGHTS: What did you do when you got unexpected results?
INSIGHTS: How would you place ads on a page?
INSIGHTS: Would you test a feature allowing users to easily switch account?
INSIGHTS: Would you merge two similar products or run them separately?
INSIGHTS: Categorical variables in regressions. Better one-hot-encoding or multiple regressions?
INSIGHTS: How would you make Airbnb booking process smoother?
INSIGHTS: Can a bug actually improve your metric? What to do if that happens?
INSIGHTS: How to improve a marketplace?
A/B TESTING: Can you do a t-test on a dummy variable like conversion (0/1/)?
A/B TESTING: How would you test the success of a new ad campaign?
A/B TESTING: Define test statistical significance in layman’s terms. Why 0.05?
A/B TESTING: What would you do if test p-value is slightly above signficance level?
A/B TESTING: What to do if after a test some metrics are up and some down?
A/B TESTING: How to test different prices?
A/B TESTING: When would you design an A/B test as a sequence of multiple tests?
A/B TESTING: How can you halve the width of a t-test confidence interval?
A/B TESTING: How to check if a metric is actually up after a successful test?
A/B TESTING: How to avoid the A/B test issue of over-optimizing for the current user base?
A/B TESTING: When would you increase a test significance level?
METRICS: Conversion rate down, but absolute number of conversions up. Is it good or bad?
METRICS: How choosing an average-based metric vs a percentile one would impact product development?
METRICS: When to run a test on multiple metrics?
METRICS: When is accuracy a bad metric for a model?
METRICS: What kind of analysis is required after a logging-related bug?
METRICS: What kind of analysis is required after a product-related bug?
METRICS: In most metrics, you expect the average to be larger or smaller than the median?

Projects with Solutions

Project: How to improve conversion rate
Project: Predicting fraud
Project: Predicting employee retention

Collection of tech company blog posts/case studies

A/B testing overview from Netflix, Scribd, MS
Theory behind A/B test from Google, StitchFix, Stackoverflow
A/B testing with connected users from Google, Instacart
Common Issues in A/B testing from Wallmart Labs, Google
Product Development from Airbnb, FB, Linkedin, Google, Netflix

Data Challenges with Solutions

Introduction
INSIGHTS: Workplace Diversity Analysis
Solution: Workplace Diversity Analysis
INSIGHTS: Funnel Analysis
Solution: Funnel Analysis
INSIGHTS: Subscription Retention Rate
Solution: Subscription Retention Rate
INSIGHTS: Sessionize user activity
Solution: Sessionize user activity
INSIGHTS: Video Sharing Analysis
Solution: Video Sharing Analysis
METRICS: Ads Analysis
Solution: Ads Analysis
METRICS: Hotel Search Data
Solution: Hotel Search Data
A/B Testing: User Referral Program
Solution: User Referral Program
A/B TESTING: Pricing Test
Solution: Pricing Test
ML: Applying for a loan
Solution: Applying for a loan
ML: Optimization of Employee Shuttle Stops
Solution: Optimization of Employee Shuttle Stops
ML: Clustering Grocery Items
Solution: Clustering Grocery Items
ML: Credit Card Transactions
Solution: Credit Card Transactions
ML: Song Recommendation
Solution: Song Recommendation

Metrics - SQL Exercises

New Coding Course
Metrics: Time delta between consecutive events
SOLUTION: Time delta between consecutive events
Metrics: Segment by mobile, web, and cross-device users
SOLUTION: Segment by mobile, web, and cross-device users
Metrics: Identify power users based on their history
SOLUTION: Identify power users based on their history
Metrics: Estimate total and running values
SOLUTION: Estimate total and running values
Metrics: Summary stats - Custom implementation of the median vs average
SOLUTION: Summary stats - Custom implementation of the median vs average
Metrics : Rank users within groups
SOLUTION: Rank users within groups

Teach online with

Sample Size Estimation

Lesson content locked

If you're already enrolled, you'll need to login.

Enroll in Course to Unlock