51 items
A/B Testing: Power, p-values, and CUPED
FlashcardSimpson's Paradox: Aggregating Stratified Rates
Code QuizSimpson's Paradox and Confounding
QuizA/B Testing: Power, P-values, CUPED
Slides / VideoConfounding and Simpson's Paradox
FlashcardTwo-Proportion Z-Test for A/B Test
Code QuizPeeking at A/B Test Results
QuizConfidence Interval From Sample Mean
Code QuizInterpreting a 95% Confidence Interval
QuizSimpson's Paradox and Confounders
Slides / VideoConfidence Intervals & the Sampling Distribution
FlashcardHypothesis Testing in A/B Experiments
FlashcardDistributions, Sampling & Confidence Intervals
Slides / VideoHypothesis Testing & A/B Design
Slides / VideoInterpreting the p-value Wrong
Code QuizInterpreting P-values Correctly
QuizBuggy Standard Deviation Function
Code QuizMean vs Median in Skewed Data
QuizP-value Significance Check Bug
Code QuizUnderstanding P-values and Correlation
QuizPopulation Variance Bug
Code QuizMean vs Median in Skewed Data
QuizHypothesis Testing, P-values & Causation
Slides / VideoP-values and Correlation vs Causation
FlashcardHypothesis Testing, p-values & Intervals
Slides / VideoP-values & Confidence Intervals
FlashcardExtracting an Eigenvector from np.linalg.eig
Code QuizData as Vectors and Matrices
Slides / VideoGradients and Partial Derivatives
Slides / VideoSingular Value Decomposition Basics
Slides / VideoMatrix Transpose and Inverse
Slides / VideoLinear Independence, Span, and Basis
Slides / VideoVector Norms and Distance Metrics
Slides / VideoDot Product and Vector Similarity
Slides / VideoMatrix Multiplication in AI
Slides / VideoMatrices and Matrix Operations
Slides / VideoVectors and Vector Operations
Slides / VideoConfidence Intervals
Slides / VideoHypothesis Testing and p-values
Slides / VideoSampling and the Central Limit Theorem
Slides / VideoDescriptive Statistics: Center and Spread
Slides / VideoMaximum Likelihood Estimation
Slides / VideoJoint, Marginal, Conditional Distributions
Slides / VideoCovariance and Correlation
Slides / VideoExpectation, Variance, Standard Deviation
Slides / VideoBayes' Theorem in AI
Slides / VideoConditional Probability and Independence
Slides / VideoGaussian, Bernoulli, Binomial Distributions
Slides / VideoDiscrete vs Continuous Distributions
Slides / VideoRandom Variables and Distributions
Slides / VideoEigenvalues and Eigenvectors Basics
Slides / VideoTwo-Proportion Z-Test for A/B Test
Spot the standard-error mistake in a two-proportion z-test used for A/B experiment analysis.
from scipy import stats
import numpy as np
def ab_test(conv_a, n_a, conv_b, n_b):
# conv_* = number of conversions, n_* = number of visitors
p_a = conv_a / n_a
p_b = conv_b / n_b
p_pool = (conv_a + conv_b) / (n_a + n_b)
se = np.sqrt(p_a * (1 - p_a) / n_a + p_b * (1 - p_b) / n_b)
z = (p_b - p_a) / se
p_value = 2 * (1 - stats.norm.cdf(abs(z)))
return z, p_value
# Example: A had 120/2400, B had 150/2450
print(ab_test(120, 2400, 150, 2450))The function is meant to run a significance test on whether B's conversion rate differs from A's. What is the bug?