Maximum Likelihood Estimation provides consistent estimators, and can be efficiently computed under many null hypotheses of practical interest.

In our last post, we introduced the potential outcomes framework as the foundational framework for causal inference. In the potential outcomes framework, each unit (e.g. each person) is represented by a pair of outcomes, corresponding to the result of the experience provided to them (treatment or control, A or B, etc.

"Why can't I take the results of an A/B test at face value? Who are you, the statistics mafia? I don't need a PhD in statistics to know that one number is greater than another." If this sounds familiar, it is helpful to remember that we do an A/B test to learn about different potential outcomes. Comparing potential outcomes is essential for smart decision making, and this framework is the cornerstone of causal inference.

Calculators for planning and analyzing A/B tests

When I started this blog, my primary objective was less about teaching others A/B testing and more about clarifying my own thoughts on A/B testing. I had been running A/B tests for about a year, and I was starting to feel uncomfortable with some of the standard methodologies.

We can plan sample sizes to control the width of confidence intervals.

The great successes of Machine Learning in recent years are based on our ability to extrapolate and predict based on data. The next big step is learning and leveraging the relationship between cause and effect to prescribe what action to take.

Statistical analysis is not complete without an estimate of residual uncertainty.

Power considerations drive the sample sizes needed for a successful experiment.

What does 'Why?' mean anyway?

© Bob Wilson 2024 · Powered by the Academic theme for Hugo.