Adventures in Why

A Machine Learning Blog

Bob Wilson

Bob Wilson

Data Scientist

Netflix

Biography

Bob Wilson (he/him/his) is a data scientist at Netflix, where he helps entertain the world by improving content quality. Prior roles include Marketing Analytics at Meta Reality Labs, Director of Data Science (Marketing) at Ticketmaster, and Director of Analytics at Tinder. His interests include causal inference and convex optimization. When not tweaking his Emacs init file, Bob enjoys gardening, listening/singing along to Broadway musical soundtracks, and surfeiting on tacos.

Interests

  • Causal Inference
  • Convex Optimization
  • Theoretical Statistics

Education

  • M.S.E.E. in Machine Learning, 2013

    Stanford University

  • B.S. in Aerospace Engineering, 2008

    University of Illinois, Urbana-Champaign

Recent Musings

Modes of Inference in Randomized Experiments

Randomization provides the “reasoned basis for inference” in an experiment. Yet some approaches to analyzing experiments ignore the special structure of randomization. Simple, familiar approaches like regression models sometimes give wrong answers when applied to experiments. Approaches exploiting randomization deliver more reliable inferences than methods neglecting it. Randomization inference should be the first method we reach for when analyzing experiments.

Sensitivity Analysis for Matched Sets with One Treated Unit

Adjusting for observed factors does not elevate an observational study to the reliability of an experiment. P-values are not appropriate measures of the strength of evidence in an observational study. Instead, sensitivity analysis allows us to identify the magnitude of hidden biases that would be necessary to invalidate study conclusions. This leads to a strength-of-evidence metric appropriate for an observational study.

Sensitivity Analysis for Matched Pairs

Observational studies involve more uncertainty than randomized experiments. Sensitivity analysis offers an approach to quantifying this uncertainty.

Projects

A/B Testing

Calculators for planning and analyzing A/B tests

gamdist

Generalized Additive Models in Python

orbpy

Orbit Propagator in Python

Homebrew-Calc

Homebrewed Beer Calculator

Unit Parser

Unit Parser and Conversions in Python

Other Papers

Star Identification via Computer Vision Techniques

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Duis posuere tellus ac convallis placerat. Proin tincidunt magna sed ex sollicitudin condimentum.

A Discussion of Relativistic Phenomena and Construction of Spacetime Diagrams

We discuss how the Special Theory of Relativity proceeds from the absence of an absolute definition of stationarity, as well as the observation that light travels at the same speed in all reference frames. Some interesting phenomena follow: two observers in relative motion cannot always agree on the length of an object, the time between two events, or even in what order the events occurred.

Recent & Upcoming Talks

Beyond A/B Testing: Getting More from Experiments

In my journey to improve the design and analysis of A/B tests, I have turned to the literature on observational causal inference. Along the way, I have learned several techniques to level up experiments. These techniques include tests of equivalence and non-inferiority, closed testing procedures, methods for non-compliance, and heterogeneous treatment effects.

Causal Inference and A/B Testing

Interana invited me to give a talk on A/B testing and analytics at Tinder.