Posts

I started a new job recently and took the opportunity to install a new version of Emacs. Emacs 29 includes tree-sitter and built-in eglot support, which I’ll write about some other time. In this post, I just want to document how I compiled Emacs on an M2 macos device.

Imagine we are attempting to identify segments within an audience, perhaps so we can market to them more effectively through personalization. A common approach to doing so is to apply some kind of clustering algorithm (such as K-means) based on various user covariates. I have never been especially happy with this approach: the resulting clusters seem pretty arbitrary.

A simple approach to heterogeneous treatment effect estimation relies on a difference in approximations to the outcome function among the two treatment groups. In this post, I derive the conditions under which this approach works.

Lately I’ve been reflecting on regularization. Early in my data science career I spent some time working with generalized additive models, but I started focusing more and more on traditional statistical methods. I am re-finding the value in regularization and expect to use more of it going forward.

When running an A/B test with many variants, say, more than 5, we often run into a phenomenon known as the Winner’s Curse, where the winning variant performs worse when we adopt it universally than it had during the test itself. In this post, we discuss why this phenomenon occurs and what to do about it.

As 2021 wrapped up, I’ve been reflecting on the past year and thinking about the next. I was similarly reflective this time last year, when I wrote about how 2020, for me, was the Year of Emacs.

In our last post, we discussed how model uncertainty poses a risk when allocating resources among productive assets. In this post, we expand the discussion to models with diminishing returns. Such models are common in economics. As before, we can incorporate model uncertainty directly into the problem, achieving good performance regardless of the true model, with minimal impact to nominal performance. Robust optimization is both powerful and practical.

Often we run an A/B test in order to inform some decision. But every A/B test involves uncertainty, no matter the sample size. This uncertainty poses a risk to our decision, which can be hedged by a process analogous to diversifying an investment portfolio. Finding a robust-optimal portfolio is both practical and fast.

The OODA loop (Observe-Orient-Decide-Act) framework was developed by USAF Colonel John Boyd to improve fighter pilot performance in the field. We can apply a similar framework to improving the efficiency with which we develop data science models. The key insight is to embrace the iterative nature of model development and streamline each component of these iterations.

Some of the most important questions data scientists investigate are causal questions. They’re also some of the hardest to answer! A well-designed A/B test often provides the cleanest answer, but when a test is infeasible, there are plenty of other causal inference techniques that may be useful. While not perfect, these techniques are much better than the alternative: ad hoc methods with no logical foundation.