Metrics Aren’t Permanent

The past couple posts highlighted two common but nuanced issues that cause metrics to be misjudged or misinterpreted: (1) a representational gap between what is measured and what is “ground truth reality”, or what one wishes to actually measure; and…

Read More →

Goodhart’s Law

It struck me as somewhat ironic, and sometimes uncomfortable, that many times I had to tell stakeholders not to trust a metric that I helped define. As a data scientist, this was always a tricky tightrope to walk. While you…

Read More →

The Metric is Not the Product

The bulk of my time as a data scientist was spent on what we called Evaluation, which entailed designing experimentation platforms, helping teams run and analyze experiments, and defining success metrics. All in service of answering the question: “how do…

Read More →

Hello Again, World

(FYI, I’m intending to cross-post upcoming entries to Substack, in case it’s easier to follow there: https://seantime.substack.com/) I recently took a break from working on Google Search, after 15 years as a data scientist and most recently as an acting…

Read More →

Non-AI Pact

(FYI, I’m intending to cross-post upcoming entries to Substack, in case it’s easier to follow there: https://seantime.substack.com/) I now have a bit of free time on my hands, and have the itch to do some personal writing again, after roughly…

Read More →