Leveling up in Product Analytics in 2024

Where to direct your free time (and a learning & development budget, if you are so lucky) to stand out as a product analyst?

When I started my product analytics endeavors - more than six years ago now, I had almost no clue what ‘product management’ was and my analytics experience was largely academic (via my economics degree) plus a couple of years of “media math” (via working in audience measurement industry). It was also before I discovered all the great online communities and got a chance to connect with peers. So I spent a great deal of time and effort on my own trying to figure out not only ‘where to learn stuff’, but also ‘what to learn’.

What I have discovered is that, sadly, improving my coding skills, while always good to do, has very much diminishing returns, and doesn’t provide as much return on investment in a product analytics role compared to learning the subject matter. Why “sadly”? Because it is much harder, especially if you are not yet in a product analytics role and can’t learn by doing. There are so many coding bootcamps and “technical” courses out there, but very few structured resources that teach you how to leverage data in the product building process with maximum impact.

Since I’ve already done all this scouring for resources, I figured I will share my top picks. In January lots of folks are planning their learning agendas and career transitions, so I hope this helps you if you are looking to get into a product analytics role, or trying to find ways to create more impact.

Before I get into my ‘top 3’ rankings, I will say that if I had to pick a single resource for someone who is just starting out in the product analytics realm, it would be the Data-Informed Product Building series by Sequoia. It is free, it is expansive, and introduces some key concepts that stayed with me throughout my career: growth accounting fundamentals and how metrics evolve throughout a startup’s growth progression, and the madlibs ‘engagement drives stickiness drives retention drives growth’ (which really is product analytics in a nutshell and you should definitely remember it during ‘case study’ analytics interviews!).

Alright, now into the rankings.

And disclaimer - I am not affiliated with any of my recommendations (except for myself!).

Books

1. Lean Analytics: Use Data to Build a Better Startup Faster. E.Ries (Purchase Link / Library Search)

   This book is definitely more product manager / founder-focused, so it is not very technical, so it can be a fairly quick read. This book is a great first foray into understanding the company’s business model and zeroing in on key product metrics that tie-in with the business outcomes. I have this book on my shelf and still use it as a reference material for initial stage of KPI framing as it conveniently groups them in product categories (e.g. ecommerce, SaaS, etc.)

2. Trustworthy Online Controlled Experiments. R. Kohavi (Purchase Link / Library Search)

   This book is considered somewhat of a “bible” of A/B testing. And in contrast to the previous pick, this is a more technical read, although it also largely leaves complex math/stats out, so don’t hesitate to pick it up. I like that this book doesn’t just cover the key pieces of an A/B test design (metrics, experiment design), but it also goes in depth on common pitfalls, elements of the technical setup (e.g. client-side vs server-side), as well as organizational stages of ramping up experimentation culture. I find that this book is written from a perspective of a more mature tech company, so it doesn’t quite get into messiness of navigating startup constraints and the day-to-day of collaborating with a product team on the experiment’s lifecycle, but it helps with a solid technical foundation.

3. Product Analytics: Applied Data Science Techniques for Actionable Consumer Insights. J. Rodrigues (Purchase Link/Library Search)

   This book is not all that well-known, so I’m excited to get more folks to check it out. It is a more applied resource - it doesn’t focus as much on the breadth of possible problems and variation of solutions depending on the product and the business model, but more so gives examples of specific statistical techniques. And it goes beyond exploratory analysis and experimentation, but also dips a little into regression and causal methods (code examples are in R only!). I use this book for inspiration when I’m not totally sure how to tackle a new problem.

Honorable mentions:

• Teresa Torres’ Continuous Discovery Habits (Buy): this book can give you some inspiration for how data insights can mesh with qualitative research/user feedback and be used for proactive/generative purposes (vs evaluating stuff that’s already built)

• Christina Woedtke’s Radical Focus (Buy): as a product analyst, you will inevitably get pulled into OKRs one way or another; while I think that analysts should never own OKR definition and spending too much time on refining OKRs is not a productive time spent, it is useful to be able to plug data and metric targets into the planning process. I like Christina’s OKR-focused books as it presents a more realistic (for a startup) framing, compared to John Doerr’s “OG” materials, and it is laid out as a story, so it is actually fun to read.

Courses

1. 💲 Product Analytics on Uplimit (Link)

   I would be remiss not to recommend my own course! Over four weeks I cover all the main aspects of analytics work that are unique to product analytics: framing product metrics in a structured way with a tie-in to business outcomes, most common analysis techniques (cohort analysis, ‘key moment’ identification), data collection and relevant infra/tools, as well as experimentation. This course is technically inclusive (all projects can be done if google sheets, with an option to use SQL or Python), but it teaches an invaluable skills of solid qualitative framing before jumping into analysis, so it will challenge you in new ways. New cohort starts Feb 19 (it can be followed both live and async). Use code ELENAD20 for a 20% discount.

   Note that Uplimit has a pretty economical ‘unlimited subscription’ option (in case you have $1k of a learning & development budget to spend). And there are a number of great courses that can be very helpful in the product analytics space too. I took the Practical A/B testing and Causal Inference for Data Science courses last year and picked up lots of great techniques from both!

2. 💲💲💲Reforge Courses

   Reforge materials are largely targeted at product managers, but the flavor of product management they teach is very much infused with quantitative foundation. So, for a data person that can transition what they learn in Google Sheets into the world of messy real data and figure out how to apply more rigor to discussed methods, this is an invaluable resource for understanding how to quantify the levers of product growth. Another valuable thing about Reforge is that their material is infused with real examples from well-known startups and live courses usually have industry speakers breaking down case studies from their personal experience.

   The downside of Reforge is that it is very expensive!

   The specific courses I would recommend are: Advanced Growth Strategy (if you can only afford one, that’s the one!), Retention and Engagement, and Experimentation + Testing. They have also recently launched a Product Analytics course that I haven’t had a chance to check out yet, but its author Crystal Widjaja is one of my favorite writers on the topic.

3. 💲💲GoPractice’s Data Driven Product Management Simulator by Sean Ellis and Oleg Ya. This is not a traditional course, but a self-paced “simulator” where you work through several product cases around launch metrics, experiments, light forecasting interspersed with bits of theory. It is great for folks just starting out that like to learn by doing vs. watching videos. It is targeted mainly at product managers, so it doesn’t introduce coding (the data work is done in Amplitude, so it is also a chance to learn the tool) or go into any meaty stats.

Blogs and Online Resources

1. Sequoia’s Data-Informed Product Building Series. As I mentioned above, if I could pick a single resource for someone just starting out within the product realm, that would be it. It goes through a gamut of metrics-related concepts from a growth angle, covers measuring impact of various phenomena (from product changes to external factors), and wraps up with a discussion on building data teams and important data skills. And it is free (although if you read the whole thing, you may need a Medium membership)

2. 💲Olga Berezovsky’s Data Analysis Journal Newsletter. While not strictly product analytics-related, Olga has recently tackled a bunch of relevant topics, from data collection to running A/B tests on small samples. And I really love her ‘digest’ issues where she collaborates with a product analytics guru Timo Dechau to highlight fresh articles, podcasts, studies on relevant topics.

3. 💲Lenny Rachitsky’s Lenny’s Newsletter and Lenny’s podcast. Also not strictly a data/analytics-focused resource, Lenny covers best practices in product building (with participation of industry folks) that often cover the use of metrics, and he on his podcast he’d interviewed many data-adjacent product folks like Ronny Kohavi (the author of the A/B testing “bible”), Crystal Widjaja (Reforge’s Product Analytics instructor), Elena Verna, and Dan Hockenmaier (both renowned thought leaders in the growth space). Lenny’s newsletter is paid, but the podcasts is, of course, free.

Honorable Mentions:

• Amplitude’s playbooks: Retention, Engagement, North Star Metric, Product Strategy. Another great free introduction to some key metrics concepts + some brief case studies.

• John Cutler’s The Beautiful Mess blog. John is a brilliant product thinker, who focuses on the organizational dynamics and interactions of various parts and processes of product-building, naturally, the topic of metrics / OKRs / quantified incentives comes up often. John used to be the Product Evangelist at Amplitude, so a lot of his earlier writing somewhat overlaps with the aforementioned playbooks.

• Reforge’s blog. Another free resource that has some comprehensive data-adjacent pieces tackling various aspects of product strategy. The ‘Why most analytics efforts fail’ post by Crystal Widjaja is an absolute classic. I also like materials on marketing attribution and “word of mouth”

• Cedric Chin’s Commoncog. Cedric’s blog covers the use of data in business decision making, so it is not strictly a product-related blog, although lots of takeaways apply to the product realm too. Cedric also has a series of posts that break down the concept of ‘weekly business review’ which drove Amazon’s data-driven product building.

Alright, I’m going to stop right here, even though I could keep going, especially getting into some analytics-adjacent areas like behavioral design, growth, (product) marketing analytics, but then we will be right where we started - abundance of options and no clear way to go.

I hope that this list was helpful! Check these materials out and report back. And let me know in the comments if I missed anything great!

---