Field Notes #004: Bandit Testing Your Content Marketing

Being “data-driven” doesn’t mean you operate without creativity. It means you use data, where you can, to make better decisions. Data literacy can become a competitive advantage, but an overreliance on historical data can keep you from innovating.

I’ll give you an example from my own personal blog. The goal of my blog is twofold: to drive affiliate revenue and to build an audience and a personal brand for myself.

To be data-driven, I need to map those goals back to KPIs. Personal branding KPIs are tough; they’re usually measured with traffic, engagement, and qualitative feedback.

Affiliate KPIs are easy: rankings, traffic, and ultimately, revenue driven by my content.

You can see already that what’s easy to measure is easy to prioritize. Building my personal brand, over the long run, might make me a lot more money than optimizing for affiliate revenue. But I can’t see that in Google Analytics, so I write more listicles:

And even with the affiliate content I choose to write, I can be swayed in a suboptimal direction by over-relying on historical data. For example, you can see that I write a lot about AI copywriting and content optimization tools. That’s because I’ve seen, with my limited data, that they convert well.

But what if there’s a software category I haven’t even written about that would convert 10X higher and bring even more revenue? There’s no data to suggest that’s the case, but that’s only because *I* don’t have access to that data.

That’s where exploration comes in.

There’s a concept in machine learning known as hill climbing. The goal is to iterate your way to the highest possible point on a multi-dimensional map, but often, you end up hitting a “local maximum.” A local maximum is the top of a hill, but it might not be the tallest hill possible.

In the beginning, you have to take some stabs in the dark because you essentially have no data, only hypotheses. It’s also hard to rank for content because your domain is relatively weak. Big swings are the only thing that registers among all the competition.

In later stages, you have a ton of data to work with and can predict, to a degree, how content will perform in search, traffic, and conversions.

But no matter how mature the company, always keep a small basket open for big swings and uncertain bets. It might be 80% data-driven, predictable, product-led content, but that still leaves 20% for novel initiatives. The novel initiatives might fall flat, or they might be 100X as valuable. We’ll never know until we try.

This is the essence of the Barbell Strategy. It’s also inspired by the “bandit problem” in machine learning, which involves balancing “exploring” to find new options, and “exploiting” where you heavily invest in actions that are already shown to work very well.

A good strategist is capable of building a plan that accounts for both known data and creating space for a portion of bets that are left open to surprise.