Last Updated on February 3, 2025
There’s something undeniably alluring about playing the armchair quarterback. From the safety of the sidelines, it’s easy to critique, hypothesize, and pontificate about what others should have done differently. It’s low risk, high reward—at least for the pundit.
But without skin in the game, most critiques are hollow. They lack the depth, context, and humility that come from being in the arena, where decisions have consequences, and the lessons you learn are earned in the trenches.
Take the recent uproar over HubSpot’s SEO traffic drop. It’s a case study in why armchair quarterbacking often misses the mark.
HubSpot’s Traffic Drop: A Lesson in Context
Last week, Ryan Law shared a post on LinkedIn that set the SEO world ablaze. According to Ahrefs, HubSpot’s blog traffic dropped from 7.5M clicks in September to 2M in January. Cue the hot takes:
- “AI is killing SEO!”
- “HubSpot should have pruned 300 pages!”
- “SEO is dead—what’s your 2025 strategy?”
The commentary was loud, dramatic, and, unsurprisingly, lacking nuance (outside of a handful of posts that I appreciated, many from former or current HubSpotters).
Here’s what the critics missed:
- It’s a single data point.The traffic numbers came from a third-party tool, not HubSpot’s internal analytics. Tools like Ahrefs are helpful but limited—they don’t capture the full picture of a site’s performance or business outcomes.
- HubSpot’s business and stock price seem to be doing fine.While their traffic fell, their stock price rose from ~$500 to ~$750 during the same period. That’s hardly a sign of a company in trouble.
- Strategy is complex.HubSpot’s SEO program was never about gaming Google for quick wins or even driving last click conversions. It was built iteratively by a smart team through many experiments and lots of data and calibration and it worked to saturate their TAM and beyond, drive mental availability, attract a large amount of both high-intent and low-intent traffic, and nurture those visitors into paying customers through email lists, Academy courses, product signups and more.
Of course, that strategy doesn’t work for everyone. We’ve actively been telling companies not to copy HubSpot’s playbook since we started our agency.
The more interesting story here isn’t what HubSpot should or shouldn’t have done (also c’mon, they inspired a generation of content marketers whether or not you think they should have written about the shrug emoji…)
The more interesting story here is that we as an industry are shockingly quick to cast stones from the outside based on a single traffic screenshot.
Deborah Carver summed it up perfectly:
“I just think it’s curious that people think they understand a content strategy story from a single chart. We, as a population of business professionals, maybe are not as good at understanding data and storytelling as we think we are.”
That last sentence is key: we’re often not as good at interpreting data—or the context behind it—as we think.
Skin in the Game: Why Critique Without Stakes is Cheap
Okay, so here’s my diatribe on armchair critics.
First, there’s no cost to being wrong. Pundits can post bold takes like “SEO is dead” or “HubSpot dropped the ball” or a million other takeaways without facing any repercussions if they’re off the mark. They reap the rewards of engagement—likes, shares, comments—without bearing any of the risks associated with being in the arena.
As Nassim Taleb said in Skin in the Game:
“Don’t tell me what you ‘think,’ just tell me what’s in your portfolio.”
And there aren’t too many HubSpots in critics’ portfolios.
Here’s the asymmetric reality of critique:
- The critic: Gains attention and engagement regardless of accuracy.
- The practitioner: Bears the consequences of every decision—good or bad.
HubSpot’s SEO team has a challenging job. They’re not just theorizing about content strategy; they’re iterating at scale, navigating constraints (and a highly volatile search landscape), and making high-stakes decisions that impact revenue, brand perception, and market positioning.
That’s the reality of being in the arena.
Armchair critics don’t have the same context or real skin in the game. Their observations often lack the humility that comes from experiencing failure or the nuance that comes from solving real-world problems.
The Halo Effect: How Cherry-Picking Misleads
One of the most common mistakes armchair quarterbacks make is falling victim to the halo effect—the tendency to reconstruct success or failure stories by cherry-picking data points that support the end result.
Here’s how it plays out:
- A company succeeds, and we attribute their success to their strategy, leadership, or culture.
- A company falters, and we retroactively identify their “mistakes,” often based on incomplete or biased information.
Books like Good to Great exemplify this phenomenon. They analyze a handful of “winning” companies and attribute their success to specific strategies, ignoring the countless others who employed the same tactics but failed.
Consider Neil Patel’s recent viral claim that 16 companies stopped posting on social media, saw a 94% traffic decline, and lost 6% in revenue. It’s an arresting data point, but without details about methodology—selection bias, control groups, and timeframes—it’s just another speculative narrative.
Critics on HubSpot’s traffic drop have pointed to “low average word counts” and “AI-driven snippets” as causes, but these explanations are reductive, a veritable Rorschach test for pundits (particularly when they haven’t even done a cursory check into Ahrefs let alone the actual internal data).
Why Context and Execution Matter
So, what can we learn from this? The same lessons I always talk about:
- Beware of Oversimplified DataA single traffic chart doesn’t tell the whole story. Real-world systems are complex, and outcomes are shaped by nuanced, interdependent factors.
- Focus on Critical Thinking and First PrinciplesBefore critiquing a strategy, ask: What was the goal? What constraints shaped the decisions? Without understanding these factors, critiques are baseless.
- Run Your Own ExperimentsThe best way to learn is by doing. Theory is a useful starting point, but it’s execution that separates the armchair quarterbacks from the players in the arena.
And obviously, I have to throw that old Teddy Roosevelt line in here somewhere:
“It is not the critic who counts…The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood; who strives valiantly; who errs, who comes short again and again, because there is no effort without error and shortcoming.”
Success Case Studies: It Goes Both Ways
In a sort of funny flip, I remember when I was at HubSpot, I stumbled on a blog post analyzing one of my campaigns.
While the author praised it, they also made questionable assumptions about our strategy. Rather, I should say, they got it almost entirely wrong. Their analysis, while flattering, felt shallow—because they lacked access to the data and constraints that informed our decisions. The campaign they praised was actually fairly lackluster in results ¯\_(ツ)_/¯
My friend Ryan Farley once wrote this incredible piece (that has since, sadly, been removed from the web – but thank you way back machine) on “fake case studies,” where he is essentially talking about breakdowns of successful companies by outsiders that attempt to reverse engineer the simple steps that led them to succeed. In it, he wrote:
“These case studies are cheap and intellectually dishonest. They are harmful because they can mislead people, no matter how good their intentions. When you produce this stuff, there’s a chance that someone actually tries to apply the ‘lessons’ you are teaching. When answers are tough to come by, it’s easy to want an easy answer or to be able to simply adapt what another has found success with.”
I get that these are fun to read, and they are easy to write. But the truly hard things about running and building a generational business, or even a world class organic growth strategy, cannot be boiled down into a single prescriptive blog post by someone who didn’t work on it or even talk to someone who did.
Postscript
Critiquing from the sidelines is easy. Execution is hard.
I want to make it abundantly clear here that I find HubSpot’s traffic drop just as interesting as everyone else does. Maybe more so given I’m an ex-employee. I’m not saying that you can’t learn anything from what happens outside of your own company or analytics account.
I’m not even saying that I’m immune to armchair quarterbacking. I’m out here chirping all the time! It’s a human tendency, this gossip thing.
We called our newsletter Field Notes because we wanted it to reflect actual experiences and lessons we were learning from working, hands-on, with marketing leaders to drive growth. And as such, I felt I had to a) give my alma mater a little credibility for building such an iconic SEO program that it caused the “traffic drop heard around the world and b) remind us to be mindful of low context analyses and opinions in this space.
Per usual, JH Scherck (I know you love it when I mentioned you in these) has the perfect encapsulation here:
“The SEO community never misses a chance to try and diagnose and armchair quarterback a path to recovery for a well known site that’s been hit.
HubSpot is operating at a scale most of us can’t comprehend. These reactionary tactical suggestions are useless and lack context+acumen”
For those in the arena: keep going. Critiques may be loud, but the lessons we learn through action are what truly count.
Want more insights like this? Subscribe to our Field Notes.
