October 7, 2021

Thinking in Experiments: Today’s Winning Mindset

The old excuses for not engaging with customers more often simply don’t apply anymore. In fact they’ve been turned on their heads.

“For every good idea there are a thousand bad ideas it is indistinguishable from. The only real way to tell the difference is to go out and try them, see what works, discard the failures and build on the successes. You have to, in other words, dare to be crap.”

— Marc Randolph, Co-founder, Netflix

Perhaps Randolph was a bit dramatic, but he makes an important point. We have to wade through a lot of crap ideas to identify those relative few that will grow our businesses and secure our futures.

Yet it’s still surprisingly common today for companies to bet big on launching a few cherished ideas into the marketplace each year, with little in the way of pre-market testing.

We don’t have to do that anymore, and we can’t afford to either. Here’s why…

The innovation deck is stacked heavily against us

A growing number of studies show that only about 10% of new business ideas will find some success in the marketplace, and just one in 250 will hit it big ($1+ billion). So despite the mythology and romance of the “big idea” in business, innovation today is as much a process of elimination as it is of inspiration. And all that elimination needs to happen quickly. So fortunately…

The opportunity cost of testing new ideas is plummeting

It has become ridiculously cheap and easy to test our ideas in the marketplace. Frameworks are maturing. Research platforms are more powerful than ever and a growing array of no-code tools are giving non-developers the ability to quickly prototype all sorts of applications on their own. Today you can drag and drop an e-commerce website into existence using an app on your phone.

Where we once rushed to launch, we can now rush to learn.Experimentation as a core business practice and leadership philosophy is exploding.

  • We have a new generation of powerful research platforms like Feedback Loop, Suzy and Remesh that allow teams to run a variety of surveys and experiments with greater speed and scale than ever before.
  • There’s a new breed of consulting firm like Exponentially, LeanApps and Precoil that test new ideas for organizations, or teach them how to run experiments themselves.
  • There’s even a movement afoot to make experimentation skills part of the core curriculum of MBA programs. See Why Business Schools Need to Teach Experimentation.

Look before you leap

No matter what your challenge, customer, product or market, there are almost always ways to for test product-market fit early.

  • A new line of cosmetics? Test the packaging first.
  • A new service offering? Create a mock brochure or sales deck and start selling now.
  • New consumer product? Create a landing page and run some inexpensive ads.

Jeff Hawkins famously tested his Palm Pilot idea by carrying around a piece of wood with drawn-on buttons. Just get your idea out there fast, listen carefully and above all don’t get too attached to it.

“If you double the number of experiments you do per year you’re going to double your inventiveness.”

— Jeff Bezos

What can go wrong when we don’t test early and often? Consider the case of streaming startup Quibi.

The $1 billion learning experience

Quibi (short for “quick bites”) seemingly had it all. A-list co-founders in Jeffrey Katzenberg and Meg Whitman. $1.7 billion in funding from investors like Disney, 21st Century Fox, NBCUniversal, Sony Pictures, Viacom and Alibaba and a level of buzz most startups can only dream about.

Yet six months after launching in April of 2020 and having spent over $1 billion, the founders announced they were shutting the service down after failing to gain traction with subscribers.

“We had a new product and we asked people to pay for it before they actually understood what it was. I think we thought there would be easier adoption,” Katzenberg said. “In the end, we didn’t get the support of consumers and customers in the way we had to to make this a successful business.”

The company did run experiments. Only too late in the game. “Over the summer we started to see a slowdown in our momentum, and we tried many different things, different packaging models, we changed our marketing, we changed the app around many different times but it was clear that for whatever reason this wasn’t going to be as successful as Jeffrey and I had hoped,” Whitman told CNBC at the time.

“Failure is part of the Innovation game, but expensive failure is not a necessary part of the process.”

— Tendayi Viki

Contrast Quibi with Farmers Insurance and its new platform aimed at millennials, Toggle.

Farmers, a Fortune 500 company in a highly regulated industry, worked with the rapid consumer insights platform Feedback Loop (formerly Alpha) to launch a new online business 13 months ahead of plan. In that time the team covered more research ground than most leaders might think possible:

  • 175 surveys and experiments conducted in nine months.
  •  54,000 research participants.
  • 40 audience segments explored.
  • Reduced the time from ideation to product launch from a projected 18 months to five months.

The old excuses for not engaging with our customers more often (no time, no budget) simply don’t apply anymore. In fact they’ve been turned on their heads.

Vision or hallucination?

There’s only one way to find out.


Recommended Reading


Subscribe to our quarterly email newsletter for the latest insights from our work.

July 29, 2021

Innovators and Entrepreneurs: Why Coding Should Be Your Last Resort

Your business – everyone's business – is product-market fit. And we rarely need valuable programming talent to find it anymore.

When creating new technology products and businesses, it was once standard practice to build first and ask questions of the marketplace later. It was a risky and expensive way to launch untested ideas, but it was how things were done.

This habit persists in many organizations today, and it’s increasingly problematic for two reasons:

  1. The cost of learning is too high. Today’s design and testing tools make it easier than ever to create realistic prototypes of ideas and validate them in the marketplace without building them first.
  2. The cost of failure is even higher. The accelerated pace of change today and the high number of bets required to discover winning ideas have made it too wasteful, slow and impractical for organizations to code software ideas before they absolutely have to.

What’s the harm if we build it and they don’t come? Consider the story of Trov, a startup provider of on-demand insurance for consumer items like bicycles, personal electronics and sporting goods. Industry website Coverager reported last June that Trov had just pivoted from its direct to consumer business model after seven years and $99 million in investment. According to Coverager, even where Trov did find a fit, the market was too small and the product too complicated to scale.

“In today’s world, it should never take years and $99 million to learn that we don’t have product-market fit.”

While Trov might be an extreme example, there are dozens of mini-Trovs unfolding today in most large organizations.

Too much, too soon?

The movement away from building early took off in 2011 when Eric Ries published The Lean Startup and introduced the idea of the Minimum Viable Product, or MVP, which allowed teams to collect “the maximum amount of validated learning about customers with the least amount of effort and investment.”

Although a huge step forward, those MVPs still required teams of engineers and a few months or longer to get working software into the hands of customers.  

In a blog post last Autumn (“Avoiding the Wrong MVP Approach”), software usability expert Jared Spool described a team that was exploring whether auto insurance customers could upload accident photos from their phones that were comparable in quality to the photos provided by expensive professional photographers (the answer was yes).

The team considered coding a streamlined version of the idea as an MVP. But as Spool put it, even a limited functionality software version for customer-supplied photos would still require a lot of work and disruption.

They’d need a way to upload the photos to the company mainframe system used for claims processing. They’d have to build something that attached the photos to customer claim records. They’d have to change the adjusters’ workflow. That’s a lot of difficult system-wide changes for one unproven idea. 

“Organizations are so used to solving every problem with software that we forget that we can learn what we need by faster, more effective means.” 

— Jared Spool, UIE

He went on to describe a series of clever offline experiments that were used to validate the idea instead. 

Now, thanks to new developments, it appears we can finally have it both ways.

Enter the no-code movement

Imagine conceiving a new generation of digital products and business models without writing a single line of code. Increasingly we can thanks to the no-code and low-code movements: A drag-and-drop, visual approach to software development that allows even non-technologists to quickly create functioning, scalable software experiences for very little cost.

Challenges and opportunities that were previously addressable only by those with technical know-how or deep enough pockets can now be taken on by any team or entrepreneur. 

The movement has been quietly gaining momentum for years, but in the past 12 months it has exploded in profile and capability. High-volume e-commerce marketplaces with payments built in, subscription publishing platforms, web and native mobile apps, voice bots and more can now be built entirely with no-code tools like Webflow, Bubble, Airtable and Glide.

Martin Slaney, a London-based entrepreneur and product leader, made some bold and exciting predictions about the potential of no-code platforms on Medium last October (“The Future of Product Management is No-Code Development”).

Slaney spent six months immersing himself in the world of no-code and he is convinced that pretty much anything that can be done with code will be done without it. There is a learning curve, he points out. But it’s much more manageable than learning to build software from scratch.

Are we putting coders out of a job?

Far from it. According to TechRepublic, an estimated one million computer programming-related jobs in the US were expected to go unfilled in 2020. As innovation leaders, we can do our part to solve the global talent shortage by utilizing programmers only when – and where – they’re really needed.


Subscribe to our quarterly email newsletter for the latest insights from our work.

Partner with us: hello@shavrick.com

Subscribe to our newsletter

©2022 Shavrick & Partners, LLC all rights reserved

Partner with us: hello@shavrick.com

Subscribe to our newsletter

©2021 Shavrick & Partners, LLC all rights reserved