Our presentation at the 2021 Innov8rs Connect Unconference.
Thanks to rapid advances in tools and techniques, the barriers to exploring new ideas and making them happen have never been lower for organizations. At the same time, this swiftness of change means that modern discovery practices are developing well ahead of where most teams operate today.
In this session from the 2021 Innov8rs Connect Unconference, S&P's Scott McDonald and Anuraag Verma of Feedback Loop share stories of how large, legacy organizations are operating at the forefront of modern research and experimentation; maneuvering like startups to identify their most promising growth opportunities and test more ideas faster. We look at the increasing frequency of customer contact, the variety of methods they’re using and the results they’re achieving.
The old excuses for not engaging with customers more often simply don’t apply anymore. In fact they’ve been turned on their heads.
“For every good idea there are a thousand bad ideas it is indistinguishable from. The only real way to tell the difference is to go out and try them, see what works, discard the failures and build on the successes. You have to, in other words, dare to be crap.”
— Marc Randolph, Co-founder, Netflix
Perhaps Randolph was a bit dramatic, but he makes an important point. We have to wade through a lot of crap ideas to identify those relative few that will grow our businesses and secure our futures.
Yet it’s still surprisingly common today for companies to bet big on launching a few cherished ideas into the marketplace each year, with little in the way of pre-market testing.
We don’t have to do that anymore, and we can’t afford to either. Here’s why…
The innovation deck is stacked heavily against us
A growing number of studies show that only about 10% of new business ideas will find some success in the marketplace, and just one in 250 will hit it big ($1+ billion). So despite the mythology and romance of the “big idea” in business, innovation today is as much a process of elimination as it is of inspiration. And all that elimination needs to happen quickly. So fortunately…
The opportunity cost of testing new ideas is plummeting
It has become ridiculously cheap and easy to test our ideas in the marketplace. Frameworks are maturing. Research platforms are more powerful than ever and a growing array of no-code tools are giving non-developers the ability to quickly prototype all sorts of applications on their own. Today you can drag and drop an e-commerce website into existence using an app on your phone.
Where we once rushed to launch, we can now rush to learn.Experimentation as a core business practice and leadership philosophy is exploding.
We have a new generation of powerful research platforms like Feedback Loop, Suzy and Remesh that allow teams to run a variety of surveys and experiments with greater speed and scale than ever before.
There’s a new breed of consulting firm like Exponentially, LeanApps and Precoil that test new ideas for organizations, or teach them how to run experiments themselves.
No matter what your challenge, customer, product or market, there are almost always ways to for test product-market fit early.
A new line of cosmetics? Test the packaging first.
A new service offering? Create a mock brochure or sales deck and start selling now.
New consumer product? Create a landing page and run some inexpensive ads.
Jeff Hawkins famously tested his Palm Pilot idea by carrying around a piece of wood with drawn-on buttons. Just get your idea out there fast, listen carefully and above all don’t get too attached to it.
“If you double the number of experiments you do per year you’re going to double your inventiveness.”
— Jeff Bezos
What can go wrong when we don’t test early and often? Consider the case of streaming startup Quibi.
The $1 billion learning experience
Quibi (short for “quick bites”) seemingly had it all. A-list co-founders in Jeffrey Katzenberg and Meg Whitman. $1.7 billion in funding from investors like Disney, 21st Century Fox, NBCUniversal, Sony Pictures, Viacom and Alibaba and a level of buzz most startups can only dream about.
Yet six months after launching in April of 2020 and having spent over $1 billion, the founders announced they were shutting the service down after failing to gain traction with subscribers.
“We had a new product and we asked people to pay for it before they actually understood what it was. I think we thought there would be easier adoption,” Katzenberg said. “In the end, we didn’t get the support of consumers and customers in the way we had to to make this a successful business.”
The company did run experiments. Only too late in the game. “Over the summer we started to see a slowdown in our momentum, and we tried many different things, different packaging models, we changed our marketing, we changed the app around many different times but it was clear that for whatever reason this wasn’t going to be as successful as Jeffrey and I had hoped,” Whitman told CNBC at the time.
“Failure is part of the Innovation game, but expensive failure is not a necessary part of the process.”
— Tendayi Viki
Contrast Quibi with Farmers Insurance and its new platform aimed at millennials, Toggle.
Farmers, a Fortune 500 company in a highly regulated industry, worked with the rapid consumer insights platform Feedback Loop (formerly Alpha) to launch a new online business 13 months ahead of plan. In that time the team covered more research ground than most leaders might think possible:
175 surveys and experiments conducted in nine months.
54,000 research participants.
40 audience segments explored.
Reduced the time from ideation to product launch from a projected 18 months to five months.
The old excuses for not engaging with our customers more often (no time, no budget) simply don’t apply anymore. In fact they’ve been turned on their heads.
The classics endure. Recently Jared Spool tweeted a link to an article he wrote in 2011 (Fast Track to a Great UX — Increased Exposure Hours) that remains as vital for innovators today as ever. For those who missed it the first time, here’s a recap.
Photo by Pavan Trikutam on Unsplash
In the article Spool described what he called the closest thing he’s ever found to a silver bullet when it comes to reliably improving the products and experiences that organizations produce.
It’s called Exposure Hours: The number of hours your team members are exposed directly to real users interacting with your or your competitors’ products. There is a direct correlation, he found, between this exposure and the improvements we see in the products a team produces.
“For more than 20 years, we’ve known that teams spending time watching users can see improvements. Yet we still see many teams with regular user research programs produce complicated, unusable products. We couldn’t understand why. Until now.”
Spool isn’t the first person to champion spending time with customers. However, he might be the first to spend several years observing teams doing it and documenting his findings in a scientific way.
Here is his deceptively simple formula for success:
Once-a-year research isn’t enough.
Ongoing, direct exposure to customers is key.
Exposure not just by design teams, but at all levels of the organization.
More exposure = better products.
“We saw many teams that conducted a study one a year or even less. These teams struggled virtually the same as teams who didn’t do any research at all.”
Direct contact
This part is crucial for teams that rely on their research department or commercial providers for customer insights. Each team member has to be exposed directly to the users themselves. Teams with dedicated user research professionals who conduct research and then disseminate the results through documents or videos don’t see the same results.
The teams with the best result were those that kept up their research on an ongoing basis, using a variety of research methods. Six weeks was the bare minimum for a two-hour exposure dose. The teams with members who spent the minimum of two hours every six weeks saw far greater improvement to their user experiences than teams who didn’t meet the minimum. And teams with more frequent exposure, say two hours every three weeks, saw even better results.
By everyone
Not just designer and developers. Teams that excluded non-designers, like executives and business stakeholders, from user contact didn’t see the same advantages as teams that included them. The tipping point came when all were included. This part will resonate with designers who regularly urge their colleagues and clients to sit in on research sessions.
“While core design team members became very familiar with what users need and want, they were constantly battling with their other colleagues who didn’t have the same experiences.”
Whoever knows their customer best wins
As Spool pointed out, exposure is wonderfully easy to measure. Just count the hours. He even saw organizations including it in their quarterly performance reviews.
Two hours of customer exposure. Every six weeks. For everyone. The classics endure.
