Former Theranos CEO Elizabeth Holmes was convicted on four counts of wire fraud and conspiracy to commit fraud in early January. The trial brought the dramatic rise and fall of this medtech start-up back to the public eye.
Holmes had a revolutionary, direct-to-consumer vision of healthcare. “The implications are mind-blowing,” said a 2014 article in Wired. “With inexpensive and easy access to the information running through their veins, people will have an unprecedented window on their own health.” (Wired, Feb. 2014).
But the vision did not materialize. Theranos developed a miniaturized blood analyzer and then covered up the fact that it didn’t work as advertised.
What could save you from Theranos' mistake?
1. Lesson: Lead with the need
User validation starts way up front. Your value proposition fails or succeeds on whether the user needs you identify in validation can be effectivley met. Theranos could have validated their “big idea” earlier and addressed failures before problems were baked in. This is not such a rare occurrence. Many innovative organizations avoid testing. And like Holmes, many neglect to test with users.
2. Lesson: Test for effectiveness not efficacy
User testing has to “get out of the building,” as Tomer Sharon says. The temptation for a scientific startup is to only test under controlled, ideal conditions. This is necessary but not sufficient. Thorough user testing involves REAL users in a REAL work context, doing REAL tasks. Iterative usability testing is observational. It often helps discover user experience flaws that cannot be found in a lab or QA testing. It studies behaviours that emerge under real conditions. Basing design decisions solely on efficacy is not sufficient.
Thorough user testing involves REAL users in a REAL work context, doing REAL tasks.
Do you really think you will learn from Theranos' mistakes?
Poor product development is not the only reason Theranos and Elizabeth Holmes are under scrutiny. But it is a lesson we ignore at our peril.
Every bonafide big idea must meet user needs, and designers need to scope the project to support tasks that are of high value to users. But, in this example, there was a limited amount of testing with real users doing real work.
This became central to the accusations against Theranos’ management team – too little validation in real-life situations. Device design was not validated even under simulated use conditions nor did the technology demonstrate that it met user needs and intended use cases.
There’s a lesson here for all of us: test with users on real-life tasks using simulated mock-ups. Iterative design and testing could have saved billions of dollars.
3. Lesson: Listen to whistleblowers in user testing
Theranos needed to address the problems with its technology discovered by various internal groups. Their lack of a rigorous method for improving effectiveness became evident in 2015 when the Wall Street Journal published exposés alleging incompetent management. It became clear the company had been aware of shortcomings discovered during field testing.
One former employee described problems with the Edison blood analyzers on 60 Minutes in 2018: “There were components that would fall off in the middle of testing that you would then have to fish out. They had doors that wouldn’t close. They would get too hot. And then they would get too cold.”
4. Lesson: Test early and often
Some biomedical experts have also expressed skepticism with the concept behind testing only a small amount of blood taken from a finger. But trial units were installed in pharmacies, without having addressed the skepticism they encountered.
We don’t know whether Theranos avoided its negative usability findings, but they could have listened more rigorously. Perhaps management wanted to please their charismatic CEO. Dismissing negative findings can make a testing environment toxic. Management could have been pressured to achieve profitability before making iterative design changes.
The company built on Holmes’ vision crumbled under the weight of the WSJ coverage and investigations by federal health regulators.
Theranos’ big idea and the device it relied on were not exposed to the appropriate scrutiny, testing and iterative development needed for a ground-breaking technology. There is a method to bring disruptive medical technology to market, and this is not it.
