Cherry-picking data was routine practice at Theranos, former lab worker says

At Theranos, lab techs would regularly omit two “outlier” data points from analysis in order to get the company's machines to pass quality checks, whistleblower Erika Cheung told jurors yesterday at the trial of Theranos founder Elizabeth Holmes. 

The process of removing outliers from an analysis isn’t necessarily unusual. But Theranos’ own lab manual did not say how outliers should be identified, so to get the company’s proprietary blood-testing devices to pass quality checks, employees could decide which results to keep, essentially “cherry-picking” data, Cheung said. 

“It was very concerning in a research context because once that translates to a patient setting, it’s giving you a good indication that the system isn’t working reliably enough to feel confident and comfortable in running patient samples,” she said.

Cheung joined Theranos after graduating from UC-Berkeley and worked both in the company’s research and development lab and in its clinical lab, running quality-control tests to validate the Edison machines’ performance. To put it bluntly, the performance does not sound good—on any given day, about 25 percent of the Theranos machines failed tests, a rate far higher than competitors’ devices, she said.

Hacking lab equipment

Cheung would know—the company used third-party equipment both as benchmarks and to run patient samples. Though Theranos had told investors that the Edison devices could perform more than 200 different blood tests, Cheung told the court that it could do 12 at best. The rest were run on machines not made by Theranos, including some that had been hacked to accept smaller blood samples than they had initially been designed for.

“The machine wasn’t built to be able to do the processing on that small of a volume,” she said.

Most blood tests are run on blood drawn from a person’s vein. Compared with the finger-prick tests that Theranos touted, vein samples are less likely to be contaminated and are larger in volume. The larger sample helps minimize variabilities that could throw a test result. Finger pricks produce less blood, making the method less likely to produce results that are representative of a person’s biology. 

Cheung told the court that she took her concerns to her superiors and eventually raised them with Chief Operating Officer Ramesh “Sunny” Balwani. “The feedback and reception that I got from him was essentially, ‘What makes you think you’re qualified to make these calls?’” she said. Cheung said she also told former secretary of state and Theranos board member George Shultz about the Edison devices' problems.

“It was starting to get very uncomfortable and [it was] very stressful for me working at the company, and I was attempting to tell as many people as I could, but it was not seeming to get through to people,” she said.

Balwani gave her an ultimatum: Run the tests or quit.

Eventually, Cheung did quit, and she sent an official complaint to the federal agency that regulates clinical labs, the Centers for Medicare and Medicaid Services. She also began talking with an investigative journalist at The Wall Street Journal (John Carreyrou, who started pulling the thread that led to the company’s unraveling.) After Cheung quit, Theranos lawyers sent her a letter saying they believed she shared confidential company information and that the company would sue her if she continued to make “false statements” about Holmes.

In cross-examination, the defense emphasized that Cheung had a low-level role, which she admitted was correct. The defense read a list of Cheung’s superiors, noting their advanced degrees, including PhDs and MDs. Holmes’ lawyers also shared Theranos’ 120-page lab standards document, which requires lab assistants like Cheung to have a bachelor’s degree in science. (As Elizabeth Lopatto pointed out at The Verge, “That means Holmes wasn’t qualified even to do that work.”)

More emails and texts

In other developments, Judge Edward Davila allowed jurors to see more emails that Cheung sent to Theranos employees but which didn’t copy Holmes. He also made the texts between Holmes and Balwani public. In the threads, the pair privately trashed Walgreens, which had partnered with Theranos to roll out the proprietary Edison blood-testing devices in its pharmacies. A year after the partnership was announced, Balwani texted Holmes with his thoughts. “We can’t scale with [Walgreens],” he said. “They’re terrible.” 

Walgreens cut ties with Theranos in 2016 after the pharmacy discovered that Theranos was editing patients’ test results, including those tests collected at Walgreens locations. Walgreens ultimately sued Theranos for $140 million—the amount of its initial investment—and recovered less than $30 million in a settlement, The Wall Street Journal has reported. 

In response to inquiries by federal regulators, Theranos voided hundreds of thousands of test results run in 2014 and 2015, including many from samples taken at Walgreens locations, and corrected tens of thousands more because the results didn’t meet the company’s own standards.