Pulse oximeters gained prominence during the Covid-19 pandemic, offering anyone a handheld tool to take stock of their blood oxygen level.
But their newfound popularity also brought new attention to their limitations — several studies have shown that pulse oximeters overestimate blood oxygen levels when used on darker skin tones. This can make patients of color appear more healthy than they really are, resulting in worse health outcomes and exacerbated inequality.
In the years since, researchers have tried to unpack what exactly is going wrong with these commonly used tools. A recent study, published in the Annals of the American Thoracic Society, adds to a growing body of evidence showing the issues with pulse oximeters are more than skin deep. In a retrospective study reviewing five years of data, researchers found that pulse oximeters at Johns Hopkins Hospital inexplicably went from overestimating blood oxygen levels to underestimating them in all patients at higher oxygen levels, though the inaccuracies were worse in patients of color.
“One of the biggest problems with pulse oximeters is that they are this black box. I have no idea why we found what we found in this paper, and why it’s changing year over year,” said Ashraf Fawzy, a pulmonologist and critical care physician at Hopkins who is an author on the paper.
In response, a spokesperson for Masimo, which manufactures the pulse oximeters used in the study, said it is “unaware of any sensor, software, or manufacturing changes during the study period evaluated in the publications that would explain the reported results.” They added that there are multiple factors that can affect a reading, including what type of sensor is used, where it is placed, signal quality, and perfusion.
“The study’s available data do not establish the cause of the reported shift from overestimation to underestimation,” they said. In addition, they pointed to two studies supported by the company that have not found differences in pulse oximeter efficacy based on skin tone.
Pulse oximeters work by shining a light through the skin at two wavelengths, one of which is absorbed by oxygenated blood and the other by deoxygenated blood. The device then compares how much of each is absorbed, and calculates what percent of the blood is oxygenated.
Using data from intensive care units across Hopkins, Fawzy’s study found that the same make of oximeter was not consistent in its readings throughout the years.
The paper was trying to answer a question posed by a Food and Drug Administration-funded study of pulse oximeters, which was published earlier this year. That paper, called EquiOx, was intended to help researchers understand why pulse oximeters overestimated blood oxygen levels in patients with darker skin tones.
Instead, it found that the devices were underestimating blood oxygen levels, a result the authors and outside commentators struggled to justify. It was one of the first studies to proactively take pulse oximeter readings from patients and compare them to an arterial blood gas test, which is the gold standard for blood oxygen levels, but much more invasive.
Taken together, these findings make it difficult for clinicians to know how to interpret pulse oximeter results. If they only overestimated or underestimated oxygen levels in patients with darker skin tones, physicians could adjust accordingly. But if pulse oximeters are inconsistent, it makes it more difficult to work around.
Thomas Valley, a health services researcher and pulmonologist at University of Colorado Anschutz who was involved in a seminal 2020 New England Journal of Medicine paper bringing attention to pulse oximeters’ racial bias, compared pulse oximeters to a broken clock. “If you have a broken clock and you know how it’s going to be five minutes slow every time, you can adjust your routines to take care of that,” he said. But now, it’s unclear whether that metaphorical clock is running fast or slow. “That’s a huge problem, given how important these devices are,” he added.
In the Hopkins study, pulse oximeters gradually shifted from overestimates to underestimates over the course of the five years when compared to arterial blood gas tests. When oximeters overestimate oxygen levels, clinicians run the risk of not providing care to patients when they need it. But underestimating can also come with issues, like providing unnecessary care or keeping patients in the hospital longer than necessary. Overall, “it is not at all clear these things are fit for use in the way that we’re using them,” said Jack Iwashyna, a pulmonologist and critical care physician who is an author on the Hopkins paper.
The recent crop of papers also raises a question about how to best study the shortcomings of pulse oximeters. The landmark 2020 paper that brought renewed attention to the devices’ bias was a retrospective, gleaning a trend from past data. Researchers at the time hoped prospective studies, where scientists actively collect data to get to the root of a problem, would be more helpful in understanding why pulse oximeters fell short in the way they do.
But thus far, such studies have mostly raised more questions than answers. Another prospective study, from Emory University, found evidence of racial bias, but that the inaccurate results were not fully accounted for by variations in skin tone. Readings are likely impacted by factors other than molecules that impact skin pigmentation, such as conditions like anemia, cardiovascular disease, or liver disease, which are more common in patients of color, said Sivasubramanium Bhavani, an ICU physician at Emory who led that study.
“It is more complicated than just skin tone,” he said. “We don’t know all the different causalities and how these things work. But, we do know that if you had the next generation of pulse oximeters, and you can claim that it was 100% calibrated for skin tone, all these other pathways are still going to affect it, and you may still have the same racial bias.”