For some languages, it can give patients misleading information
By Nicole Wetsman, theverge.com
Google Translate still isn’t reliable enough to use for medical instructions for people who don’t speak English, according to a new study published last week. Sometimes, it works: it was the most accurate when translating emergency department discharge instructions into Spanish. But a lot of the time, especially with less common languages, it doesn’t — the study found it was only 55 percent accurate for Armenian. That’s a big problem when it comes to health information, where any misunderstanding can be dangerous.
“All you need is one error that creates confusion for a patient, and they don’t take their blood thinner or they take too much of their blood thinner,” says study author Lisa Diamond, a health disparities researcher at Memorial Sloan Kettering Cancer Center in New York. “And you end up with medical emergency.”
Federal guidelines say that hospitals and health care organizations have to provide interpreters and translators for patients who don’t speak English. The guidelines are designed to fill a vital need — these patients are at a higher risk of medical complications because they may not understand instructions given by their doctors.
However, in practice, many hospitals don’t offer interpreters to every patient who needs one — they’re expensive, and many health care groups struggle with the cost. Even if a hospital does have interpreters on staff or a subscription to a phone interpreting service for verbal communication, they’re less likely to have a way to translate written instructions. “There’s a clear gap in the ability to provide written information for patients,” says study author Breena Taira, an associate professor of clinical emergency medicine at UCLA Health.
It’s become common for doctors to resort to Google Translate in medical settings, Taira says. “You can imagine that a well-meaning emergency department provider thinking, ‘I really want to provide my patient with instructions in their own language, and my hospital doesn’t have a mechanism to do this — why don’t I use this automated translation software,” she says.
The new study evaluated 400 emergency department discharge instructions translated by Google Translate into seven different languages: Spanish, Chinese, Vietnamese, Tagalog, Korean, Armenian, and Farsi. Native speakers read the translations and evaluated their accuracy. Overall, the translated instructions were over 80 percent accurate.
That’s an improvement from 2014, when an analysis found that Google Translate was less than 60 percent accurate for medical information. Google Translate improved in 2016, when it started using a new algorithm — since then, one 2019 study found that it can be over 90 percent accurate in Spanish.
But the new analysis also found that accuracy varied between languages. Like the 2019 study, it found that Google Translate was over 90 percent accurate for Spanish. Tagalog, Korean, and Chinese had accuracy rates ranging from 80 to 90 percent. There was a big drop-off for Farsi, which had a 67 percent accuracy, and Armenian, which had a 55 percent accuracy. In one example, Google Translate turned “You can take over the counter ibuprofen as needed for pain” into Armenian as “You may take anti-tank missile as much as you need for pain.”
Even languages like Spanish and Chinese that were usually accurate could have Google Translate errors that could confuse patients. An instruction for a patient taking the blood-thinning medication Coumadin read “Your Coumadin level was too high today. Do not take any more Coumadin until your doctor reviews the results.” It was translated into Chinese as “Your soybean level was too high today. Do not take anymore soybean until your doctor reviews the results.”
One of the main problems with relying on machine translation is that it can’t account for context, Diamond says. The program might not recognize that a word is the name of a medication, for example. “It loses the meaning of what you’re trying to say,” she says.
Eventually, machine translation programs might improve to the point where they can accurately and safely translate medical information. But based on the way they work now, they aren’t a good approach.
Instead, doctors should write out instructions in English and have an interpreter go over those instructions verbally with a patient, Taira says. But that’s just a stop-gap — ideally, health systems should give doctors a way to get professional translations of materials. Each doctor is going to do the best they can with the resources they have available. “What we need to do, really as a system, is to make things easier for the provider,” Taira says.