Can AI predict which heart failure patients will get worse within a year? | MIT News

Characterized by weak or damaged heart muscles, heart failure causes a gradual accumulation of fluid in the patient’s lungs, legs, feet and other parts of the body. This condition is chronic and incurable, often leading to arrhythmia or sudden cardiac arrest. For centuries, bleeding and bloodletting were the treatments of choice, favored by barbers in Europe, during a time when doctors rarely operated on patients.

In the 21st century, the management of heart failure has become less common in middle age: Today, patients receive a combination of healthy lifestyle changes, prescription medications, and sometimes using pacemakers. Yet heart failure remains one of the leading causes of morbidity and mortality, placing a huge burden on health care systems worldwide.

“About half of people diagnosed with heart disease will die within five years of diagnosis,” said Teya Bergamaschi, an MIT PhD student in the lab of Nina T. and Robert H. Rubin Professor Collin Stultz and first author of a new paper introducing a deep learning model for predicting heart failure. “Understanding how a patient will fare after hospitalization is critical to the allocation of limited resources.”

Paper, published in The Lancet in Clinical Medicine by a group of researchers at MIT, Mass General Brigham, and Harvard Medical School, share the results from the development and testing of PULSE-HF, which stands for “Predicting changes in left ventricular Systolic function from ECGs in patients with Heart Failure.” The project is being done in Shultz’s lab, which is affiliated with the MIT Abdul Latif Jameel Clinic for Machine Learning in Health. Developed and retested across three different patient cohorts from Massachusetts General Hospital, Brigham and Women’s Hospital, and MIMIC-IV (publicly available dataset), the deep learning model accurately predicts changes in left ventricular ejection fraction (LVEF), which is the percentage of blood pumped out of the heart’s left ventricle.

A healthy human heart pumps about 50 to 70 percent of the blood from the left ventricle with each beat—anything less is considered a sign of a potential problem. “The model takes the [electrocardiogram] and predict whether or not there will be an ejection fraction within the next year that falls below 40 percent,” said Tiffany Yau, an MIT PhD student in Stultz’s lab who is also first author of the PULSE-HF paper.

If PULSE-HF predicts that a patient’s ejection fraction may worsen within a year, the physician can prioritize the patient for follow-up. Later, low-risk patients can reduce their number of hospital visits and the time spent getting 10 electrodes attached to their bodies with a 12-lead ECG. The model can also be deployed in low-resource clinical settings, including doctors’ offices in rural areas that often do not have a dedicated cardiac sonographer to perform daily ultrasounds.

“It’s a big differentiator [PULSE-HF] from other ECG methods of heart failure rather than detection, it is predictive.” This paper notes that to date, there are no other methods available to predict future LVEF decline among patients with heart disease.

During the testing and validation process, the researchers used a metric known as the “area under the receiver operating characteristic curve” (AUROC) to measure PULSE-HF’s performance. AUROC is often used to measure a model’s ability to discriminate between classes on a scale from 0 to 1, with 0.5 being random and 1 being perfect. PULSE-HF achieved AUROCs ranging from 0.87 to 0.91 across the three patient cohorts.

Notably, the researchers also developed a PULSE-HF version of the ECG lead, which means that only one electrode has to be placed on the body. Although 12-lead ECGs are generally considered superior for being wider and more accurate, the performance of the single-lead version of the PULSE-HF was as strong as the 12-lead version.

Despite the great beauty behind the PULSE-HF concept, like most clinical AI research, it’s opposite execution is troubling. “It took years [to complete this project],” Bergamaschi recalled.

One of the team’s biggest challenges was collecting, processing, and cleaning the ECG and echocardiogram data datasets. While the model aimed to predict the patient discharge fraction, training data labels were not always readily available. Like a student reading from a book with an answer key, labeling is essential to helping machine learning models correctly identify patterns in data.

Clean, linear text in the form of TXT files usually works best when training models. But echocardiogram files usually come in the form of PDFs, and when PDFs are converted to TXT files, the text (separated by line breaks and formatting) becomes difficult for the model to read. The unpredictable nature of real-life situations, such as a restless patient or a loose lead, also corrupted the data. “There are a lot of art symbols that need to be cleaned,” Bergamaschi said. “It’s kind of a never-ending rabbit hole.”

Although Bergamaschi and Yau agree that more sophisticated methods can help filter the data to find better signals, there is a limit to using these methods. “What time do you stop?” Yau asked. “You have to think about the use case – is it really easy to have this model run on slightly corrupted data? Because it probably will be.”

The researchers expect that the next step for PULSE-HF will be to test the model in a prospective study with real patients, whose future ejection fraction is unknown.

Despite the challenges of bringing clinical AI tools like PULSE-HF to the finish line, including the potential risk of extending the PhD by another year, the students felt that the years of hard work were worth it.

“I think things are a little more rewarding because they are challenging,” Bergamaschi said. “A friend said to me, ‘If you think you’re going to find your calling after graduation, if your calling really hits, it’s going to be there in the one extra year it takes you to graduate.’ … How we are measured as researchers [the ML and health] space is different from other researchers in the ML space. Everyone in this community understands the unique challenges that exist here.”

“There’s a lot of suffering in the world,” said Yau, who joined the Shultz lab after a health event that made him realize the value of machine learning in health care. Anything that tries to alleviate suffering is something I would consider a valuable use of my time.