CMS Announces Winner of Artificial Intelligence Challenge

CMS has announced ClosedLoop.ai as the winner and Geisinger as the runner-up in its Artificial Intelligence Health Outcomes Challenge, a multi-stage competition that began in 2019.

The AI Health Outcomes Challenge was operated by CMS’s Center for Medicare and Medicaid Innovation (CMS Innovation Center) in partnership with the American Academy of Family Physicians (AAFP) and Arnold Ventures.

The competition aimed to accelerate the development of AI solutions for predicting health outcomes for Medicare beneficiaries for potential use by the Innovation Center.

The competition has substantially increased the Innovation Center’s knowledge of AI and its potential applications, leading to better design and implementation of future models as part of its mission to test innovative payment and service delivery models.

“Clinicians are eager to use the latest innovations to better help identify patients at risk, provide higher quality care, and improve health outcomes. The use of artificial intelligence has the potential to achieve these aims by providing important information to clinicians that may be helpful in providing higher quality care,” said CMS Acting Administrator Liz Richter.

“CMS’s AI Health Outcomes Challenge has brought this exciting future one step closer to reality by engaging with some of the country’s brightest AI innovators. We congratulate ClosedLoop.ai and Geisinger on their achievements, and all the Challenge finalists and participants for helping increase the information available to clinicians to improve patient care.”

ClosedLoop.ai was chosen as the grand prize winner out of more than 300 initial entries and will receive up to $1 million in prize money. The company is based in Austin, Texas, and is a leader in developing healthcare data science solutions. As the runner-up, Geisinger will receive up to $230,000 in prize money.

The CMS AI Health Outcomes Challenge attracted investors from all sectors to leverage AI solutions to predict outcomes for Medicare beneficiaries. From the initial group of more than 300 entries, the Challenge progressed through several stages and CMS narrowed down participants to the top 25 and then seven finalists.

CMS chose ClosedLoop.ai and Geisinger among the seven finalists chosen in October 2020 to participate in the final stage of the competition. In this last stage, participants further refined the solutions they developed in the prior stage to help predict unplanned hospital and skilled nursing facility admissions and adverse events.

Participants also developed predictive algorithms to identify beneficiaries at risk of mortality in 12 months. Finalists worked to address sources of bias in their solutions that could have the potential to affect health disparities in their submissions. Finalists also demonstrated how their solutions would easily explain the predictions to clinicians.

“Avoidable hospitalizations and skilled nursing facility stays are bad for patients and make our healthcare system costlier and less sustainable for everyone,” said Mark Miller, executive vice president of health care at Arnold Ventures. “We are eager to see how the winners of the competition use new data approaches to identify solutions to improve care in Medicare." 

Each finalist in the AI Health Outcomes Challenge exhibited superior performance on at least one required element of the competition. ClosedLoop.ai and Geisinger demonstrated a consistently strong performance across all competition elements while generating the best prediction accuracy results.

To select the winner and runner-up, CMS conducted a rigorous evaluation process supported by a team of AI scientists. Clinicians from AAFP reviewed and scored the explainability element. A panel of CMS senior leadership reviewed the submissions and selected the winners.

Of the $1.65 million in total prizes to participants, Arnold Ventures will contribute up to $300,000 and the AAFP is contributing up to $340,000.

“We are excited about the early successes and great potential of Artificial Intelligence to dramatically improve health outcomes, reduce administrative burden, and create smarter health IT,” said American Academy of Family Physicians EVP/CEO Shawn Martin.

“We were pleased to partner with the CMS Innovation Center on the AI Health Outcomes Challenge and to bring family physicians into the evaluation of the contestants to help identify the winners. We look forward to seeing the winning, and all of the great, solutions in the market.”