Researchers Propose New Digital Health Assessment Framework

A group of researchers has proposed a new framework to assess evidence for digital health interventions.

Known as Evidence in Digital health for EFfectiveness of INterventions with Evaluative Depth (Evidence DEFINED), the framework aims to enable standardized and rapid assessments of digital health intervention evidence in organizations and guide digital health solutions providers in generating evidence that drives the adoption of these interventions.

The framework was developed by researchers from numerous organizations, including the Digital Medicine Society, Elevance Health, AstraZeneca, and Case Western Reserve University, and published in njp digital medicine.  

"When assessing clinical evidence in digital health, details matter. Careful evidence assessment can mean the difference between identifying critical evidence flaws and failing to do so. This can, in turn, impact countless patients, by dictating whether patients get access to digital health interventions that are effective and safe," the researchers wrote.

After examining 78 existing assessment frameworks for digital health intervention evidence, the researchers determined that none met the needs of specific stakeholder organizations, such as health systems, payers, pharmacy benefit managers, and pharmaceutical companies.

Thus, the researchers set out to develop a new framework, Evidence DEFINED, which includes four steps:

1. Screen for failure to meet absolute requirements, for example, compliance with data privacy standards.

2. Apply an established evidence assessment methodology that was developed for non-digital interventions

3. Apply the Evidence DEFINED supplementary checklist.

4. Use evidence-to-recommendation guidelines to suggest adoption levels that may be appropriate for the digital health intervention.

The Evidence DEFINED framework also aims to address gaps in previous frameworks, including not being adapted to address evidence considerations unique to digital health and not leveraging existing methodologies developed for assessing non-digital interventions.

The Evidence DEFINED framework's supplementary checklist includes evidence quality criteria unique to digital health, and the framework uses "established evidence assessment methodologies wherever possible," the researchers noted.

In addition, the framework focuses on optimizing the screening process and deprioritizing steps that may have limited value, thereby increasing the speed of assessments. 

According to the researchers, the framework is designed to be "agile and flexible." The Digital Medicine Society will coordinate efforts to update the framework. The organization will create a public website and collaborate with interested partners to gather suggestions for updates and distribute the framework. Updates will be implemented every six to 12 months.

As digital health solutions proliferate, numerous organizations have proposed frameworks to guide the development and assessment of these tools.

Last May, the American Telemedicine Association, American College of Physicians, and the Organization for the Review of Care and Health Applications released the Digital Health Assessment Framework. The framework aims to support the adoption of high-quality digital health technologies and help clinicians and patients make informed decisions. It includes four components: data and privacy; clinical assurance and safety; usability and accessibility; and technical security and stability.

Industry trade organizations have also proposed guidance for implementing and evaluating telehealth services.

For instance, in April, the American College of Physicians published a paper in the Annals of Internal Medicine describing steps for applying, integrating, and evaluating telehealth performance in ambulatory care environments. Their recommendations included ensuring that telehealth performance is measured based on the same criteria as those used for in-person care evaluations and conducting performance testing for quality assessment measures that are deemed appropriate.

The organization also recommended making mechanisms available to allow physicians to access information generated through telehealth before performance evaluation and reviewing existing performance measures to ensure they are appropriate for a telehealth setting.