Patients who use mobile applications to manage medical conditions including depression and chronic pain might assume the apps have been evaluated by regulatory agencies to be safe and effective. But that isn’t necessarily the case.
Most of the more than 55,000 medical apps that claim to diagnose or treat a condition—or ones that provide clinical decision support, known as “therapeutic” apps—have never been assessed by any trusted neutral bodies or regulatory agencies to evaluate them for technical soundness, ethical design, or clinical benefit. The apps often don’t comply with regional data security and privacy laws to protect people’s sensitive health information.
Medical apps differ from traditional wellness apps, which provide users with insights into becoming healthier by, for example, tracking fitness activities, monitoring blood pressure, and analyzing sleep patterns.
There is no reliable way to verify that therapeutic apps deliver the results they indicate. To help ensure such apps are credible, the IEEE Standards Association (IEEE SA) recently launched the IEEE Global Medical Mobile App Assessment and Registry. The publicly searchable directory is designed to list apps that have been vetted by experts across several criteria including technical soundness, ethical design, compliance with data security and privacy regulations, and clinical efficacy, which is evidence of a clinical benefit for the patient.
“Patients, clinicians, payers, and health care systems often struggle to distinguish clinically meaningful therapeutic apps from those that are simply well-marketed,” says IEEE Senior Member Yuri Quintana, chair of the assessment and registry program. He is chief of the clinical informatics division at Beth Israel Deaconess Medical Center, in Boston. “Our goal is to establish a standardized review method using criteria developed by experts.”
Why regulation is lacking
Because the apps are intended for medical use without being part of a medical implement, they fall under the designation of software as a medical device (SaMD), according to the International Medical Device Regulators Forum. SaMD is supposed to be regulated by public health agencies such as the U.S. Food and Drug Administration, but the apps have developed and grown in popularity so quickly that regulators haven’t been able to keep up, Quintana says. Some companies have received approval, but most have not, he says.
Many users are unaware of the regulatory gap, he says.
“Seeing an app from a well-known company often creates the impression that it has been meaningfully vetted for safety and efficacy, even when that is not the case,” he says.
Some companies are using deceptive advertising to sell their product, he adds. Marketing materials might claim that all of a company’s health apps are certified, even though only one app has been approved by a regulatory body to treat a particular condition. Or the verbiage might imply the company has clinical evidence proving its application works, even though the app has never been tested independently.
Another concern is that updated apps aren’t being vetted, says Maria Palombini, IEEE SA’s director of health care and life sciences global practice lead.
“The original app might have received approval from a regulatory agency, but not the updated version,” Palombini says. “There could have been significant changes from the original.”
“Not every medical-related app triggers the same regulatory classification or review across jurisdictions,” Quintana adds. “That leaves a large gray zone of clinically relevant but lower-risk apps that haven’t undergone an independent assessment. The IEEE registry was created to help fill these gaps.
“IEEE is the best organization to address this problem because this is fundamentally a standards, trust, interoperability, and conformity assessment challenge,” he says. IEEE “is the world’s largest technical professional organization, with deep expertise in developing globally recognized standards including in health care, cybersecurity, AI ethics, and interoperability.”
“Through the IEEE Conformity Assessment Program, we already run rigorous assessment and registry programs,” Palombini says. “Our neutral, consensus-driven, multidisciplinary approach—bringing together clinicians, regulators, developers, and ethicists without commercial bias—makes IEEE uniquely positioned to create trustworthy global guardrails that can scale across jurisdictions and support regulatory harmonization.”
How the registry works
The assessment framework was developed by a multidisciplinary group of 35 volunteer experts from 10 countries, Quintana says. The panel includes academics, AI experts, app developers, clinicians, ethicists, mental health experts, patient advocates, regulators, researchers, technologists, and those who assess safety in health care.
The registry is for any app used for clinical care or therapeutics that claims to demonstrate a medical benefit. That includes apps designed for cardiology, diabetes, mental health, neurology, oncology, rehabilitation, and respiratory diseases, Quintana says.
Initially, he says, the focus will be on apps that aim to treat mental health conditions, given the large number of offerings in that area and the registry committee’s expertise.
The submission of apps is voluntary. There is no government mandate that requires a company to use the IEEE registry.
The products will be evaluated against about 150 consensus-based criteria across three major areas:
- Clinical efficacy including therapeutic effectiveness, any sustained benefits, risk management, comparison to standard care, user engagement, and real clinical value.
- Technical soundness including accessibility, privacy and security, error handling, interoperability, AI governance, usability, and operational quality.
- Ethical design including bias prevention, patient consent, data governance, conflict-of-interest transparency, responsible use of AI and large language models, and prioritization of public health benefits.
IEEE charges a nonrefundable submission fee that covers the cost of the assessment plus the registry’s annual subscription for the first year.
Developers first must demonstrate they are a legally established entity before they can complete the app publisher registration form and then submit documentation and attestations about the product.
The IEEE review of an app is estimated to take six to eight weeks, Palombini says. The assessment results will be privately shared with the app publisher, she says, and to be listed in the registry, an app must achieve more than 85 percent compliance in each category.
Upgraded apps must be submitted and reassessed, Palombini says. Similar to how users are notified when an app on their smart devices has , the registry will be notified when listed apps have a new update available, she says.
Applicants who do not pass the assessment are to receive feedback explaining why. They will be given an opportunity to make changes or provide additional documentation, Palombini says.
“It’s a pretty methodological process, with checks and balances,” Quintana says. “We’re being very transparent about the process.”
Approved apps added to the registry receive an IEEE certification badge and submission identifier, which the company can display on its website, app store listings, and marketing materials.
“The badge serves as visible proof that the app has met the independent, consensus-based assessment for clinical value, technical robustness, and ethical design,” Quintana says.
The registry will be publicly available at no cost, he says.
Patients and families seeking safe, trustworthy apps—and payers and insurers evaluating reimbursement potential—will find the registry helpful, he says.
The application website is open. The public registry page does not yet list a specific count of approved apps because assessments are ongoing. Approved apps and their unique identifiers are to be published when the initial reviews are completed.
To learn more, you can watch a webinar recorded in March.
The assessment framework that underpins the registry is supporting the formal recognition of IEEE P3962 Standard for Criteria Assessment Framework f
