Clinical trials for central nervous system (CNS) disorders face several challenges, notably due to subjective diagnosis systems leading to heterogeneity in the definition of patient populations and the clinical measures used in trials, especially for psychiatric disorders. In this context, digital health technologies (DHTs), in combination with clinical observation and existing subjective tools, could enhance CNS drug research and development¹. DHTs allow remote measurement of patient-generated data, such as specific behaviours, aspects of cognition, physiological parameters and motor skills, by means of sensor-based devices, mobile applications or wearables. Such technologies could enable drugs to be tested for their impact on new digital phenotypes or symptom clusters, enrich patient populations and reduce heterogeneity, allow continuous and objective measurements of highly variable outcomes, and facilitate collection of real-world evidence.

Given this potential, DHTs are coming to regulatory attention for potential uses in clinical trials for CNS disorders, including selection of patients and the development of composite end points in which DHTs are combined with clinical judgement; for example, face-recognition technology combined with a clinical scale to assess treatment effect on mood and/or social interactions. The use of DHTs in clinical trials poses regulatory challenges that require multiple types of expertise to address, and despite the availability of multiple regulatory pathways, there is not a structured and unified process for development of DHTs as there is for medicinal products. It is therefore necessary to reach a consensus on the quality and quantity of evidence needed and on the methods to use to collect that evidence.

In this article, we discuss some general principles applicable to DHT development. We also highlight some of the regulatory challenges of the implementation of DHTs in general and in CNS drug development in particular, using the regulatory environment in the European Union (EU) as the basis for the discussion.

Principles and challenges for DHT development

Although some of the steps to develop a DHT for use in drug development are similar to those needed for a traditional measure, DHTs have unique characteristics that combine software and hardware development as well as clinical development. DHT development comprises at least three phases: verification, analytical validation and clinical validation², each of which poses different challenges.

The verification phase pertains to the technical evaluation of the DHT to ascertain accuracy, precision and reliability. DHTs comprise a broad category of products, some of which meet the definition of a medical device and some of which do not. In the EU, all medical devices are assessed for safety and performance, and, depending on risk, require the involvement of a notified body designated by individual member states. If the medical device is found to be in conformity with the applicable requirements, a CE mark is affixed that authorizes the device to be marketed in the EU.

The analytical validation phase addresses whether the DHT measures the clinical event in the target population. Data are collected in a naturalistic setting and the algorithms are tested against a human rater or a known validated traditional measure to ascertain the face-validity of the DHT in measuring the construct (or constructs) of interest. Suitability for a specific patient population is typically explored in usability and feasibility studies; however, a consensus has not yet been reached on how such studies should be designed or conducted. A recent review of feasibility studies highlighted a substantial lack of standardization in reporting methodology, a lack of detailed population information and several limitations in information sharing in public databases³.

During the clinical validation phase, the new tool is used in a prospective clinical development setting to provide information on its performance in a given context of use; for example, as an end point, a biomarker or an enrichment tool. There is no standard framework for clinical validation, as it depends on the clinical question that the use of the DHT intends to answer.

A DHT typically monitors a patient remotely and continuously and, for this reason, it is likely to produce a large volume of data. In contrast to traditional data, such data can be composed of both raw data and algorithm-processed data, which poses additional statistical and clinical questions. DHTs have the potential to minimize the amount of missing data by allowing measurements in the patient’s environment, aided by specific training sessions for patients and caregivers. Nevertheless, a strategy for handling missing data is particularly important. Furthermore, the safety of a DHT should be demonstrated to avoid potential (bio)hazards, including inappropriate use or erroneous measurements.

Importantly, DHT components may need to be updated or optimized during the phases of development. For substantial changes, additional data may be required for validation, including modifications needed to adapt the DHT to a specific subpopulation. So, transparency is required in the use of a DHT in drug development, and if changes such as modification of algorithms for data processing occur during a study, they should be discussed with regulators.

Close and early interaction with regulators is needed, such as the Innovation Task Force Meetings at the EMA. Developers may also enter the EMA qualification process, leading to a Qualification Opinion or a Qualification Advice. The EMA would qualify the clinical measures but not the tool used to produce and record the measures.

Examples of DHT development in the CNS field

In the CNS field, the EMA has recently qualified stride velocity, measured by an ankle wearable device, as a secondary end point in Duchenne muscular dystrophy (DMD)⁴. During the analytical validation phase, the gait measures were tested by demonstrating that the distance measured by reconstruction of the ankle trajectory of ambulant patients as assessed by the magneto-inertial sensor corresponded to the actual distance when measured manually. The measure was then clinically validated against the 6-minute walk test, which is the current gold standard measure in DMD drug trials.

Although DHTs aim to overcome the limitations of established end points in CNS trials, anchoring the produced measures to existing gold standards is considered a necessary step for the clinical validation of a new DHT. In some cases, however, the DHT may measure completely new construct(s), which could be a domain or a symptom cluster within a disorder that was not measurable before or for which no gold standard existed. This is particularly relevant in CNS disorders when moving from subjective to objective measurements of behavioural parameters such as socialization or psychomotor activity. The development of such novel end points should start from a consensus among experts, clinicians, regulators and patients on the clinical meaningfulness of the concept of interest. The capability of the DHT to measure a novel end point should then be justified by significant clinical benchmarks in the context of the disease of interest (functional outcomes or clinical scales) and eventually in its capability to capture disease changes as a result of drug treatment. Several public–private initiatives such as the Innovative Medicines Initiative are fostering the development of DHTs as novel end points in the pre-competitive space and aim to coordinate the efforts of different stakeholders including small and medium-sized enterprises. These research efforts are focused on validating quantifiable biological parameters — including digital biomarkers — for symptomatic dimensions across different neuropsychiatric disorders⁵ and assessing digital biomarkers as outcomes in randomized clinical trials in disorders such as autism.

Nevertheless, some of the programs for DHT development in CNS disorders have failed to deliver actionable end points or biomarkers. To reduce the risk of such failures, it is particularly important to take into account the role of the DHT and also the construct it measures in the clinical context of the population under study or in relation to the expected treatment effect.

Conclusion

New DHT tools can help move the field of CNS drug development forward, but their disruptive potential can only be realized through coordinated and early interactions between all the stakeholders. This can be facilitated by the availability of multiple regulatory pathways in Europe, and funding bodies and consortia working in the DHT field are strongly encouraged to seek regulatory and scientific advice from the EMA.