Why do we need to evaluate
digital patient experience?
For future
Emerging digital technologies promise to shape the future healthcare industry. Despite the growing number of DHIs, timely, cost-effective, and robust evaluations have not matched the growth in numbers.
As a key point
PEx is a key intent of patient-centered care and a core measure of care quality in digital health. The health system is not responding adequately to the need for improved PEx.
influence experience
Low-quality digital health may disrupt the user experience, resulting in low acceptance, and some may even be harmful.
part 2
Define Evaluation
Evaluation is an essential part of the design process; it ensures designers can efficiently create high-quality design outputs that align with initial objectives and iteratively improve design solutions. To effectively improve the digital patient experience, knowing when to measure, what to measure, and how to measure it is as important as knowing what factors influence it and how to design it. In this section, we provide you with an evaluation guide to help you generate an appropriate evaluation plan to assess whether your design solution aligns with your design objectives. After this section, you are expected to know how to clarify your own evaluation objectives, identify stakeholder groups, consider reasonable evaluation timing, choose relevant evaluation indicators, and select appropriate evaluation approaches.
EVALUATION GUIDELINES
How to evaluate the digital patient experience?
STEP 1 | Identify evaluation objectives (target audience)
STEP 2 | Determine the maturity stage of intervention
STEP 3 | Select the appropriate influencing factors or evaluation indicators
STEP 4 | Consider the timing of evaluation and data collection
STEP 5 | Determine study design, data collection and analysis methods
STEP 6 | Report your evaluation results, theoretical and practical implications
STEP 3: EVALUATION INDICATORS
What indicators can measure the digital patient experience?
All
Intervention Outputs
Patient Outcomes
Healthcare System Impact
STEP 4: tIMING Considerations
When to measure the digital patient experience?
Intervention Maturity Stages
Effectiveness
Assess whether the DHI achieves the intended results in non-research or uncontrolled setting.
Implementation
Assess the uptake, institutionalization and sustainability of evidence based DHIs in a given context, including policies and practices.
Timing of the Evaluation
Before Intervention
A pre-test is performed before individuals adopt or implement the intervention. It assesses individuals’ initial status and their anticipated perception of the intervention.
During Intervention
An evaluation performed during the use of intervention is to monitor individuals' real-time feedback and reactions.
After Intervention
The post-test is performed right after or a long time after the completion of the interventions by individuals. It assesses individuals’ changes regarding of using the intervention.
Timing of Data Collection
Immediate Evaluation
Aims to collect “real-time” data on patients’ experiences during or immediately after their treatment.
Delayed Intervention
Aims to obtain more substantial responses after the intervention's completion over a long period of time.
Momentary Evaluation
Aims to collect transient information from individuals at a specific moment.
Continuous Evaluation
Aims to gather feedback from individuals at different points along the care pathway.
STEP 5: EVALUATION APPROACHES
What measurement approach can be used?
Study Design
Descriptive Study
Aims to define the "who, what, when, and where" of observed phenomena and include qualitative research concerning both individuals and populations.
Analytical Study
Aims to quantify the relationship between the intervention and the outcomes of interest, usually with the specific aim of demonstrating a causative link between the two, including experimental and observational studies.
Data Collection Methods
Qualitative Methods
Qualitative research is expressed in words. It is used to understand concepts, thoughts, or experiences. Common qualitative methods include interviews with open-ended questions, observations described in words, and literature reviews that explore concepts and theories.
Quantitative Methods
Quantitative research is expressed in numbers and graphs. It is used to test or confirm theories and assumptions. Common quantitative methods include experiments, observations recorded as numbers, and surveys with closed-ended questions.
Data Analysis Approaches
Qualitative Analysis
Qualitative data consists of text, images, or videos instead of numbers. Content analysis, thematic analysis, and discourse analysis are common approaches used to analyze this type of data.
Quantitative Analysis
Quantitative data is based on numbers. Simple math or more advanced statistical analysis is used to discover commonalities or patterns in the data.
TEMPLATE
Create your digital health evaluation plan
