Dataset Information

The Impact of Intervention Design on User Engagement in Digital Therapeutics Research: Factorial Experiment With a Mixed Methods Study.

ABSTRACT:

Background

User engagement is crucial for digital therapeutics (DTx) effectiveness; due to variations in the conceptualization of engagement and intervention design, assessment and retention of engagement remain challenging.

Objective

We investigated the influence of the perceived acceptability of experimental intervention components and satisfaction with core intervention components in DTx on user engagement, while also identifying potential barriers and facilitators to user engagement.

Methods

We conducted a mixed methods study with a 2 × 2 factorial design, involving 12 outpatients with atopic dermatitis. Participants were randomized into 4 experimental groups based on push notification ("basic" or "advanced") and human coach ("on" or "off") experimental intervention components. All participants engaged in self-monitoring and learning courses as core intervention components within an app-based intervention over 8 weeks. Data were collected through in-app behavioral data, physician- and self-reported questionnaires, and semistructured interviews assessed at baseline, 4 weeks, and 8 weeks. Descriptive statistics and thematic analysis were used to evaluate user engagement, perceived acceptability of experimental intervention components (ie, push notification and human coach), satisfaction with core intervention components (ie, self-monitoring and learning courses), and intervention effectiveness through clinical outcomes.

Results

The primary outcome indicated that group 4, provided with "advanced-level push notifications" and a "human coach," showed higher completion rates for self-monitoring forms and learning courses compared to the predetermined threshold of clinical significance. Qualitative data analysis revealed three key themes: (1) perceived acceptability of the experimental intervention components, (2) satisfaction with the core intervention components, and (3) suggestions for improvement in the overall intervention program. Regarding clinical outcomes, the Perceived Stress Scale and Dermatology Life Quality Index scores presented the highest improvement in group 4.

Conclusions

These findings will help refine the intervention and inform the design of a subsequent randomized trial to test its effectiveness. Furthermore, this design may serve as a model for broadly examining and optimizing overall engagement in DTx and for future investigation into the complex relationship between engagement and clinical outcomes.

Trial registration

Clinical Research Information Service KCT0007675; http://tinyurl.com/2m8rjrmv.

SUBMITTER: Lee H

PROVIDER: S-EPMC10891489 | biostudies-literature | 2024 Feb

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

The Impact of Intervention Design on User Engagement in Digital Therapeutics Research: Factorial Experiment With a Mixed Methods Study.

Lee Hyerim H Choi Eung Ho EH Shin Jung U JU Kim Tae-Gyun TG Oh Jooyoung J Shin Bokyoung B Sim Jung Yeon JY Shin Jaeyong J Kim Meelim M

JMIR formative research 20240209

<h4>Background</h4>User engagement is crucial for digital therapeutics (DTx) effectiveness; due to variations in the conceptualization of engagement and intervention design, assessment and retention of engagement remain challenging.<h4>Objective</h4>We investigated the influence of the perceived acceptability of experimental intervention components and satisfaction with core intervention components in DTx on user engagement, while also identifying potential barriers and facilitators to user enga ...[more]

PMID: 38335015

Similar Datasets

Project description:BackgroundDigital health interventions have become increasingly popular in recent years, expanding the possibilities for treatment for various patient groups. In clinical research, while the design of the intervention receives close attention, challenges with research participant engagement and retention persist. This may be partially due to the use of digital health platforms, which may lack adequacy for participants.ObjectiveThis systematic literature review aims to investigate the relationship between digital health platforms and participant engagement and retention in clinical research. It will map and analyze key definitions of engagement and retention, as well as identify design characteristics that influence them.MethodsWe will carry out a mixed methods systematic literature review, analyzing qualitative and quantitative studies. The search strategy includes the electronic databases PubMed, IEEE Xplore, CINAHL, Scopus, Web of Science, APA PsycINFO, and the ACM Digital Library. The review will encompass studies published between January 2018 and June 2024. Criteria for inclusion will be the presence of digital health care interventions conducted through digital health platforms like websites, web and mobile apps used by patients, and informal caregivers as research participants. The main outcome will be a narrative analysis with key findings on the definitions of participant engagement and retention and critical factors that affect them. Quality assessment and appraisal will be done through the Mixed-Methods Assessment Tool. Data analysis and synthesis will follow the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) 2020 flow diagram. Quantitative data will be qualified and integrated into qualitative data, which will be analyzed using thematic analysis and synthesis.ResultsThe study expects to map and summarize critical definitions of participant engagement and retention, and the characteristics of digital health platforms that influence them. The systematic review is expected to be completed in June 2025.ConclusionsThis systematic review will contribute to the growing discussion on how the design of digital health intervention platforms can promote participant engagement and retention in clinical research.Trial registrationPROSPERO CRD42024561650; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=561650.International registered report identifier (irrid)PRR1-10.2196/65099.

Project description:BackgroundHealth care systems are rapidly deploying digital tools for disease management; however, few studies have evaluated their usability by vulnerable populations. To understand the barriers to app usage among vulnerable populations, we employed user-centered design (UCD) methods in the development of a new text messaging app.ObjectiveThe study aimed to describe variations in patients' engagement in the app design process, focusing on limited health literacy (LHL), limited English proficiency (LEP), and limited digital literacy (LDL).MethodsWe conducted 20 in-depth semistructured interviews with primary care patients at a public health care system, used open-ended discussions and card sorting tasks to seek input about mobile phones and text messaging, and used open coding to categorize the patterns of mobile phone usage and to evaluate engagement in the card sorting process. We examined qualitative differences in engagement by examining the extensiveness of participant feedback on existing and novel text messaging content and calculated the proportion of patients providing extensive feedback on existing and novel content, overall and by health literacy, English proficiency, and digital literacy.ResultsThe average age of the 20 participants was 59 (SD 8) years; 13 (65%) were female, 18 (90%) were nonwhite, 16 (80%) had LHL, and 13 (65%) had LEP. All had depression, and 14 (70%) had diabetes. Most participants had smartphones (18/20, 90%) and regularly used text messaging (15/20, 75%), but 14 (70%) of them reported having difficulty texting because of inability to type, physical disability, and low literacy. We identified 10 participants as specifically having LDL; 7 of these participants had LEP, and all 10 had LHL. Half of the participants required a modification of the card sorting activity owing to not understanding it or not being able to read the cards in the allotted time. The proportion of participants who gave extensive feedback on existing content was lower in participants with limited versus adequate English proficiency (4/13, 30% vs 5/7, 71%), limited versus adequate health literacy (7/16, 44% vs 3/4, 75%), and limited versus adequate digital literacy (4/10, 40% vs 6/10, 60%); none of these differences were statistically significant. When examining the proportion of patients who gave extensive feedback for novel messaging content, those with LHL were less engaged than those with adequate health literacy (8/16, 50% vs 4/4, 100%); there were no statistical differences by any subgroup.ConclusionsDespite widespread mobile phone use, digital literacy barriers are common among vulnerable populations. Engagement in the card sorting activity varied among participants and appeared to be lower among those with LHL, LEP, and LDL. Researchers employing traditional UCD methods should routinely measure these communication domains among their end-user samples. Future work is needed to replicate our findings in larger samples, but augmentation of card sorting with direct observation and audiovisual cues may be more productive in eliciting feedback for those with communication barriers.

Project description:BackgroundEarly excess and inadequate gestational weight gain (GWG) have been associated with negative outcomes for mother and child. The use of digital media to deliver pregnancy lifestyle interventions is increasing, but there is little data on participant engagement. The Pregnancy Lifestyle Activity and Nutrition (PLAN) intervention pilot study was an electronic health and dietetic-delivered intervention program promoting healthy GWG in early pregnancy.ObjectiveThis study aims to explore the interactions of participants with the program and to assess its acceptability.MethodsThis study uses both quantitative and qualitative methods using data from parent randomized controlled trial (ACTRN12617000725369). Quantitative data from 22 participants in the intervention arm who completed the study provided measures of the interactions participants had with the digital components of the program and with dietetic consultations. A descriptive qualitative analysis employed semistructured interviews with 9 participants to elicit views on the acceptability of the intervention and its components.ResultsThe electronic delivery of information and recording of weight from 8 to 20 weeks of gestation were universally accepted. Component (face-to-face dietitian, weight tracker, website information delivery, and SMS goal prompting) acceptability and engagement differed between individuals. A total of 4 key themes emerged from the qualitative analysis: supporting lifestyle change, component acceptability and value, delivery platforms, and engagement barriers.ConclusionsThe PLAN intervention and its delivery via a blend of personal dietetic consultations and digital program delivery was found to be acceptable and valuable to pregnant women. Individuals responded differently to various components, emphasizing the importance of including women in the development of lifestyle interventions and allowing participants to choose and tailor programs. Larger randomized controlled trials using these insights in a broader section of the community are needed to inform the iterative development of practical, time-efficient, and cost-effective ways of supporting optimal GWG with the potential to optimize outcomes for pregnant women and their child.

Project description:BackgroundEngagement with digital health interventions (DHIs) may be regarded as a prerequisite for the intervention to achieve positive health or behavior change outcomes. One method employed to promote engagement is the use of prompts such as emails and text messages. However, little is known about the characteristics of prompts that promote engagement. This study explored the association between the content and delivery mode of prompts and the users' engagement with HeLP-Diabetes (Healthy Living for People with type 2 Diabetes), a DHI that aimed to promote self-management in adults with type 2 diabetes.ObjectiveThe objective of this study was to identify the characteristics of prompts, specifically the content and delivery mode, which were associated with increased engagement.MethodsThis was a mixed-methods study. Email and text message prompts were sent to the registered users of HeLP-Diabetes. Use of the intervention was recorded and examined to identify which email and text message prompts were associated with subsequent visits to the DHI. Characteristics of prompts that were identified as particularly effective or ineffective were explored through think-aloud interviews with the participants.ResultsOf a total of 39 email prompts, 49% (19/39) prompts showed a significant association with subsequent visits to the DHI. However, none of the text message prompts were associated with subsequent visits to the DHI. Furthermore, think-aloud interviews were carried out with 6 experienced participants with type 2 diabetes. The findings suggest that these participants preferred email prompts that were clear, relatively short, and empowering; used nondirective advice; included health professional references; were visually appealing; and contained news and updates.ConclusionsThe findings of this study contribute to the existing evidence supporting the role of email prompts in promoting and maintaining engagement with DHIs. This study described the content of prompts that may be engaging. However, the results should be interpreted with caution, as prompts may be context-specific interventions and the results may not be generalizable across other DHIs or other types of interventions targeting self-management of type 2 diabetes.

Project description:BackgroundIn patient-oriented research (POR), patients contribute their valuable knowledge and lived-experiences to work together as active research partners at all stages of the health research cycle. However, research looking to understand how patient research partners (PRPs) and researchers work together in meaningful and collaborative ways remains limited. This study aims to evaluate patient engagement with the RePORT Patient Advisory Council (PAC) and to identify barriers and facilitators to meaningful patient engagement encountered within research partnerships involving patient research partners and researchers.MethodsThe RePORT PAC members included nine PRPs and nine researchers (clinician-researchers, research staff, patient engagement experts) from both Alberta and British Columbia. All members were contacted and invited to complete an anonymous online survey (Public and Patient Engagement Evaluation (PPEET) tool) at two different project times points. The PAC was invited for a semi-structured interview to gain in-depth understanding of their experiences working together. Interviews were audio-recorded, transcribed, and the data was thematically analyzed with the support of a qualitative analysis software, NVivo.ResultsA total of nine PRPs (100%) and three researchers (33%) participated in the baseline survey in February 2022 while six PRPs (67%) responded and three researchers (33%) completed the follow up survey in May 2022. For the semi-structured interviews, nine PRPs (100%) and six researchers (67%) participated. According to the survey results, PAC members agreed that the supports (e. g. training, compensation) needed to contribute to the project were available throughout the project. The survey responses also showed that most members of the PAC felt their opinions and views were heard. Responses to the survey regarding diversity within the PAC were mixed. There were many suggestions for improving diversity and collaboration provided by PAC members during the semi-structured interviews. PAC members mentioned that PAC PRPs informed the co-development of research materials such as recruitment posters and interview guides for the RePORT study.ConclusionsThrough fostering a collaborative environment, we can engage a diverse group of people to work together meaningfully in health research. We have identified what works well, and areas for improvement within our research partnership involving PRPs and researchers as well as recommendations for POR projects more broadly, going forward.

Project description:BackgroundDigital innovations in medicine are disruptive technologies that can change the way diagnostic procedures and treatments are delivered. Such innovations are typically designed in teams with different disciplinary backgrounds. This paper concentrates on 2 interdisciplinary research teams with 20 members from the medicine and engineering sciences working jointly on digital health solutions.ObjectiveThe aim of this paper was to identify factors on the individual, team, and organizational levels that influence the implementation of interdisciplinary research projects elaborating on digital applications for medicine and, based on the results, to draw conclusions for the proactive design of the interdisciplinary research process to make these projects successful.MethodsTo achieve this aim, 2 interdisciplinary research teams were observed, and a small case study (response rate: 15/20, 75%) was conducted using a web-based questionnaire containing both closed and open self-report questions. The Spearman rank correlation coefficient was calculated to analyze the quantitative data. The answers to the open-ended questions were subjected to qualitative content analysis.ResultsWith regard to the interdisciplinary research projects investigated, the influencing factors of the three levels presented (individual, team, and organization) have proven to be relevant for interdisciplinary research cooperation.ConclusionsWith regard to recommendations for the future design of interdisciplinary cooperation, management aspects are addressed, that is, the installation of a coordinator, systematic definition of goals, required resources, and necessary efforts on the part of the involved interdisciplinary research partners. As only small groups were investigated, further research in this field is necessary to derive more general recommendations for interdisciplinary research teams.Trial registrationGerman Clinical Trials Register, DRKS00023909, https://www.drks.de/drks_web/navigate.do?navigationId=trial.HTML&TRIAL_ID=DRKS00023909 ; German Clinical Trials Register, DRKS00025077, https://www.drks.de/drks_web/navigate.do?navigationId=trial.HTML&TRIAL_ID=DRKS00025077.

Project description:BackgroundTreatment and prevention guidelines highlight the key role of health information and evidence-based psychosocial interventions for adolescent depression. Digital health technologies and psychoeducational interventions have been recommended to help engage young people and to provide accurate health information, enhance self-management skills, and promote social support. However, few digital psychoeducational interventions for adolescent depression have been robustly developed and evaluated in line with research guidance.ObjectiveWe aimed to evaluate the feasibility, acceptability, and potential impact of a theory-informed, co-designed digital intervention program, MoodHwb.MethodsWe used a mixed methods (quantitative and qualitative) approach to evaluate the program and the assessment process. Adolescents with or at elevated risk of depression and their parents and carers were recruited from mental health services, school counselors and nurses, and participants from a previous study. They completed a range of questionnaires before and after the program (related to the feasibility and acceptability of the program and evaluation process, and changes in mood, knowledge, attitudes, and behavior), and their Web usage was monitored. A subsample was also interviewed. A focus group was conducted with professionals from health, education, social, and youth services and charities. Interview and focus group transcripts were analyzed using thematic analysis with NVivo 10 (QSR International Pty Ltd).ResultsA total of 44 young people and 31 parents or carers were recruited, of which 36 (82%) young people and 21 (68%) parents or carers completed follow-up questionnaires. In all, 19 young people and 12 parents or carers were interviewed. Overall, 13 professionals from a range of disciplines participated in the focus group. The key themes from the interviews and groups related to the design features, sections and content, and integration and context of the program in the young person's life. Overall, the participants found the intervention engaging, clear, user-friendly, and comprehensive, and stated that it could be integrated into existing services. Young people found the "Self help" section and "Mood monitor" particularly helpful. The findings provided initial support for the intervention program theory, for example, depression literacy improved after using the intervention (difference in mean literacy score: 1.7, 95% CI 0.8 to 2.6; P<.001 for young people; 1.3, 95% CI 0.4 to 2.2; P=.006 for parents and carers).ConclusionsFindings from this early stage evaluation suggest that MoodHwb and the assessment process were feasible and acceptable, and that the intervention has the potential to be helpful for young people, families and carers as an early intervention program in health, education, social, and youth services and charities. A randomized controlled trial is needed to further evaluate the digital program.

Dataset Information

The Impact of Intervention Design on User Engagement in Digital Therapeutics Research: Factorial Experiment With a Mixed Methods Study.

Background

Objective

Methods

Results

Conclusions

Trial registration

Publications

The Impact of Intervention Design on User Engagement in Digital Therapeutics Research: Factorial Experiment With a Mixed Methods Study.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets