Long-term participant retention and engagement patterns in an app and wearable-based multinational remote digital depression study.
Ontology highlight
ABSTRACT:
SUBMITTER: Zhang Y
PROVIDER: S-EPMC9938183 | biostudies-literature | 2023 Feb
REPOSITORIES: biostudies-literature
Publications
Long-term participant retention and engagement patterns in an app and wearable-based multinational remote digital depression study.
Zhang Yuezhou Y Pratap Abhishek A Folarin Amos A AA Sun Shaoxiong S Cummins Nicholas N Matcham Faith F Vairavan Srinivasan S Dineley Judith J Ranjan Yatharth Y Rashid Zulqarnain Z Conde Pauline P Stewart Callum C White Katie M KM Oetzmann Carolin C Ivan Alina A Lamers Femke F Siddi Sara S Rambla Carla Hernández CH Simblett Sara S Nica Raluca R Mohr David C DC Myin-Germeys Inez I Wykes Til T Haro Josep Maria JM Penninx Brenda W J H BWJH Annas Peter P Narayan Vaibhav A VA Hotopf Matthew M Dobson Richard J B RJB
NPJ digital medicine 20230217 1
Recent growth in digital technologies has enabled the recruitment and monitoring of large and diverse populations in remote health studies. However, the generalizability of inference drawn from remotely collected health data could be severely impacted by uneven participant engagement and attrition over the course of the study. We report findings on long-term participant retention and engagement patterns in a large multinational observational digital study for depression containing active (survey ...[more]
Similar Datasets
Project description:BackgroundMobile health (mHealth) apps are revolutionizing the way clinicians and researchers monitor and manage the health of their participants. However, many studies using mHealth apps are hampered by substantial participant dropout or attrition, which may impact the representativeness of the sample and the effectiveness of the study. Therefore, it is imperative for researchers to understand what makes participants stay with mHealth apps or studies using mHealth apps.ObjectiveThis study aimed to review the current peer-reviewed research literature to identify the notable factors and strategies used in adult participant engagement and retention.MethodsWe conducted a systematic search of PubMed, MEDLINE, and PsycINFO databases for mHealth studies that evaluated and assessed issues or strategies to improve the engagement and retention of adults from 2015 to 2020. We followed the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. Notable themes were identified and narratively compared among different studies. A binomial regression model was generated to examine the factors affecting retention.ResultsOf the 389 identified studies, 62 (15.9%) were included in this review. Overall, most studies were partially successful in maintaining participant engagement. Factors related to particular elements of the app (eg, feedback, appropriate reminders, and in-app support from peers or coaches) and research strategies (eg, compensation and niche samples) that promote retention were identified. Factors that obstructed retention were also identified (eg, lack of support features, technical difficulties, and usefulness of the app). The regression model results showed that a participant is more likely to drop out than to be retained.ConclusionsRetaining participants is an omnipresent challenge in mHealth studies. The insights from this review can help inform future studies about the factors and strategies to improve participant retention.
| S-EPMC9092233 | biostudies-literature
Project description:BackgroundEarly study exit is detrimental to statistical power and increases the risk for bias in Alzheimer's disease clinical trials. Previous analyses in early phase academic trials demonstrated associations between rates of trial incompletion and participants' study partner type, with participants enrolling with non-spouse study partners being at greater risk.MethodsWe conducted secondary analyses of two multinational phase III trials of semagacestat, an oral gamma secretase inhibitor, for mild-to-moderate AD dementia. Cox's proportional hazards regression model was used to estimate the relationship between study partner type and the risk of early exit from the trial after adjustment for a priori identified potential confounding factors. Additionally, we used a random forest model to identify top predictors of dropout.ResultsAmong participants with spousal, adult child, and other study partners, respectively, 35%, 38%, and 36% dropped out or died prior to protocol-defined study completion, respectively. In unadjusted models, the risk of trial incompletion differed by study partner type (unadjusted p value = 0.027 for test of differences by partner type), but in models adjusting for potential confounding factors, the differences were not statistically significant (p value = 0.928). In exploratory modeling, participant age was identified as the primary characteristic to explain the relationship between study partner type and the risk of failing to complete the trial. Participant age was also the strongest predictor of trial incompletion in the random forest model.ConclusionsAfter adjustment for age, no differences in the risk of incompletion were observed when comparing participants with different study partner types in these trials. Differences between our findings and the findings of previous studies may be explained by differences in trial phase, size, geographic regions, or the composition of academic and non-academic sites.
| S-EPMC7791680 | biostudies-literature
Project description:BackgroundMany current research needs can only be addressed using very large cohorts. In such studies, traditional one-on-one phone, face-to-face, or paper-based engagement may not be feasible. The only realistic mechanism for maintaining engagement and participation at this scale is via digital methods. Given the substantial investment being made into very large birth cohort studies, evidence for optimal methods of participant engagement, participation, and retention over sustained periods without in-person contact from researchers is paramount.ObjectiveThis study aims to provide an overview of systematic reviews and meta-analyses evaluating alternative strategies for maximizing participant engagement and retention rates in large-scale studies using digital methods.MethodsWe used a rapid review method by searching PubMed and Ovid MEDLINE databases from January 2012 to December 2019. Studies evaluating at least 1 e-engagement, participation, or retention strategy were eligible. Articles were screened for relevance based on preset inclusion and exclusion criteria. The methodological quality of the included reviews was assessed using the AMSTAR-2 (Assessing the Methodological Quality of Systematic Reviews 2) measurement tool, and a narrative synthesis of the data was conducted.ResultsThe literature search yielded 19 eligible reviews. Overall, 63% (n=12) of these reviews reported on the effectiveness of e-engagement or participation promotion strategies. These evaluations were generally not conducted within very large observational digital cohorts. Most of the contributing reviews included multipurpose cohort studies (with both observational and interventional elements) conducted in clinical and research settings. Email or SMS text message reminders, SMS text messages or voice notifications, and incentives were the most commonly used design features to engage and retain participants. For parental outcomes, engagement-facilitation interventions influenced uptake and behavior change, including video feedback, goal setting, and intensive human facilitation and support. Participant-stated preferences for content included new knowledge, reminders, solutions, and suggestions about health issues presented in a clear, short, and personalized way. Perinatal and postpartum women valued self-monitoring and personalized feedback. Digital reminders and multiple SMS text messages were specific strategies that were found to increase adherence to medication and clinic attendance, respectively.ConclusionsThis review adds to the growing literature evaluating methods to optimize engagement and participation that may apply to large-scale studies using digital methods; it is promising that most e-engagement and participation promotion strategies appear to be effective. However, these reviews canvassed relatively few strategies, suggesting that few alternative strategies have been experimentally evaluated. The reviews also revealed a dearth of experimental evidence generated within very large observational digital cohort studies, which may reflect the small number of such studies worldwide. Thus, very large studies may need to proactively build in experimental opportunities to test engagement and retention approaches to enhance the success of their own and other large digital contact studies.
| S-EPMC8164122 | biostudies-literature
Project description:BackgroundThe Internet provides us with tools (user metrics or paradata) to evaluate how users interact with online interventions. Analysis of these paradata can lead to design improvements.ObjectiveThe objective was to explore the qualities of online participant engagement in an online intervention. We analyzed the paradata in a randomized controlled trial of alternative versions of an online intervention designed to promote consumption of fruit and vegetables.MethodsVolunteers were randomized to 1 of 3 study arms involving several online sessions. We created 2 indirect measures of breadth and depth to measure different dimensions and dynamics of program engagement based on factor analysis of paradata measures of Web pages visited and time spent online with the intervention materials. Multiple regression was used to assess influence of engagement on retention and change in dietary intake.ResultsBaseline surveys were completed by 2513 enrolled participants. Of these, 86.3% (n = 2168) completed the follow-up surveys at 3 months, 79.6% (n = 2027) at 6 months, and 79.4% (n = 1995) at 12 months. The 2 tailored intervention arms exhibited significantly more engagement than the untailored arm (P < .01). Breadth and depth measures of engagement were significantly associated with completion of follow-up surveys (odds ratios [OR] = 4.11 and 2.12, respectively, both P values < .001). The breadth measure of engagement was also significantly positively associated with a key study outcome, the mean increase in fruit and vegetable consumption (P < .001).ConclusionsBy exploring participants' exposures to online interventions, paradata are valuable in explaining the effects of tailoring in increasing participant engagement in the intervention. Controlling for intervention arm, greater engagement is also associated with retention of participants and positive change in a key outcome of the intervention, dietary change. This paper demonstrates the utility of paradata capture and analysis for evaluating online health interventions.Trial registrationNCT00169312; http://clinicaltrials.gov/ct2/show/NCT00169312 (Archived by WebCite at http://www.webcitation.org/5u8sSr0Ty).
| S-EPMC3056524 | biostudies-literature
Project description:BackgroundLow engagement with mental health apps continues to limit their impact. New approaches to help match patients to the right app may increase engagement by ensuring the app they are using is best suited to their mental health needs.ObjectiveThis study aims to pilot how digital phenotyping, using data from smartphone sensors to infer symptom, behavioral, and functional outcomes, could be used to match people to mental health apps and potentially increase engagement.MethodsAfter 1 week of collecting digital phenotyping data with the mindLAMP app (Beth Israel Deaconess Medical Center), participants were randomly assigned to the digital phenotyping arm, receiving feedback and recommendations based on those data to select 1 of 4 predetermined mental health apps (related to mood, anxiety, sleep, and fitness), or the control arm, selecting the same apps but without any feedback or recommendations. All participants used their selected app for 4 weeks with numerous metrics of engagement recorded, including objective screentime measures, self-reported engagement measures, and Digital Working Alliance Inventory scores.ResultsA total of 82 participants enrolled in the study; 17 (21%) dropped out of the digital phenotyping arm and 18 (22%) dropped out from the control arm. Across both groups, few participants chose or were recommended the insomnia or fitness app. The majority (39/47, 83%) used a depression or anxiety app. Engagement as measured by objective screen time and Digital Working Alliance Inventory scores were higher in the digital phenotyping arm. There was no correlation between self-reported and objective metrics of app use. Qualitative results highlighted the importance of habit formation in sustained app use.ConclusionsThe results suggest that digital phenotyping app recommendation is feasible and may increase engagement. This approach is generalizable to other apps beyond the 4 apps selected for use in this pilot, and practical for real-world use given that the study was conducted without any compensation or external incentives that may have biased results. Advances in digital phenotyping will likely make this method of app recommendation more personalized and thus of even greater interest.
| S-EPMC11615540 | biostudies-literature
Project description:Mobile phone applications ("apps") are potentially an effective, low-burden method to collect patient-reported outcomes outside the clinical setting. Using such apps consistently and in a timely way is critical for complete and accurate data capture, but no studies of concurrent reporting by cancer patient-caregiver dyads have been published in the peer-reviewed literature. This study assessed app engagement, defined as adherence, timing, and attrition with two smartphone applications, one for adult cancer patients and one for their informal caregivers. This was a single-arm, pilot study in which adult cancer patients undergoing IV chemotherapy or immunotherapy used the DigiBioMarC app, and their caregivers used the TOGETHERCare app, for approximately one month to report weekly on the patients' symptoms and wellbeing. Using app timestamp metadata, we assessed user adherence, overall and by participant characteristics. Fifty patient-caregiver dyads completed the study. Within the one-month study period, both adult cancer patients and their informal caregivers were highly adherent, with app activity completion at 86% for cancer patients and 84% for caregivers. Caregivers completed 86% of symptom reports, while cancer patients completed 89% of symptom reports. Cancer patients and their caregivers completed most activities within 48 h of availability on the app. These results suggest that the DigiBioMarC and TOGETHERCare apps can be used to collect patient- and caregiver-reported outcomes data during intensive treatment. From our research, we conclude that metadata from mobile apps can be used to inform clinical teams about study participants' engagement and wellbeing outside the clinical setting.
| S-EPMC10858186 | biostudies-literature
Project description:BackgroundFollowing the need for the prevention of noncommunicable diseases, mobile health (mHealth) apps are increasingly used for promoting lifestyle behavior changes. Although mHealth apps have the potential to reach all population segments, providing accessible and personalized services, their effectiveness is often limited by low participant engagement and high attrition rates.ObjectiveThis study concerns a large-scale, open-access mHealth app, based in the Netherlands, focused on improving the lifestyle behaviors of its participants. The study examines whether periodic email prompts increased participant engagement with the mHealth app and how this effect evolved over time. Points gained from the activities in the app were used as an objective measure of participant engagement with the program. The activities considered were physical workouts tracked through the mHealth app and interactions with the web-based coach.MethodsThe data analyzed covered 22,797 unique participants over a period of 78 weeks. A hidden Markov model (HMM) was used for disentangling the overtime effects of periodic email prompts on participant engagement with the mHealth app. The HMM accounted for transitions between latent activity states, which generated the observed measure of points received in a week.ResultsThe HMM indicated that, on average, 70% (15,958/22,797) of the participants were in the inactivity state, gaining 0 points in total per week; 18% (4103/22,797) of the participants were in the average activity state, gaining 27 points per week; and 12% (2736/22,797) of the participants were in the high activity state, gaining 182 points per week. Receiving and opening a generic email was associated with a 3 percentage point increase in the likelihood of becoming active in that week, compared with the weeks when no email was received. Examining detailed email categories revealed that the participants were more likely to increase their activity level following emails that were in line with the program's goal, such as emails regarding health campaigns, while being resistant to emails that deviated from the program's goal, such as emails regarding special deals.ConclusionsParticipant engagement with a behavior change mHealth app can be positively influenced by email prompts, albeit to a limited extent. Given the relatively low costs associated with emails and the high population reach that mHealth apps can achieve, such instruments can be a cost-effective means of increasing participant engagement in the stride toward improving program effectiveness.
| S-EPMC10214118 | biostudies-literature
Project description:BackgroundWhile many digital mental health interventions (DMHIs) have been found to be efficacious, patient engagement with DMHIs has increasingly emerged as a concern for implementation in real-world clinical settings. To address engagement, we must first understand what standard engagement levels are in the context of randomized controlled trials (RCTs) and how these compare with other treatments.ObjectiveThis scoping review aims to examine the state of reporting on intervention engagement in RCTs of mobile app-based interventions intended to treat symptoms of depression. We sought to identify what engagement metrics are and are not routinely reported as well as what the metrics that are reported reflect about standard engagement levels.MethodsWe conducted a systematic search of 7 databases to identify studies meeting our eligibility criteria, namely, RCTs that evaluated use of a mobile app-based intervention in adults, for which depressive symptoms were a primary outcome of interest. We then extracted 2 kinds of information from each article: intervention details and indices of DMHI engagement. A 5-element framework of minimum necessary DMHI engagement reporting was derived by our team and guided our data extraction. This framework included (1) recommended app use as communicated to participants at enrollment and, when reported, app adherence criteria; (2) rate of intervention uptake among those assigned to the intervention; (3) level of app use metrics reported, specifically number of uses and time spent using the app; (4) duration of app use metrics (ie, weekly use patterns); and (5) number of intervention completers.ResultsDatabase searching yielded 2083 unique records. Of these, 22 studies were eligible for inclusion. Only 64% (14/22) of studies included in this review specified rate of intervention uptake. Level of use metrics was only reported in 59% (13/22) of the studies reviewed. Approximately one-quarter of the studies (5/22, 23%) reported duration of use metrics. Only half (11/22, 50%) of the studies reported the number of participants who completed the app-based components of the intervention as intended or other metrics related to completion. Findings in those studies reporting metrics related to intervention completion indicated that between 14.4% and 93.0% of participants randomized to a DMHI condition completed the intervention as intended or according to a specified adherence criteria.ConclusionsFindings suggest that engagement was underreported and widely varied. It was not uncommon to see completion rates at or below 50% (11/22) of those participants randomized to a treatment condition or to simply see completion rates not reported at all. This variability in reporting suggests a failure to establish sufficient reporting standards and limits the conclusions that can be drawn about level of engagement with DMHIs. Based on these findings, the 5-element framework applied in this review may be useful as a minimum necessary standard for DMHI engagement reporting.
| S-EPMC9617183 | biostudies-literature
Project description:BackgroundAlthough digital maternity records (DMRs) have been evaluated in the past, no previous work investigated usability or acceptance through an observational usability study.ObjectiveThe primary objective was to assess the usability and perception of a DMR smartphone app for pregnant women. The secondary objective was to assess personal preferences and habits related to online information searching, wearable data presentation and interpretation, at-home examination, and sharing data for research purposes during pregnancy.MethodsA DMR smartphone app was developed. Key features such as wearable device integration, study functionalities (eg, questionnaires), and common pregnancy app functionalities (eg, mood tracker) were included. Women who had previously given birth were invited to participate. Participants completed 10 tasks while asked to think aloud. Sessions were conducted via Zoom. Video, audio, and the shared screen were recorded for analysis. Task completion times, task success, errors, and self-reported (free text) feedback were evaluated. Usability was measured through the System Usability Scale (SUS) and User Experience Questionnaire (UEQ). Semistructured interviews were conducted to explore the secondary objective.ResultsA total of 11 participants (mean age 34.6, SD 2.2 years) were included in the study. A mean SUS score of 79.09 (SD 18.38) was achieved. The app was rated "above average" in 4 of 6 UEQ categories. Sixteen unique features were requested. We found that 5 of 11 participants would only use wearables during pregnancy if requested to by their physician, while 10 of 11 stated they would share their data for research purposes.ConclusionsPregnant women rely on their medical caregivers for advice, including on the use of mobile and ubiquitous health technology. Clear benefits must be communicated if issuing wearable devices to pregnant women. Participants that experienced pregnancy complications in the past were overall more open toward the use of wearable devices in pregnancy. Pregnant women have different opinions regarding access to, interpretation of, and reactions to alerts based on wearable data. Future work should investigate personalized concepts covering these aspects.
| S-EPMC10750977 | biostudies-literature
Project description:AimsConditional average treatment effects are often reported in intervention studies, in which assumptions are made regarding how effects are similar across a heterogeneous sample. Nonetheless, differing factors, such as genetics, age, and sex, can impact an intervention's effect on outcomes. The study aimed to estimate the individualized effects of a digital alcohol intervention among individuals looking online to reduce their drinking.MethodsWe used data from a randomized controlled trial (RCT), including 2129 adults from the Swedish general population. The RCT concerned a text message-based alcohol intervention that sought to engender change through increasing knowledge on how to change and instilling confidence in changing behaviour. Outcomes were total weekly alcohol consumption and monthly heavy episodic drinking. Individualized treatment effects were modelled using baseline characteristics (age, gender, alcohol consumption, and psychosocial variables) and engagement with the intervention content.ResultsWe found evidence that the effects of the digital alcohol intervention were heterogeneous concerning participants' age, baseline alcohol consumption, confidence, and importance. For heavy episodic drinking, there was evidence that effects were heterogeneous concerning age, sex, and baseline alcohol consumption. Overall, women, older individuals, and heavier drinkers benefitted more from the intervention in terms of effect size. In addition, participants who engaged more with the goal-setting and screening content reported better outcomes.ConclusionsThe results highlight how different individuals respond differently to a digital alcohol intervention. This allows insight into who benefits the most and least from the intervention and highlights the potential merit of designing interventions adapted to different individuals' needs.
| S-EPMC11260484 | biostudies-literature