Ecological Momentary Assessment: A Meta-Analysis on Designs, Samples, and Compliance Across Research Fields.
Ontology highlight
ABSTRACT:
SUBMITTER: Wrzus C
PROVIDER: S-EPMC9999286 | biostudies-literature | 2023 Apr
REPOSITORIES: biostudies-literature
Publications
Ecological Momentary Assessment: A Meta-Analysis on Designs, Samples, and Compliance Across Research Fields.
Wrzus Cornelia C Neubauer Andreas B AB
Assessment 20220111 3
Ecological Momentary Assessments (i.e., EMA, repeated assessments in daily life) are widespread in many fields of psychology and related disciplines. Yet, little knowledge exists on how differences in study designs and samples predict study compliance and dropout-two central parameters of data quality in (micro-)longitudinal research. The current meta-analysis included k = 477 articles (496 samples, total <i>N</i> = 677,536). For each article, we coded the design, sample characteristics, complia ...[more]
Similar Datasets
Project description:Background and aimsWhile there are considerable benefits to Ecological Momentary Assessment (EMA), poor compliance with assessment protocols has been identified as a limitation, particularly in substance users. Our aim was to identify the pooled compliance rate of EMA studies in substance users and examine variables that may influence compliance with EMA protocols, such as the length and frequency of assessments.DesignA meta-analysis and meta-regression of all possible studies (randomized controlled trials and longitudinal) which incorporated EMA protocols, examining substance use.SettingStudies took place from 1998 to 2017, in numerous countries world-wide.ParticipantsOne hundred and twenty-six studies were identified, contributing a total of 19 431 participants (52.32% male, mean age = 28.86).MeasurementsCompliance data, the proportion of responses to the study protocol, were extracted from each study alongside prompt frequency, total length of assessment period, substance use population and device used to administer EMA prompts.FindingsThe pooled compliance rate across all studies was 75.06% [95% confidence interval (CI) = 72.37%, 77.65%]. There was no evidence that compliance rates were significantly associated with prompt frequency [Q(3) = 7.35, P = 0.061], length of assessment period [Q(2) = 2.40, P = 0.301], substance type [Q(3) = 6.30, P = 0.098] or device administration [Q(4) = 4.28, P = 0.369]. However, dependent samples (69.80%) had lower compliance rates than non-dependent samples [76.02%; Q(1) = 4.13, P = 0.042].ConclusionsThe pooled compliance rate for Ecological Momentary Assessment studies in substance-using populations from 1998 to 2017 was lower than the recommended rate of 80%, and was not associated with frequency or duration of assessments.
| S-EPMC6492133 | biostudies-literature
Project description:BackgroundBorderline personality disorder (BPD) is characterized by frequent and intense moment-to-moment changes in affect, behavior, identity, and interpersonal relationships, which typically result in significant and negative deterioration of the person's overall functioning and well-being. Measuring and characterizing the rapidly changing patterns of instability in BPD dysfunction as they occur in a person's daily life can be challenging. Ecological momentary assessment (EMA) is a method that can capture highly dynamic processes in psychopathology research and, thus, is well suited to study intense variability patterns across areas of dysfunction in BPD. EMA studies are characterized by frequent repeated assessments that are delivered to participants in real-life, real-time settings using handheld devices capable of registering responses to short self-report questions in daily life. Compliance in EMA research is defined as the proportion of prompts answered by the participant, considering all planned prompts sent. Low compliance with prompt schedules can compromise the relative advantages of using this method. Despite the growing EMA literature on BPD in recent years, findings regarding study design features that affect compliance with EMA protocols have not been compiled, aggregated, and estimated.ObjectiveThis systematic meta-analytic review aimed to investigate the relationship between study design features and participant compliance in EMA research of BPD.MethodsA systematic review was conducted on November 12, 2021, following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) and MOOSE (Meta-analyses of Observational Studies in Epidemiology) guidelines to search for articles featuring EMA studies of BPD that reported compliance rates and included sufficient data to extract relevant design features. For studies with complete data, random-effect models were used to estimate the overall compliance rate and explore its association with design features.ResultsIn total, 28 peer-reviewed EMA studies comprising 2052 participants were included in the study. Design features (sampling strategy, average prompting frequency, number of items, response window, sampling device, financial incentive, and dropout rate) showed a large variability across studies, and many studies did not report design features. The meta-analytic synthesis was restricted to 64% (18/28) of articles and revealed a pooled compliance rate of 79% across studies. We did not find any significant relationship between design features and compliance rates.ConclusionsOur results show wide variability in the design and reporting of EMA studies assessing BPD. Compliance rates appear to be stable across varying setups, and it is likely that standard design features are not directly responsible for improving or diminishing compliance. We discuss possible nonspecific factors of study design that may have an impact on compliance. Given the promise of EMA research in BPD, we also discuss the importance of unifying standards for EMA reporting so that data stemming from this rich literature can be aggregated and interpreted jointly.
| S-EPMC10131785 | biostudies-literature
Project description:ObjectiveEcological momentary assessment (EMA) refers to the repeated sampling of information about an individual's symptoms and behaviours, enabling the capture of ecologically meaningful real-time information in a timely manner. Compliance with EMA is critical in determining the validity of an assessment. However, there is limited evidence related to how the elderly comply with EMA programmes or the factors that are associated with compliance.DesignSystematic review and meta-analysis.Data sourcesPubMed, Embase, the Cochrane Library and Web of Science were searched up to 17 July 2022.Eligibility criteriaWe included observational studies on EMA in the elderly reported in English.Data extraction and synthesisTwo investigators independently performed screening and data extraction. Discrepancies were resolved by discussion or a third investigator. A systematic review was carried out to characterise the basic characteristics of the participants and EMA programmes. Random-effects meta-analysis was conducted to assess overall compliance and to explore factors associated with differences in compliance among the elderly.ResultsA total of 20 studies with 2047 participants were included in the systematic review and meta-analysis. Meta-analysis showed that the combined compliance rate was 86.41% (95% CI: 77.38% to 92.20%; I2=96.4%; p<0.001). Subgroup analysis revealed high levels of heterogeneity in terms of the methods used to assess population classification, assessment method and assessment frequency, although these may not be the sources of heterogeneity. Meta-regression analysis showed that population classification and assessment period might have a significant impact on heterogeneity (p<0.05). Egger's test indicated significant publication bias (p<0.001).ConclusionsCompliance with EMA programmes is high in the elderly. It is recommended that scholars design reasonable EMA programmes according to the health status of the elderly in the future.
| S-EPMC10347474 | biostudies-literature
Project description:BackgroundMobile device-based ecological momentary assessment (mobile-EMA) is increasingly used to collect participants' data in real-time and in context. Although EMA offers methodological advantages, these advantages can be diminished by participant noncompliance. However, evidence on how well participants comply with mobile-EMA protocols and how study design factors associated with participant compliance is limited, especially in the youth literature.ObjectiveTo systematically and meta-analytically examine youth's compliance to mobile-EMA protocols and moderators of participant compliance in clinical and nonclinical settings.MethodsStudies using mobile devices to collect EMA data among youth (age ≤18 years old) were identified. A systematic review was conducted to describe the characteristics of mobile-EMA protocols and author-reported factors associated with compliance. Random effects meta-analyses were conducted to estimate the overall compliance across studies and to explore factors associated with differences in youths' compliance.ResultsThis review included 42 unique studies that assessed behaviors, subjective experiences, and contextual information. Mobile phones were used as the primary mode of EMA data collection in 48% (20/42) of the reviewed studies. In total, 12% (5/42) of the studies used wearable devices in addition to the EMA data collection platforms. About half of the studies (62%, 24/42) recruited youth from nonclinical settings. Most (98%, 41/42) studies used a time-based sampling protocol. Among these studies, most (95%, 39/41) prompted youth 2-9 times daily, for a study length ranging from 2-42 days. Sampling frequency and study length did not differ between studies with participants from clinical versus nonclinical settings. Most (88%, 36/41) studies with a time-based sampling protocol defined compliance as the proportion of prompts to which participants responded. In these studies, the weighted average compliance rate was 78.3%. The average compliance rates were not different between studies with clinical (76.9%) and nonclinical (79.2%; P=.29) and studies that used only a mobile-EMA platform (77.4%) and mobile platform plus additional wearable devices (73.0%, P=.36). Among clinical studies, the mean compliance rate was significantly lower in studies that prompted participants 2-3 times (73.5%) or 4-5 times (66.9%) compared with studies with a higher sampling frequency (6+ times: 89.3%). Among nonclinical studies, a higher average compliance rate was observed in studies that prompted participants 2-3 times daily (91.7%) compared with those that prompted participants more frequently (4-5 times: 77.4%; 6+ times: 75.0%). The reported compliance rates did not differ by duration of EMA period among studies from either clinical or nonclinical settings.ConclusionsThe compliance rate among mobile-EMA studies in youth is moderate but suboptimal. Study design may affect protocol compliance differently between clinical and nonclinical participants; including additional wearable devices did not affect participant compliance. A more consistent compliance-related result reporting practices can facilitate understanding and improvement of participant compliance with EMA data collection among youth.
| S-EPMC5425774 | biostudies-literature
Project description:BackgroundAdolescence through emerging adulthood represents a critical period associated with changes in lifestyle behaviors. Understanding the dynamic relationships between cognitive, social, and environmental contexts is informative for the development of interventions aiming to help youth sustain physical activity and limit sedentary time during this life stage. Ecological momentary assessment (EMA) is an innovative method involving real-time assessment of individuals' experiences and behaviors in their naturalistic or everyday environments; however, EMA compliance can be problematic due to high participant burdens.ObjectiveThis systematic review synthesized existing evidence pertaining to compliance in EMA studies that investigated wake-time movement behaviors among adolescent and emerging adult populations. Differences in EMA delivery scheme or protocol, EMA platforms, prompting schedules, and compensation methods-all of which can affect participant compliance and overall study quality-were examined.MethodsAn electronic literature search was conducted in PubMed, PsycINFO, and Web of Science databases to select relevant papers that assessed movement behaviors among the population using EMA and reported compliance information for inclusion (n=52) in October 2022. Study quality was assessed using a modified version of the Checklist for Reporting of EMA Studies (CREMAS).ResultsSynthesizing the existing evidence revealed several factors that influence compliance. The platform used for EMA studies could affect compliance and data quality in that studies using smartphones or apps might lessen additional burdens associated with delivering EMAs, yet most studies used web-based formats (n=18, 35%). Study length was not found to affect EMA compliance rates, but the timing and frequency of prompts may be critical factors associated with missingness. For example, studies that only prompted participants once per day had higher compliance (91% vs 77%), but more frequent prompts provided more comprehensive data for researchers at the expense of increased participant burden. Similarly, studies with frequent prompting within the day may provide more representative data but may also be perceived as more burdensome and result in lower compliance. Compensation type did not significantly affect compliance, but additional motivational strategies could be applied to encourage participant response.ConclusionsUltimately, researchers should consider the best strategies to limit burdens, balanced against requirements to answer the research question or phenomena being studied. Findings also highlight the need for greater consistency in reporting and more specificity when explaining procedures to understand how EMA compliance could be optimized in studies examining physical activity and sedentary time among youth.Trial registrationPROSPERO CRD42021282093; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=282093.
| S-EPMC11862778 | biostudies-literature
Project description:BackgroundEcological momentary assessment (EMA) is increasingly used to evaluate behavioral health processes over extended time periods. The validity of EMA for providing representative, real-world data with high temporal precision is threatened to the extent that EMA compliance drops over time.ObjectiveThis research builds on prior short-term studies by evaluating the time course of EMA compliance over 9 weeks and examines predictors of weekly compliance rates among cigarette-using adults.MethodsA total of 257 daily cigarette-using adults participating in a randomized controlled trial for smoking cessation completed daily smartphone EMA assessments, including 1 scheduled morning assessment and 4 random assessments per day. Weekly EMA compliance was calculated and multilevel modeling assessed the rate of change in compliance over the 9-week assessment period. Participant and study characteristics were examined as predictors of overall compliance and changes in compliance rates over time.ResultsCompliance was higher for scheduled morning assessments (86%) than for random assessments (58%) at the beginning of the EMA period (P<.001). EMA compliance declined linearly across weeks, and the rate of decline was greater for morning assessments (2% per week) than for random assessments (1% per week; P<.001). Declines in compliance were stronger for younger participants (P<.001), participants who were employed full-time (P=.03), and participants who subsequently dropped out of the study (P<.001). Overall compliance was higher among White participants compared to Black or African American participants (P=.001).ConclusionsThis study suggests that EMA compliance declines linearly but modestly across lengthy EMA protocols. In general, these data support the validity of EMA for tracking health behavior and hypothesized treatment mechanisms over the course of several months. Future work should target improving compliance among subgroups of participants and investigate the extent to which rapid declines in EMA compliance might prove useful for triggering interventions to prevent study dropout.Trial registrationClinicalTrials.gov NCT03262662; https://clinicaltrials.gov/ct2/show/NCT03262662.
| S-EPMC10337346 | biostudies-literature
Project description:Mind-wandering (MW) is a universal human phenomenon and revealing its nature contributes to understanding consciousness. The ecological momentary assessment (EMA), in which subjects report a momentary mental state, is a suitable method to investigate MW in a natural environment. Previous studies employed EMA to study MW and attempted to answer the most fundamental question: How often do we let our minds wander? However, reported MW occupancies vary widely among studies. Further, while some experimental settings may induce bias in MW reports, these designs have not been explored. Therefore, we searched PubMed and Web of Science for articles published until the end of 2020 and systematically reviewed 25 articles, and performed meta-analyses on 17 of them. Our meta-analysis found that people spend 34.504% of daily life in mind-wandering, and meta-regression revealed that using subject smartphones for EMA, frequent sampling, and long experimental duration significantly affect MW reports. This result indicates that EMA using subject smartphones may tend to collect sampling under habitual smartphone use. Furthermore, these results indicate the existence of reactivity, even in MW research. We provide fundamental knowledge of MW and discuss rough standards for EMA settings in future MW studies.
| S-EPMC9938857 | biostudies-literature
Project description:BackgroundMobile ecological momentary assessment (mEMA) permits real-time capture of self-reported participant behaviors and perceptual experiences. Reporting of mEMA protocols and compliance has been identified as problematic within systematic reviews of children, youth, and specific clinical populations of adults.ObjectiveThis study aimed to describe the use of mEMA for self-reported behaviors and psychological constructs, mEMA protocol and compliance reporting, and associations between key components of mEMA protocols and compliance in studies of nonclinical and clinical samples of adults.MethodsIn total, 9 electronic databases were searched (2006-2016) for observational studies reporting compliance to mEMA for health-related data from adults (>18 years) in nonclinical and clinical settings. Screening and data extraction were undertaken by independent reviewers, with discrepancies resolved by consensus. Narrative synthesis described participants, mEMA target, protocol, and compliance. Random effects meta-analysis explored factors associated with cohort compliance (monitoring duration, daily prompt frequency or schedule, device type, training, incentives, and burden score). Random effects analysis of variance (P≤.05) assessed differences between nonclinical and clinical data sets.ResultsOf the 168 eligible studies, 97/105 (57.7%) reported compliance in unique data sets (nonclinical=64/105 [61%], clinical=41/105 [39%]). The most common self-reported mEMA target was affect (primary target: 31/105, 29.5% data sets; secondary target: 50/105, 47.6% data sets). The median duration of the mEMA protocol was 7 days (nonclinical=7, clinical=12). Most protocols used a single time-based (random or interval) prompt type (69/105, 65.7%); median prompt frequency was 5 per day. The median number of items per prompt was similar for nonclinical (8) and clinical data sets (10). More than half of the data sets reported mEMA training (84/105, 80%) and provision of participant incentives (66/105, 62.9%). Less than half of the data sets reported number of prompts delivered (22/105, 21%), answered (43/105, 41%), criterion for valid mEMA data (37/105, 35.2%), or response latency (38/105, 36.2%). Meta-analysis (nonclinical=41, clinical=27) estimated an overall compliance of 81.9% (95% CI 79.1-84.4), with no significant difference between nonclinical and clinical data sets or estimates before or after data exclusions. Compliance was associated with prompts per day and items per prompt for nonclinical data sets. Although widespread heterogeneity existed across analysis (I2>90%), no compelling relationship was identified between key features of mEMA protocols representing burden and mEMA compliance.ConclusionsIn this 10-year sample of studies using the mEMA of self-reported health-related behaviors and psychological constructs in adult nonclinical and clinical populations, mEMA was applied across contexts and health conditions and to collect a range of health-related data. There was inconsistent reporting of compliance and key features within protocols, which limited the ability to confidently identify components of mEMA schedules likely to have a specific impact on compliance.
| S-EPMC7970161 | biostudies-literature
Project description:ObjectiveBinge eating has been associated with increased hunger, suggesting a role for impaired appetite regulation. Ecological momentary assessment (EMA) is ideally suited to examine whether hunger is a precipitant of binge eating but results from such studies have not been systematically reviewed. This study provides a meta-analysis of EMA studies that have examined hunger as an antecedent of binge eating.MethodElectronic database and manual searches produced seven EMA studies with N = 180 participants. Meta-analyses were conducted to compare: (1) pre-binge eating hunger to average ratings of hunger, and (2) pre-binge eating hunger to hunger before regular eating.ResultsAcross studies, hunger was significantly greater before binge eating compared with average hunger ratings, but was significantly lower before binge eating compared with before other eating episodes.DiscussionExcessive hunger does not appear to be a precipitant of binge eating because higher levels of hunger are observed before regular eating episodes. However, lower hunger before food consumption may contribute to the experience of a particular eating episode as a binge.
| S-EPMC3111846 | biostudies-literature
Project description:BackgroundNew methods for assessing diet in research are being developed to address the limitations of traditional dietary assessment methods. Mobile device-assisted ecological momentary diet assessment (mEMDA) is a new dietary assessment method that has not yet been optimized and has the potential to minimize recall biases and participant burden while maximizing ecological validity. There have been limited efforts to characterize the use of mEMDA in behavioral research settings.ObjectiveThe aims of this study were to summarize mEMDA protocols used in research to date, to characterize key aspects of these assessment approaches, and to discuss the advantages and disadvantages of mEMDA compared with the traditional dietary assessment methods as well as implications for future mEMDA research.MethodsStudies that used mobile devices and described mEMDA protocols to assess dietary intake were included. Data were extracted according to Preferred Reporting of Systematic Reviews and Meta-Analyses and Cochrane guidelines and then synthesized narratively.ResultsThe review included 20 studies with unique mEMDA protocols. Of these, 50% (10/20) used participant-initiated reports of intake at eating events (event-contingent mEMDA), and 50% (10/20) used researcher-initiated prompts requesting that participants report recent dietary intake (signal-contingent mEMDA). A majority of the study protocols (60%, 12/20) enabled participants to use mobile phones to report dietary data. Event-contingent mEMDA protocols most commonly assessed diet in real time, used dietary records for data collection (60%, 6/10), and provided estimates of energy and nutrient intake (60%, 6/10). All signal-contingent mEMDA protocols used a near real-time recall approach with unannounced (ie, random) abbreviated diet surveys. Most signal-contingent protocols (70%, 7/10) assessed the frequency with which (targeted) foods or food groups were consumed. Relatively few (30%, 6/20) studies compared mEMDA with the traditional dietary assessment methods.ConclusionsThis review demonstrates that mEMDA has the potential to reduce participant burden and recall bias, thus advancing the field beyond current dietary assessment methods while maximizing ecological validity.
| S-EPMC6280032 | biostudies-literature