Dataset Information

Reliability of Data Collected via Ecological Momentary Assessment on the Example of FeverApp Registry.

ABSTRACT: The FeverApp registry is an ambulant ecological momentary assessment (EMA) model registry focusing on research of fever in children. Verification of EMA reliability is a challenge, due to absence of other source data. To ensure the reliability of EMA data, 973 families were invited to reassess their documentation in a survey. The survey contained questions (a) regarding the number of children, (b) genuineness of entries, (c) completeness of submitted fever episodes, (d) medication, (e) usefulness and further usage of the app. Of those invited, 438 families (45% response rate) participated in the survey. Of these, 363 (83%) families have registered all their children, 208 families have one child. The majority (n = 325, 74.2%) of families stated that they only made genuine entries in the app. Agreement between survey and app for fever episodes is 90% (Cohen's κ = 0.75 [0.66, 0.82]). Medication shows 73.7% agreement, κ = 0.49 [0.42; 0.54]. The majority (n = 245, 55.9%) consider the app as an additional benefit and 87.3% would like to use it further. Email surveys are a possible approach to evaluate EMA based registry data. The possible observation units (children and fever episodes) show an adequate reliability. With this approach, surveys of further samples and variables could help to improve the quality of EMA based registries.

SUBMITTER: Rathjens L

PROVIDER: S-EPMC9954832 | biostudies-literature | 2023 Feb

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Reliability of Data Collected via Ecological Momentary Assessment on the Example of FeverApp Registry.

Rathjens Larisa L Gwiasda Moritz M Schwarz Silke S Möhler Ricarda R Martin David D DD Jenetzky Ekkehart E

Children (Basel, Switzerland) 20230215 2

The FeverApp registry is an ambulant ecological momentary assessment (EMA) model registry focusing on research of fever in children. Verification of EMA reliability is a challenge, due to absence of other source data. To ensure the reliability of EMA data, 973 families were invited to reassess their documentation in a survey. The survey contained questions (a) regarding the number of children, (b) genuineness of entries, (c) completeness of submitted fever episodes, (d) medication, (e) usefulnes ...[more]

PMID: 36832514

Similar Datasets

Project description:BackgroundEmotional clarity has often been assessed with self-report measures, but efforts have also been made to measure it passively, which has advantages such as avoiding potential inaccuracy in responses stemming from social desirability bias or poor insight into emotional clarity. Response times (RTs) to emotion items administered in ecological momentary assessments (EMAs) may be an indirect indicator of emotional clarity. Another proposed indicator is the drift rate parameter, which assumes that, aside from how fast a person responds to emotion items, the measurement of emotional clarity also requires the consideration of how careful participants were in providing responses.ObjectiveThis paper aims to examine the reliability and validity of RTs and drift rate parameters from EMA emotion items as indicators of individual differences in emotional clarity.MethodsSecondary data analysis was conducted on data from 196 adults with type 1 diabetes who completed a 2-week EMA study involving the completion of 5 to 6 surveys daily. If lower RTs and higher drift rates (from EMA emotion items) were indicators of emotional clarity, we hypothesized that greater levels (ie, higher clarity) should be associated with greater life satisfaction; lower levels of neuroticism, depression, anxiety, and diabetes distress; and fewer difficulties with emotion regulation. Because prior literature suggested emotional clarity could be valence specific, EMA items for negative affect (NA) and positive affect were examined separately.ResultsReliability of the proposed indicators of emotional clarity was acceptable with a small number of EMA prompts (ie, 4 to 7 prompts in total or 1 to 2 days of EMA surveys). Consistent with expectations, the average drift rate of NA items across multiple EMAs had expected associations with other measures, such as correlations of r=-0.27 (P<.001) with depression symptoms, r=-0.27 (P=.001) with anxiety symptoms, r=-0.15 (P=.03) with emotion regulation difficulties, and r=0.63 (P<.001) with RTs to NA items. People with a higher NA drift rate responded faster to NA emotion items, had greater subjective well-being (eg, fewer depression symptoms), and had fewer difficulties with overall emotion regulation, which are all aligned with the expectation for an emotional clarity measure. Contrary to expectations, the validities of average RTs to NA items, the drift rate of positive affect items, and RTs to positive affect items were not strongly supported by our results.ConclusionsStudy findings provided initial support for the validity of NA drift rate as an indicator of emotional clarity but not for that of other RT-based clarity measures. Evidence was preliminary because the sample size was not sufficient to detect small but potentially meaningful correlations, as the sample size of the diabetes EMA study was chosen for other more primary research questions. Further research on passive emotional clarity measures is needed.

Project description:BackgroundThe current methods of evaluating cognitive functioning typically rely on a single time point to assess and characterize an individual's performance. However, cognitive functioning fluctuates within individuals over time in relation to environmental, psychological, and physiological contexts. This limits the generalizability and diagnostic utility of single time point assessments, particularly among individuals who may exhibit large variations in cognition depending on physiological or psychological context (eg, those with type 1 diabetes [T1D], who may have fluctuating glucose concentrations throughout the day).ObjectiveWe aimed to report the reliability and validity of cognitive ecological momentary assessment (EMA) as a method for understanding between-person differences and capturing within-person variation in cognition over time in a community sample and sample of adults with T1D.MethodsCognitive performance was measured 3 times a day for 15 days in the sample of adults with T1D (n=198, recruited through endocrinology clinics) and for 10 days in the community sample (n=128, recruited from TestMyBrain, a web-based citizen science platform) using ultrabrief cognitive tests developed for cognitive EMA. Our cognitive EMA platform allowed for remote, automated assessment in participants' natural environments, enabling the measurement of within-person cognitive variation without the burden of repeated laboratory or clinic visits. This allowed us to evaluate reliability and validity in samples that differed in their expected degree of cognitive variability as well as the method of recruitment.ResultsThe results demonstrate excellent between-person reliability (ranging from 0.95 to 0.99) and construct validity of cognitive EMA in both the sample of adults with T1D and community sample. Within-person reliability in both samples (ranging from 0.20 to 0.80) was comparable with that observed in previous studies in healthy older adults. As expected, the full-length baseline and EMA versions of TestMyBrain tests correlated highly with one another and loaded together on the expected cognitive domains when using exploratory factor analysis. Interruptions had higher negative impacts on accuracy-based outcomes (β=-.34 to -.26; all P values <.001) than on reaction time-based outcomes (β=-.07 to -.02; P<.001 to P=.40).ConclusionsWe demonstrated that ultrabrief mobile assessments are both reliable and valid across 2 very different clinic versus community samples, despite the conditions in which cognitive EMAs are administered, which are often associated with more noise and variability. The psychometric characteristics described here should be leveraged appropriately depending on the goals of the cognitive assessment (eg, diagnostic vs everyday functioning) and the population being studied.

Project description:IntroductionDigital health interventions are an effective way to treat depression, but it is still largely unclear how patients' individual symptoms evolve dynamically during such treatments. Data-driven forecasts of depressive symptoms would allow to greatly improve the personalisation of treatments. In current forecasting approaches, models are often trained on an entire population, resulting in a general model that works overall, but does not translate well to each individual in clinically heterogeneous, real-world populations. Model fairness across patient subgroups is also frequently overlooked. Personalised models tailored to the individual patient may therefore be promising.MethodsWe investigate different personalisation strategies using transfer learning, subgroup models, as well as subject-dependent standardisation on a newly-collected, longitudinal dataset of depression patients undergoing treatment with a digital intervention ( N=65 patients recruited). Both passive mobile sensor data as well as ecological momentary assessments were available for modelling. We evaluated the models' ability to predict symptoms of depression (Patient Health Questionnaire-2; PHQ-2) at the end of each day, and to forecast symptoms of the next day.ResultsIn our experiments, we achieve a best mean-absolute-error (MAE) of 0.801 (25% improvement) for predicting PHQ-2 values at the end of the day with subject-dependent standardisation compared to a non-personalised baseline ( MAE=1.062 ). For one day ahead-forecasting, we can improve the baseline of 1.539 by 12% to a MAE of 1.349 using a transfer learning approach with shared common layers. In addition, personalisation leads to fairer models at group-level.DiscussionOur results suggest that personalisation using subject-dependent standardisation and transfer learning can improve predictions and forecasts, respectively, of depressive symptoms in participants of a digital depression intervention. We discuss technical and clinical limitations of this approach, avenues for future investigations, and how personalised machine learning architectures may be implemented to improve existing digital interventions for depression.

Project description:PurposeMost of what is known about resident burnout and wellness comes from cross-sectional snapshot surveys. The purpose of this study was to elicit qualitative perspectives on wellness from a cohort of internal residents over time using ecological momentary assessment.MethodDrawing on principles of ecological momentary assessment, 13 different open-ended survey prompts were delivered between October and March during the 2019-2020 academic year. Participants were 88 randomly selected internal medicine residents from 4 internal medicine training programs in the Northeast.ResultsThe response rate was 95%. Three main themes regarding wellness were self, program/education environment, and medical/structural system. A fourth theme, the desire to provide quality patient care, cut across all other themes. The patient care theme repeatedly stressed residents' desire to spend more time with patients. The self theme primarily reflected messages about personal emotions and the need for work-life balance and wellness. The program/education environment theme reflected the value of learning, teamwork and community, and program culture. The medical/structural system theme showed that residents' experiences were shaped by the efficiency of their days and largely a product of their schedules and administrative support. Closing advice to future trainees was optimistic and reassuring.ConclusionsWhile findings support much of what has been learned via single-occasion survey snapshots, an ecological momentary assessment design allowed a deeper dive into contextual associations. The results affirm the primacy of patient care and also highlight the value of teamwork and culture. Peers and program leaders are heavily influential in setting the tone for the learning experience, whether for the day or with a more enduring message of respect and support. There is opportunity to maximize high- or higher-value learning experiences for residents and find solutions to reduce and reframe the perceived "low-value administrative work" that is part of care coordination.

Dataset Information

Reliability of Data Collected via Ecological Momentary Assessment on the Example of FeverApp Registry.

Publications

Reliability of Data Collected via Ecological Momentary Assessment on the Example of FeverApp Registry.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets