Dataset Information

Improving the patient-reported outcome sections of clinical trial protocols: a mixed methods evaluation of educational workshops.

ABSTRACT:

Introduction

Failure to incorporate key patient-reported outcome (PRO) content in trial protocols affects the quality and interpretability of the collected data, contributing to research waste. Our group developed evidence-based training specifically addressing PRO components of protocols. We aimed to assess whether 2-day educational workshops improved the PRO completeness of protocols against consensus-based minimum standards provided in the SPIRIT-PRO Extension in 2018.

Method

Annual workshops were conducted 2011-2017. Participants were investigators/trialists from cancer clinical trials groups. Although developed before 2018, workshops covered 15/16 SPIRIT-PRO items. Participant feedback immediately post-workshop and, retrospectively, in November 2017 was summarised descriptively. Protocols were evaluated against SPIRIT-PRO by two independent raters for workshop protocols (developed post-workshop by participants) and control protocols (contemporaneous non-workshop protocols). SPIRIT-PRO items were assessed for completeness (0 = not addressed, 10 = fully addressed). Mann-Whitney U tests assessed whether workshop protocols scored higher than controls by item and overall.

Results

Participants (n = 107) evaluated the workshop positively. In 2017, 16/41 survey responders (39%) reported never applying in practice; barriers included role restrictions (14/41, 34%) and lack of time (5/41, 12%). SPIRIT-PRO overall scores did not differ between workshop (n = 13, median = 3.81/10, interquartile range = 3.24) and control protocols (n = 9, 3.51/10 (2.14)), (p = 0.35). Workshop protocols scored higher than controls on two items: 'specify PRO concepts/domains' (p = 0.05); 'methods for handling missing data' (p = 0.044).

Conclusion

Although participants were highly satisfied with these workshops, the completeness of PRO protocol content generally did not improve. Additional knowledge translation efforts are needed to assist protocol writers address SPIRIT-PRO guidance and avoid research waste that may eventuate from sub-optimal PRO protocol content.

SUBMITTER: King MT

PROVIDER: S-EPMC9470723 | biostudies-literature | 2022 Oct

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Improving the patient-reported outcome sections of clinical trial protocols: a mixed methods evaluation of educational workshops.

King Madeleine T MT Tait Margaret-Ann MA Campbell Rachel R Müller Fabiola F Rutherford Claudia C Beckmore Corinna C Chima Sophie S Langbecker Danette D Shaw Joanne J Mercieca-Bebber Rebecca R

Quality of life research : an international journal of quality of life aspects of treatment, care and rehabilitation 20220512 10

<h4>Introduction</h4>Failure to incorporate key patient-reported outcome (PRO) content in trial protocols affects the quality and interpretability of the collected data, contributing to research waste. Our group developed evidence-based training specifically addressing PRO components of protocols. We aimed to assess whether 2-day educational workshops improved the PRO completeness of protocols against consensus-based minimum standards provided in the SPIRIT-PRO Extension in 2018.<h4>Method</h4>A ...[more]

PMID: 35553325

Similar Datasets

Project description:ObjectivesTo examine the acceptability and validity of two patient-reported outcome measures (PROMs) for adult acne, comparing them to the validated Acne-specific Quality of Life (Acne-QoL) measure.DesignMixed-methods validation study.SettingParticipants were recruited by (1) mail-out through primary care if they had ever consulted for acne and received a prescription for acne treatment within the last 6 months, (2) opportunistically in secondary care and (3) poster advertisement in community venues.Participants221 (204 quantitative and 17 qualitative) participants with acne, aged 18-50 years.Outcome measuresQuantitative sub-study participants completed Acne-QoL, Skindex-16 and Comprehensive Acne Quality of Life Scale (CompAQ) at baseline, 24 hours and 6 weeks. Qualitative sub-study participants took part in cognitive think-aloud interviews, while completing the same measures. Transcribed audio recordings were analysed using inductive thematic analysis.ResultsQuantitative analyses suggested high internal consistency (Cronbach's alpha 0.74-0.96) and reliability (intraclass correlation coefficient values 0.88-0.97) for both questionnaires. Both scales showed floor effects on some subdomains. Skindex-16 and CompAQ showed good evidence of construct validity when compared with Acne-QoL with Spearman's correlation coefficients 0.54-0.81, and good repeatability over 24 hours.Qualitative data uncovered wide-ranging views regarding usability and acceptability. Interviewees held strong but differing views about layout, question/response wording, redundant/similar questions and guidance notes. Similarly, interviewees differed in perceptions of acceptability of the different scales, particularly on relatability of questions and emotive reactions to scales.ConclusionsAll PROMs performed well in statistical analyses. No PROM showed superior usability and acceptability in the qualitative study. Any PROM should be acceptable for further research in adult acne but researchers should consider the different domains and whether they will measure only facial or facial and trunk acne before making a selection. A new PROM or further evaluation of novel PROMs may be beneficial.

Project description:BackgroundDespite the increasing use of patient-reported outcome measures (PROMs) for collecting self-reported data among hospital outpatients, clinicians' use of these data remains suboptimal. Insight into this issue and strategies to enhance the use of PROMs are critical but limited.ObjectiveThis study aimed to examine clinicians' use of PROM data for value-based outpatient consultations and identify efforts to enhance their use of PROMs in a Dutch university hospital. First, we aimed to investigate clinicians' use of outpatients' PROM data in 2023, focusing on adoption, implementation, and maintenance. Second, we aimed to develop insights into the organizational-level strategies implemented to enhance clinicians' use of PROM data from 2020 to 2023. This included understanding the underlying rationales for these strategies and identifying strategies that appeared to be missing to address barriers or leverage facilitators. Third, we aimed to explore the key factors driving and constraining clinicians' use of PROMs in 2023.MethodsWe integrated data from 4 sources: 1-year performance data on clinicians' use of PROMs (n=70 subdepartments), internal hospital documents from a central support team (n=56), a survey among clinicians (n=47), and interviews with individuals contributing to the organizational-level implementation of PROMs (n=20). The Reach, Effectiveness, Adoption, Implementation, and Maintenance framework was used to analyze clinicians' adoption, implementation, and maintenance of PROMs. Strategies were analyzed using the Expert Recommendations for Implementing Change taxonomy, and results were structured around the constructs of capability, opportunity, and motivation.ResultsOn average, around 2023, clinicians accessed PROM data for approximately 3 of 20 (14%) patients during their outpatient consultation, despite numerous strategies to improve this practice. We identified issues in adoption, implementation, and maintenance. The hospital's strategies, shaped organically and pragmatically, were related to 27 (37%) out of 73 Expert Recommendations for Implementing Change strategies. These strategies focused on enhancing clinicians' capability, opportunity, and motivation. We found shortcomings in the quality of execution and completeness of strategies in relation to addressing all barriers and leveraging facilitators. We identified variations in the factors influencing the use of PROMs among frequent PROM users, occasional users, and nonusers. Challenges to effective facilitation were apparent, with certain desired strategies being unfeasible or impeded.ConclusionsEnhancing clinicians' use of PROMs has remained challenging despite various strategies aimed at improving their capability, opportunity, and motivation. The use of PROMs may require more substantial changes than initially expected, necessitating a shift in clinicians' professional attitudes and practices. Hospitals can facilitate rather than manage clinicians' genuine use of PROMs. They must prioritize efforts to engage clinicians with PROMs for value-based outpatient care. Specific attention to their professionalization may be warranted. Tailored strategies, designed to address within-group differences in clinicians' needs and motivation, hold promise for future efforts.

Project description:New models of primary care include patient-reported outcome measures (PROMs) to promote patient-centered care. PROMs provide information on patient functional status and well-being, can be used to enhance care quality, and are proposed for use in assessing performance. Our objective was to identify a short list of candidate PROMs for use in primary care practice and to serve as a basis for performance measures (PMs). We used qualitative and quantitative methods to identify relevant patient-reported outcome (PRO) domains for use in performance measurement (PRO-PM) and their associated PROMs. We collected data from key informant groups: patients (n = 13; one-on-one and group interviews; concept saturation analysis), clinical thought leaders (n = 9; group discussions; thematic analysis), primary care practices representatives (n = 37; six focus groups; thematic analysis), and primary care payer representatives (n = 10; 12-question survey; frequencies of responses). We merged the key informant group information with findings from environmental literature scans. We conducted a targeted evidence review of measurement properties for candidate PROMs. We used a scoping review and key informant groups to identify PROM evaluation criteria, which were linked to the National Quality Forum measure evaluation criteria. We developed a de novo schema to score candidate PROMs against our criteria. We identified four PRO domains and 10 candidate PROMs: 3 for depressive symptoms, 2 for physical function, 3 for self-efficacy, 2 for ability to participate. Five PROMs met ≥ 70% of the evidence criteria for three PRO domains: PHQ-9 or PROMIS Depression (depression), PF-10 or PROMIS-PF (physical functioning), and PROMIS Self-Efficacy for Managing Treatments and Medications (self-efficacy). The PROMIS Ability to Participate in Social Roles and Activities met 68% of our criteria and might be considered for inclusion. Existing evidence and key informant data identified 5 candidate PROMs to use in primary care. These instruments can be used to develop PRO-PMs.

Project description:BackgroundThird sector organisations such as charities and community groups are using Patient Reported Outcome Measures (PROMs) at an aggregated service level to demonstrate their impact to commissioners to generate or retain funding. Despite this motivation, organisations can struggle with implementing PROMs. Previous studies have identified facilitators including organisations using an appropriate measure, co-producing the PROMs process with staff, and investing resources to support the use of measures. However, to date no studies have applied this learning to third sector organisations to evaluate whether taking an evidence-informed implementation approach improves the use of PROMs.MethodsA Community-Based Participatory Research approach was used which involved university-based researchers supporting two third sector organisations to implement PROMs. The researchers provided evidence-informed advice and training. The organisations were responsible for implementing PROMs. The researchers evaluated implementation through a mixed methods approach including five key informant interviews, four evaluation groups and analysis of collected PROMs data (n = 313).ResultsBoth third sector organisations faced considerable constraints in incorporating known facilitators and addressing barriers. The organisations involved staff in choosing an acceptable measure. However, competing priorities including external pressures to use specific PROMs, busy workloads and staff opinions created challenges to using measures. Investment of time and energy into developing an outcomes-based organisational culture was key to enable the prioritisation of PROMs. For example, discussing PROMs in supervision so that they were viewed as part of people's job roles. Organisations found that implementation took several years and was disrupted by other pressures.ConclusionsWhilst organisations were motivated to implement PROMs to obtain or retain funding, they faced considerable practical and ideological challenges. Consequently, some stakeholders felt that alternative methods to measuring impact could potentially be more feasible than PROMs.

Project description:Background and objectivesPatient-reported outcome measures invite patients to self-report aspects of their quality of life and have been reported to enhance communication with clinicians. We aimed to examine how routine use of patient-reported outcome measures in in-center hemodialysis units influenced patient-clinician communication.Design, setting, participants, & measurementsA concurrent, longitudinal, mixed-methods approach was used. We used data from a cluster randomized controlled trial of 17 hemodialysis units in northern Alberta that introduced a patient-reported outcome measures intervention. Patient-clinician communication was assessed using a modified Communication Assessment Tool. Using interpretive description, we explored patients' and nurses' perceptions of communication pertaining to routine patient-reported outcome measure use. Through purposeful sampling, we interviewed ten patients and eight nurses and conducted six observations in the dialysis units, which were documented in field notes. We reviewed 779 patient responses to open-ended survey questions from randomized controlled trial data. Qualitative data were thematically analyzed.ResultsOverall, patient-reported outcome measure use did not substantively improve patient-clinician communication. There was a small positive change in mean total Communication Assessment Tool scores (range, 1-5) from baseline to 12 months in patient-reported outcome measure use units (0.25) but little difference from control group units that did not use patient-reported outcome measures (0.21). The qualitative findings provide in-depth insights into why patient-reported outcome measure use did not improve patient-clinician communication. The purpose of patient-reported outcome measure use was not always understood by patients and clinicians; patient-reported outcome measures were not implemented as originally intended in the trial, despite clinician training; there were challenges using patient-reported outcome measures as a means to communicate; and patient-reported outcome measure use was perceived to have limited value.ConclusionsWhile patient-reported outcome measures use did not improve patient-clinician communication, qualitative data suggest implementation challenges, including limited clarity of purpose and perceived limited value, that may have limited the effectiveness of the intervention.

Project description:BackgroundQualitative evidence suggests patient-reported outcome (PRO) information is frequently absent from clinical trial protocols, potentially leading to inconsistent PRO data collection and risking bias. Direct evidence regarding PRO trial protocol content is lacking. The aim of this study was to systematically evaluate the PRO-specific content of UK National Institute for Health Research (NIHR) Health Technology Assessment (HTA) programme trial protocols.Methods and findingsWe conducted an electronic search of the NIHR HTA programme database (inception to August 2013) for protocols describing a randomised controlled trial including a primary/secondary PRO. Two investigators independently reviewed the content of each protocol, using a specially constructed PRO-specific protocol checklist, alongside the 'Standard Protocol Items: Recommendations for Interventional Trials' (SPIRIT) checklist. Disagreements were resolved through discussion with a third investigator. 75 trial protocols were included in the analysis. Protocols included a mean of 32/51 (63%) SPIRIT recommendations (range 16-41, SD 5.62) and 11/33 (33%) PRO-specific items (range 4-18, SD 3.56). Over half (61%) of the PRO items were incomplete. Protocols containing a primary PRO included slightly more PRO checklist items (mean 14/33 (43%)). PRO protocol content was not associated with general protocol completeness; thus, protocols judged as relatively 'complete' using SPIRIT were still likely to have omitted a large proportion of PRO checklist items.ConclusionsThe PRO components of HTA clinical trial protocols require improvement. Information on the PRO rationale/hypothesis, data collection methods, training and management was often absent. This low compliance is unsurprising; evidence shows existing PRO guidance for protocol developers remains difficult to access and lacks consistency. Study findings suggest there are a number of PRO protocol checklist items that are not fully addressed by the current SPIRIT statement. We therefore advocate the development of consensus-based supplementary guidelines, aimed at improving the completeness and quality of PRO content in clinical trial protocols.

Project description:BackgroundThe Do-Live-Well (DLW) framework is an occupation-focused health promotion approach. Occupational therapists (OTs) have been interested in training opportunities regarding this framework. Traditionally, in-person continuing educational interventions are the main way that OTs obtain knowledge, but web-based learning has become popular among health care professionals. However, its effectiveness and learners' experience in web-based learning have not been well-studied in occupational therapy education.ObjectiveThis study aims to evaluate the effectiveness of the web-based and in-person educational DLW workshops for Canadian OTs and to understand their experiences in both workshop types.MethodsAn explanatory sequential mixed methods study design was used, where quantitative data were collected first, then qualitative data were used to explain the quantitative findings. A quasi-experimental design and interpretative description methodology were used in the quantitative and qualitative phases, respectively.ResultsQuantitative results were as follows: a total of 43 OTs completed pre-, post-, and follow-up evaluations (in-person group: 21/43, 49%; web-based group: 22/43, 51%). Practice settings of the participants varied, including geriatric, hospital, long-term, mental health, pediatric, and primary settings. The primary outcome was as follows: there were no statistically significant differences in knowledge changes at the 3 time points (P=.57 to P=.99) between the groups. In the web-based group, the knowledge scores at follow-up were lower compared with the posttest results, meaning that knowledge gain was reduced over time (P=.001). The secondary outcomes were as follows: there were statistically significant differences between the groups in factors influencing DLW adoption at posttest (P=.001) and in satisfaction with the workshop (P<.001) at posttest in favor of the in-person group. Qualitative results were as follows: a total of 18 OTs (9/18, 50% from each group) participated in an individual interview. Five themes were identified regarding learners' workshop experiences: relevance to their practices and interests may improve learning, a familiar learning environment may facilitate learning, synchronous in-person interaction is valuable in the learning process, ease of access to learning should be considered, and flexibility in web-based learning can be both beneficial and challenging.ConclusionsThe quantitative results of this study reported no difference in knowledge acquisition between the in-person and web-based groups, indicating that web-based education is as effective as in-person workshops. However, participants' satisfaction with the workshop was statistically significantly higher for the in-person workshop. The qualitative findings described the participants' perceived benefits and challenges of each educational format. The participants in both the web-based and in-person workshop groups valued in-person interactions in learning, but the participants in the web-based workshop group expressed web-based learning lacked in-person-like interactions. Thus, adding synchronous in-person interactions to web-based learning may improve learners' educational experiences in web-based occupational therapy and continuing education.

Project description:BACKGROUND:Measures that reflect patients' assessment of their health are of increasing importance as outcome measures in randomised controlled trials. The methodological approach used in the pre-validation development of new instruments (item generation, item reduction and question formatting) should be robust and transparent. The totality of the content of existing PRO instruments for a specific condition provides a valuable resource (pool of items) that can be utilised to develop new instruments. Such 'top down' approaches are common, but the explicit pre-validation methods are often poorly reported. This paper presents a systematic and generalisable 5-step pre-validation PRO instrument methodology. METHODS:The method is illustrated using the example of the Aberdeen Glaucoma Questionnaire (AGQ). The five steps are: 1) Generation of a pool of items; 2) Item de-duplication (three phases); 3) Item reduction (two phases); 4) Assessment of the remaining items' content coverage against a pre-existing theoretical framework appropriate to the objectives of the instrument and the target population (e.g. ICF); and 5) qualitative exploration of the target populations' views of the new instrument and the items it contains. RESULTS:The AGQ 'item pool' contained 725 items. Three de-duplication phases resulted in reduction of 91, 225 and 48 items respectively. The item reduction phases discarded 70 items and 208 items respectively. The draft AGQ contained 83 items with good content coverage. The qualitative exploration ('think aloud' study) resulted in removal of a further 15 items and refinement to the wording of others. The resultant draft AGQ contained 68 items. CONCLUSIONS:This study presents a novel methodology for developing a PRO instrument, based on three sources: literature reporting what is important to patient; theoretically coherent framework; and patients' experience of completing the instrument. By systematically accounting for all items dropped after the item generation phase, our method ensures that the AGQ is developed in a transparent, replicable manner and is fit for validation. We recommend this method to enhance the likelihood that new PRO instruments will be appropriate to the research context in which they are used, acceptable to research participants and likely to generate valid data.

Project description:BackgroundBarriers experienced by patients influence the uptake of colorectal cancer (CRC) screening. Prior research has quantified how often patients encounter these challenges but has generally not revealed their complex perspective and experience with barriers.PurposeThis mixed-methods study was conducted to understand current perspectives on CRC screening.MethodsA two-part, mixed-methods study was conducted of primary care patients recruited from Virginia Ambulatory Care Outcomes Research Network practices. First, in June-July 2005 a survey was mailed to 660 patients aged 50-75 years posing an open-ended question about "the most important barrier" to CRC screening. Second, beginning in October 2005, seven gender- and largely race-specific focus groups involving 40 patients aged 45-75 years were conducted. Beginning in October 2005, survey verbatim responses were coded and quantitatively analyzed and focus group transcripts were qualitatively analyzed.ResultsResponses to the open-ended survey question, answered by 74% of respondents, identified fear and the bowel preparation as the most important barriers to screening. Only 1.6% of responses cited the absence of physician advice. Focus group participants cited similar issues and other previously reported barriers, but their remarks exposed the intricacies of complex barriers, such as fear, lack of information, time, the role of physicians, and access to care. Participants also cited barriers that have little documentation in the literature, such as low self-worth, "para-sexual" sensitivities, fatalism, negative past experiences with testing, and skepticism about the financial motivation behind screening recommendations.ConclusionsMixed-methods analysis helps to disaggregate the complex nuances that influence patient behavior. In the present study, patients explained the web of influences on knowledge, motivation, and ability to undergo CRC screening, which clinicians and policymakers should consider in designing interventions to increase the level of screening.

Project description:IntroductionPatient-reported outcome measures (PROMs) provide an opportunity for meaningful patient engagement and shared decision-making. The objective of this research programme is to improve health outcomes for paediatric solid organ transplant patients by implementing PROMs into clinical care. The current study aims to create Voxe, a paediatric user-centred electronic PROM platform, by engaging patients and healthcare providers throughout the design and development process.Methods and analysisThe creation of Voxe will occur over two phases that build on previous research. The user interface design phase employs a 'user-centric' approach to identify end-users' needs and iteratively refine the look and layout of Voxe to meet these needs. Transplant recipients, aged 10-17, and healthcare providers will participate in three rounds of testing (24 participants total). Participants will: (1) complete task-based activities (outcomes-effectiveness and efficiency), (2) complete questionnaires (outcome-satisfaction) and (3) participate in a semi-structured interview. The following phase involves software development and Voxe usability testing. Transplant recipients, aged 8-17, and healthcare providers will participate in four rounds of iterative testing (24-40 participants total). The think-aloud technique will be employed, and participants will describe their thoughts and feelings while interacting with a Voxe prototype. Participants will: (1) log into Voxe and complete tasks (outcomes-time on task, successful task completion, frequency of critical and non-critical errors and error-free rate), (2) complete questionnaires (outcome-satisfaction) and (3) participate in a semi-structured interview. Findings will result in the creation and launch of a user-centred electronic PROM platform.Ethics and disseminationResearch ethics board approval has been provided by The Hospital for Sick Children. This research is critical to answering methodological and operational questions to inform Voxe implementation in paediatric clinical settings and facilitate PROM data collection. Future investigations will include an implementation-effectiveness evaluation.

Dataset Information

Improving the patient-reported outcome sections of clinical trial protocols: a mixed methods evaluation of educational workshops.

Introduction

Method

Results

Conclusion

Publications

Improving the patient-reported outcome sections of clinical trial protocols: a mixed methods evaluation of educational workshops.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets