Dataset Information

Best-worst scaling methodology to evaluate constructs of the Consolidated Framework for Implementation Research: application to the implementation of pharmacogenetic testing for antidepressant therapy.

ABSTRACT:

Background

Despite the increased demand for pharmacogenetic (PGx) testing to guide antidepressant use, little is known about how to implement testing in clinical practice. Best-worst scaling (BWS) is a stated preferences technique for determining the relative importance of alternative scenarios and is increasingly being used as a healthcare assessment tool, with potential applications in implementation research. We conducted a BWS experiment to evaluate the relative importance of implementation factors for PGx testing to guide antidepressant use.

Methods

We surveyed 17 healthcare organizations that either had implemented or were in the process of implementing PGx testing for antidepressants. The survey included a BWS experiment to evaluate the relative importance of Consolidated Framework for Implementation Research (CFIR) constructs from the perspective of implementing sites.

Results

Participating sites varied on their PGx testing platform and methods for returning recommendations to providers and patients, but they were consistent in ranking several CFIR constructs as most important for implementation: patient needs/resources, leadership engagement, intervention knowledge/beliefs, evidence strength and quality, and identification of champions.

Conclusions

This study demonstrates the feasibility of using choice experiments to systematically evaluate the relative importance of implementation determinants from the perspective of implementing organizations. BWS findings can inform other organizations interested in implementing PGx testing for mental health. Further, this study demonstrates the application of BWS to PGx, the findings of which may be used by other organizations to inform implementation of PGx testing for mental health disorders.

SUBMITTER: Salloum RG

PROVIDER: S-EPMC9107643 | biostudies-literature | 2022 May

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Best-worst scaling methodology to evaluate constructs of the Consolidated Framework for Implementation Research: application to the implementation of pharmacogenetic testing for antidepressant therapy.

Salloum Ramzi G RG Bishop Jeffrey R JR Elchynski Amanda L AL Smith D Max DM Rowe Elizabeth E Blake Kathryn V KV Limdi Nita A NA Aquilante Christina L CL Bates Jill J Beitelshees Amber L AL Cipriani Amber A Duong Benjamin Q BQ Empey Philip E PE Formea Christine M CM Hicks J Kevin JK Mroz Pawel P Oslin David D Pasternak Amy L AL Petry Natasha N Ramsey Laura B LB Schlichte Allyson A Swain Sandra M SM Ward Kristen M KM Wiisanen Kristin K Skaar Todd C TC Van Driest Sara L SL Cavallari Larisa H LH Tuteja Sony S

Implementation science communications 20220514 1

<h4>Background</h4>Despite the increased demand for pharmacogenetic (PGx) testing to guide antidepressant use, little is known about how to implement testing in clinical practice. Best-worst scaling (BWS) is a stated preferences technique for determining the relative importance of alternative scenarios and is increasingly being used as a healthcare assessment tool, with potential applications in implementation research. We conducted a BWS experiment to evaluate the relative importance of impleme ...[more]

PMID: 35568931

Similar Datasets

Project description:BackgroundCommunity behavioral health clinicians, supervisors, and administrators play an essential role in implementing new psychosocial evidence-based practices (EBP) for patients receiving psychiatric care; however, little is known about these stakeholders' values and preferences for implementation strategies that support EBP use, nor how best to elicit, quantify, or segment their preferences. This study sought to quantify these stakeholders' preferences for implementation strategies and to identify segments of stakeholders with distinct preferences using a rigorous choice experiment method called best-worst scaling.MethodsA total of 240 clinicians, 74 clinical supervisors, and 29 administrators employed within clinics delivering publicly-funded behavioral health services in a large metropolitan behavioral health system participated in a best-worst scaling choice experiment. Participants evaluated 14 implementation strategies developed through extensive elicitation and pilot work within the target system. Preference weights were generated for each strategy using hierarchical Bayesian estimation. Latent class analysis identified segments of stakeholders with unique preference profiles.ResultsOn average, stakeholders preferred two strategies significantly more than all others-compensation for use of EBP per session and compensation for preparation time to use the EBP (P < .05); two strategies were preferred significantly less than all others-performance feedback via email and performance feedback via leaderboard (P < .05). However, latent class analysis identified four distinct segments of stakeholders with unique preferences: Segment 1 (n = 121, 35%) strongly preferred financial incentives over all other approaches and included more administrators; Segment 2 (n = 80, 23%) preferred technology-based strategies and was younger, on average; Segment 3 (n = 52, 15%) preferred an improved waiting room to enhance client readiness, strongly disliked any type of clinical consultation, and had the lowest participation in local EBP training initiatives; Segment 4 (n = 90, 26%) strongly preferred clinical consultation strategies and included more clinicians in substance use clinics.ConclusionsThe presence of four heterogeneous subpopulations within this large group of clinicians, supervisors, and administrators suggests optimal implementation may be achieved through targeted strategies derived via elicitation of stakeholder preferences. Best-worst scaling is a feasible and rigorous method for eliciting stakeholders' implementation preferences and identifying subpopulations with unique preferences in behavioral health settings.

Project description:Objective To investigate the importance of key characteristics relating to diagnostic testing for ovarian cancer and to understand how previous test experience influences priorities. Design Case 1 best–worst scaling embedded in an online survey. Setting Primary care diagnostic testing in England and Wales. Participants 150 women with ovaries over 40 years old living in England and Wales. Methods We used best–worst scaling, a preference-based survey method, to elicit the relative importance of 25 characteristics relating to ovarian cancer testing following a systematic review. Responses were modelled using conditional logit regression. Subgroup analysis investigated variations based on testing history. Main outcome measures Relative importance scores. Results ‘Chance of dying from ovarian cancer’ (0.380, 95% CI 0.26 to 0.49) was the most important factor to respondents, closely followed by ‘test sensitivity’ (0.308, 95% CI 0.21 to 0.40). In contrast, ‘time away from usual activities’ (−0.244, 95% CI −0.33 to −0.15) and ‘gender of healthcare provider’ (−0.243, 95% CI −0.35 to −0.14) were least important to respondents overall. Women who had previously undergone testing placed higher importance on certain characteristics including ‘openness of healthcare providers’ and ‘chance of diagnosing another condition’ at the expense of reduced emphasis on characteristics such as ‘pain and discomfort’ and ‘time away from usual activities’. Conclusions The results clearly demonstrated items at the extreme, which were most and least important to women considering ovarian cancer testing. Differences in priorities by testing history demonstrate an experience effect, whereby preferences adapt over time based on evidence and experience. Acknowledging these differences helps to identify underlying barriers and facilitators for women with no test experience as well as shortcomings of current service based on women with experience.

Project description:Epilepsy is a common, serious condition. Fortunately, seizure risk decreases with increasing seizure-free time on antiseizure medications (ASMs). Eventually, patients may consider whether to stop ASMs, which requires weighing treatment benefit versus burden. We developed a questionnaire to quantify patient preferences relevant to ASM decision-making. Respondents rated how concerning they would finding relevant items (e.g., seizure risks, side effects, cost) on a Visual Analogue Scale (VAS, 0-100) and then repeatedly chose the most and least concerning item from subsets (best-worst scaling, BWS). We pretested with neurologists, then recruited adults with epilepsy who were seizure-free at least one year. Primary outcomes were recruitment rate, and qualitative and Likert-based feedback. Secondary outcomes included VAS ratings and best-minus-worst scores. Thirty-one of 60 (52%) contacted patients completed the study. Most patients felt VAS questions were clear (28; 90%), easy to use (27; 87%), and assessed preferences well (25; 83%). Corresponding results for BWS questions were 27 (87%), 29 (97%), and 23 (77%). Physicians suggested adding a 'warmup' question showing a completed example and simplifying terminology. Patients suggested ways to clarify instructions. Cost, inconvenience of taking medication, and laboratory monitoring were the least concerning items. Cognitive side effects and a 50% seizure risk in the next year were the most concerning items. Twelve (39%) of patients made at least one 'inconsistent choice' for example ranking a higher seizure risk as lower concern compared with a lower seizure risk, though 'inconsistent choices' represented only 3% of all question blocks. Our recruitment rate was favorable, most patients agreed the survey was clear, and we describe areas for improvement. 'Inconsistent' responses may lead us to collapse seizure probability items into a single 'seizure' category. Evidence regarding how patients weigh benefits and harms may inform care and guideline development.

Project description:BackgroundThe preferences of autism stakeholders regarding the top priorities for future autism research are largely unknown.ObjectiveThis study had two objectives: First, to examine what autism stakeholders think new research investments should be and the attributes of investment that they consider important, and second, to explore the feasibility, acceptability and outcomes of two prioritization exercises among autism stakeholders regarding their priorities for future research in autism.DesignThis was a prospective stakeholder-engaged iterative study consisting of best-worst scaling (BWS) and direct prioritization exercise.Setting and participantsA national snowball sample of 219 stakeholders was included: adults with autism, caregivers, service providers and researchers.Main outcome measuresThe main outcomes measures were attributes that participants value in future research investments, and priority research investments for future research.ResultsTwo hundred and nineteen participants completed the exercises, of whom 11% were adults with autism, 58% were parents/family members, 37% were service providers and 21% were researchers. Among stakeholders, the BWS exercises were easier to understand than the direct prioritization, less frequently skipped and yielded more consistent results. The proportion of children with autism affected by the research was the most important attribute for all types of stakeholders. The top three priorities among future research investments were (1) evidence on which child, family and intervention characteristics lead to the best/worst outcomes; (2) evidence on how changes in one area of a child's life are related to changes in other areas; and (3) evidence on dietary interventions. Priorities were similar for all stakeholder types.ConclusionsThe values and priorities examined here provide a road map for investigators and funders to pursue autism research that matters to stakeholders.Patient or public contributionStakeholders completed a BWS and direct prioritization exercise to inform us about their priorities for future autism research.

Project description:Background and objectiveBest-worst scaling is a theory-driven method that can be used to prioritize objects in health. We sought to characterize all studies of best-worst scaling to prioritize objects in health, to assess trends of using best-worst scaling in prioritization over time, and to assess the relationship between a legacy measure of quality (PREFS) and a novel assessment of subjective quality and policy relevance.MethodsA systematic review identified studies published through to the end of 2021 that applied best-worst scaling to study priorities in health (PROSPERO CRD42020209745), updating a prior review published in 2016. The PubMed, EBSCOhost, Embase, Scopus, APA PsychInfo, Web of Science, and Google Scholar databases were used and were supplemented by a hand search. Data describing the application, development, design, administration/analysis, quality, and policy relevance were summarized and we tested for trends by comparing articles before and after 1 January, 2017. Multivariate statistics were then used to assess the relationships between PREFS, subjective quality, policy relevance, and other possible indicators.ResultsFrom a total of 2826 unique papers identified, 165 best-worst scaling studies were included in this review. Applications of best-worst scaling to study priorities in health have continued to grow (p < 0.01) and are now used in all regions of the world, most often to study the priorities of patients/consumers (67%). Several key trends can be observed over time: increased use of pretesting (p < 0.05); increased use of online administration (p < 0.01), and decreased use of paper self-administered surveys (p = 0.02); increased use of heterogeneity analysis (p = 0.02); an increase in having a clearly stated purpose (p < 0.01); and a decrease in comparing respondents to non-respondents (p = 0.01). The average sample size has more than doubled, from 228 to 472 respondents, but formal sample size justifications remain low (5.3%) and unchanged over time (p = 0.68). While the average PREFS score remained unchanged at 3.1/5, both subjective quality and policy relevance trended up, but changes were not statistically significant (p = 0.06 and p = 0.13). Most of the variation in subjective quality was driven by PREFS (R2 = 0.42), but it was also positively assosciated with policy relevance, heterogeneity analysis, and using a balanced incomplete block design, and was negatively associated with not using developmental methods and an increasing sample size.ConclusionsUsing best-worst scaling to prioritize objects is now commonly used around the world to assess the priorities of patients and other stakeholders in health. Best practices are clearly emerging for best-worst scaling. Although legacy measures (PREFS) to measure study quality are reasonable, there may need to be new tools to assess both study quality and policy relevance.

Dataset Information

Best-worst scaling methodology to evaluate constructs of the Consolidated Framework for Implementation Research: application to the implementation of pharmacogenetic testing for antidepressant therapy.

Background

Methods

Results

Conclusions

Publications

Best-worst scaling methodology to evaluate constructs of the Consolidated Framework for Implementation Research: application to the implementation of pharmacogenetic testing for antidepressant therapy.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets