Dataset Information

Defining decision thresholds for judgments on health benefits and harms using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Evidence to Decision (EtD) frameworks: a protocol for a randomised methodological study (GRADE-THRESHOLD).

ABSTRACT:

Introduction

The Grading of Recommendations Assessment, Development and Evaluation (GRADE) and similar Evidence to Decision (EtD) frameworks require its users to judge how substantial the effects of interventions are on desirable and undesirable people-important health outcomes. However, decision thresholds (DTs) that could help understand the magnitude of intervention effects and serve as reference for interpretation of findings are not yet available.The objective of this study is an approach to derive and use DTs for EtD judgments about the magnitude of health benefits and harms. We hypothesise that approximate DTs could have the ability to discriminate between the existing four categories of EtD judgments (Trivial, Small, Moderate, Large), support panels of decision-makers in their work, and promote consistency and transparency in judgments.

Methods and analysis

We will conduct a methodological randomised controlled trial to collect the data that allow deriving the DTs. We will invite clinicians, epidemiologists, decision scientists, health research methodologists, experts in Health Technology Assessment (HTA), members of guideline development groups and the public to participate in the trial. Then, we will investigate the validity of our DTs by measuring the agreement between judgments that were made in the past by guideline panels and the judgments that our DTs approach would suggest if applied on the same guideline data.

Ethics and dissemination

The Hamilton Integrated Research Ethics Board reviewed this study as a quality improvement study and determined that it requires no further consent. Survey participants will be required to read a consent statement in order to participate in this study at the beginning of the trial. This statement reads: You are being invited to participate in a research project which aims to identify indicative DTs that could assist users of the GRADE EtD frameworks in making judgments. Your input will be used in determining these indicative thresholds. By completing this survey, you provide consent that the anonymised data collected will be used for the research study and to be summarised in aggregate in publication and electronic tools.

Protocol registration number

NCT05237635.

SUBMITTER: Morgano GP

PROVIDER: S-EPMC8915269 | biostudies-literature | 2022 Mar

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Defining decision thresholds for judgments on health benefits and harms using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Evidence to Decision (EtD) frameworks: a protocol for a randomised methodological study (GRADE-THRESHOLD).

Morgano Gian Paolo GP Mbuagbaw Lawrence L Santesso Nancy N Xie Feng F Brozek Jan L JL Siebert Uwe U Bognanni Antonio A Wiercioch Wojtek W Piggott Thomas T Darzi Andrea J AJ Akl Elie A EA Verstijnen Ilse M IM Parmelli Elena E Saz-Parkinson Zuleika Z Alonso-Coello Pablo P Schünemann Holger J HJ

BMJ open 20220310 3

<h4>Introduction</h4>The Grading of Recommendations Assessment, Development and Evaluation (GRADE) and similar Evidence to Decision (EtD) frameworks require its users to judge how substantial the effects of interventions are on desirable and undesirable people-important health outcomes. However, decision thresholds (DTs) that could help understand the magnitude of intervention effects and serve as reference for interpretation of findings are not yet available.The objective of this study is an ap ...[more]

PMID: 35273045

Similar Datasets

Project description:ObjectiveTo describe a framework for people making and using evidence-informed health system and public health recommendations and decisions.BackgroundWe developed the GRADE Evidence to Decision (EtD) framework for health system and public health decisions as part of the DECIDE project, in which we simultaneously developed frameworks for these and other types of healthcare decisions, including clinical recommendations, coverage decisions and decisions about diagnostic tests.Developing the frameworkBuilding on GRADE EtD tables, we used an iterative approach, including brainstorming, consultation of the literature and with stakeholders, and an international survey of policy-makers. We applied the framework to diverse examples, conducted workshops and user testing with health system and public health guideline developers and policy-makers, and observed and tested its use in real-life guideline panels.FindingsAll the GRADE EtD frameworks share the same basic structure, including sections for formulating the question, making an assessment and drawing conclusions. Criteria listed in the assessment section of the health system and public health framework cover the important factors for making these types of decisions; in addition to the effects and economic impact of an option, the priority of the problem, the impact of the option on equity, and its acceptability and feasibility are important considerations that can inform both whether and how to implement an option. Because health system and public health interventions are often complex, detailed implementation considerations should be made when making a decision. The certainty of the evidence is often low or very low, but decision-makers must still act. Monitoring and evaluation are therefore often important considerations for these types of decisions. We illustrate the different components of the EtD framework for health system and public health decisions by presenting their application in a framework adapted from a real-life guideline.DiscussionThis framework provides a structured and transparent approach to support policy-making informed by the best available research evidence, while making the basis for decisions accessible to those whom they will affect. The health system and public health EtD framework can also be used to facilitate dissemination of recommendations and enable decision-makers to adopt, and adapt, recommendations or decisions.

Project description:BackgroundAlthough the potential impact of food fortification to improve the micronutrient status of populations has been demonstrated beyond a doubt, it is constrained in practice by critical gaps in program design and implementation. These are partly linked to suboptimal decision making.ObjectivesWe aimed to demonstrate how the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) Evidence to Decision (EtD) framework for health system and public health decisions can be applied to formulate recommendations and make decisions in national food fortification programming.MethodsFollowing a program impact pathway, we reviewed the literature to define the key decision types and identify the corresponding decision makers necessary for designing and implementing effective food fortification programs. We then applied the GRADE EtD framework to the Nigerian fortification program to illustrate how evidence-informed assessments and conclusions can be made.ResultsFortification program decisions were classified into 5 types: 1) program initiation; 2) program design; 3) program delivery; 4) program impact; and 5) program continuation. Policymakers, food processors, and (in cases dependent on or considering external funding) development partners are the main decision makers in a fortification program, whereas technical partners play important roles in translating evidence into contextualized recommendations. The availability and certainty of evidence for fortification programs are often low (e.g., quality and coverage data are not routinely collected and there are challenges evaluating impact in such population-based programs using randomized controlled trials) yet decisions must still be made, underscoring the importance of using available evidence. Furthermore, when making program initiation and continuation decisions, coordination with overlapping micronutrient interventions is needed where they coexist.ConclusionsThis framework is a practical tool to strengthen decision-making processes in fortification programs. Using evidence in a systematic and transparent way for decision making can improve fortification program design, delivery, and ultimately health impacts.

Project description:Context and objectiveStandards for clinical practice guidelines require explicit statements regarding how values and preferences influence recommendations. However, no cancer screening guideline has addressed the key question of what magnitude of benefit people require to undergo screening, given its harms and burdens. This article describes the development of a new method for guideline developers to address this key question in the absence of high-quality evidence from published literature.Summary of methodThe new method was developed and applied in the context of a recent BMJ Rapid Recommendation clinical practice guideline for colorectal cancer (CRC) screening. First, we presented the guideline panel with harms and burdens (derived from a systematic review) associated with the CRC screening tests under consideration. Second, each panel member completed surveys documenting their views of expected benefits on CRC incidence and mortality that people would require to accept the harms and burdens of screening. Third, the panel discussed results of the surveys and agreed on thresholds for benefits at which the majority of people would choose screening. During these three steps, the panel had no access to the actual benefits of the screening tests. In step four, the panel was presented with screening test benefits derived from a systematic review of clinical trials and microsimulation modelling. The thresholds derived through steps one to three were applied to these benefits, and directly informed the panel's recommendations.ConclusionWe present the development and application of a new, four-step method enabling incorporation of explicit and transparent judgements of values and preferences in a screening guideline. Guideline panels should establish their view regarding the magnitude of required benefit, given burdens and harms, before they review screening benefits and make their recommendations accordingly. Making informed screening decisions requires transparency in values and preferences judgements that our new method greatly facilitates.

Project description:BackgroundFaculty and students debate the fairness and accuracy of medical student clerkship grades. Group decision-making is a potential strategy to improve grading.ObjectiveTo explore how one school's grading committee members integrate assessment data to inform grade decisions and to identify the committees' benefits and challenges.DesignThis qualitative study used semi-structured interviews with grading committee chairs and members conducted between November 2017 and March 2018.ParticipantsParticipants included the eight core clerkship directors, who chaired their grading committees. We randomly selected other committee members to invite, for a maximum of three interviews per clerkship.ApproachInterviews were recorded, transcribed, and analyzed using inductive content analysis.Key resultsWe interviewed 17 committee members. Within and across specialties, committee members had distinct approaches to prioritizing and synthesizing assessment data. Participants expressed concerns about the quality of assessments, necessitating careful scrutiny of language, assessor identity, and other contextual factors. Committee members were concerned about how unconscious bias might impact assessors, but they felt minimally impacted at the committee level. When committee members knew students personally, they felt tension about how to use the information appropriately. Participants described high agreement within their committees; debate was more common when site directors reviewed students' files from other sites prior to meeting. Participants reported multiple committee benefits including faculty development and fulfillment, as well as improved grading consistency, fairness, and transparency. Groupthink and a passive approach to bias emerged as the two main threats to optimal group decision-making.ConclusionsGrading committee members view their practices as advantageous over individual grading, but they feel limited in their ability to address grading fairness and accuracy. Recommendations and support may help committees broaden their scope to address these aspirations.

Project description:BackgroundDecision thresholds play important role in medical decision-making. Individual decision-making differences may be attributable to differences in subjective judgments or cognitive processes that are captured through the decision thresholds. This systematic scoping review sought to characterize the literature on non-expected utility decision thresholds in medical decision-making by identifying commonly used theoretical paradigms and contextual and subjective factors that inform decision thresholds.MethodsA structured search designed around three concepts-individual decision-maker, decision threshold, and medical decision-was conducted in MEDLINE (Ovid) and Scopus databases from inception to July 2023. ProQuest (Dissertations and Theses) database was searched to August 2023. The protocol, developed a priori, was registered on Open Science Framework and PRISMA-ScR guidelines were followed for reporting on this study. Titles and abstracts of 1,618 articles and the full texts for the 228 included articles were reviewed by two independent reviewers. 95 articles were included in the analysis. A single reviewer used a pilot-tested data collection tool to extract study and author characteristics, article type, objectives, theoretical paradigm, contextual or subjective factors, decision-maker, and type of medical decision.ResultsOf the 95 included articles, 68 identified a theoretical paradigm in their approach to decision thresholds. The most common paradigms included regret theory, hybrid theory, and dual processing theory. Contextual and subjective factors that influence decision thresholds were identified in 44 articles.ConclusionsOur scoping review is the first to systematically characterizes the available literature on decision thresholds within medical decision-making. This study offers an important characterization of the literature through the identification of the theoretical paradigms for non-expected utility decision thresholds. Moreover, this study provides insight into the various contextual and subjective factors that have been documented within the literature to influence decision thresholds, as well as these factors juxtapose theoretical paradigms.

Dataset Information

Defining decision thresholds for judgments on health benefits and harms using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Evidence to Decision (EtD) frameworks: a protocol for a randomised methodological study (GRADE-THRESHOLD).

Introduction

Methods and analysis

Ethics and dissemination

Protocol registration number

Publications

Defining decision thresholds for judgments on health benefits and harms using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Evidence to Decision (EtD) frameworks: a protocol for a randomised methodological study (GRADE-THRESHOLD).

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets