Assessing the Preferences for Criteria in Multi-Criteria Decision Analysis in Treatments for Rare Diseases.
ABSTRACT: Background: Increasingly, multi-criteria decision analysis has gained importance as a method by which to assess the value of orphan drugs. However, very little attention has been given to the weight (relative preferences) of the individual criteria used in a framework. Aims: This study sought to gain an understanding of the preferential weights that should be allocated in a multi-criteria decision analysis framework for orphan drugs, from a multi-stakeholder perspective. Method: Using key MCDA criteria for orphan drugs reported in the literature, we developed an interactive web-based survey tool to capture preferences for different criteria from a general stakeholder sample who were requested to assign weights from a reimbursement perspective. Each criterion could be assigned a weight on a sliding scale from 0 to 100% as long as the sum of all the criteria was 100%. We subsequently used the interactive tool with an expert focus group, followed up with a group discussion regarding each criterion and their perspectives on the weight that each criterion should be allocated when assessing an orphan drug. The expert focus group participants were then able to adjust their weights, if the group discussion had changed their perspectives. Results: The interactive tool was completed by 120 general stakeholder sample from a wide range of countries and professional backgrounds and an expert focus group of ten members. The results showed the differences in perspectives on the importance of criteria. Both groups considered Treatment efficacy to be the most important criterion. The general stakeholder sample weighted Treatment safety at 12.03% compared to the expert focus group's average of 20%. The results also demonstrated the value of the group discussion, which provided additional insights into the perspectives on the importance of criteria in assessing orphan drugs. Conclusion: This study aimed to contribute to the important aspect of preferences for different criteria in MCDA. This study sheds light on the important aspect of the preferences of the different criteria. All respondents agreed on the relative importance of Treatment efficacy and Treatment safety, criteria that are captured in conventional cost-effectiveness studies, but they also expressed the view that in addition to those, several disease-related and drug-related criteria should be included in MCDA frameworks for assessing orphan drugs.
Project description:Well- or moderately differentiated gastroenteropancreatic neuroendocrine tumors (GEP-NETs) are often slow-growing, and some patients with unresectable, asymptomatic, non-functioning tumors may face the choice between watchful waiting (WW), or somatostatin analogues (SSA) to delay progression. We developed a comprehensive multi-criteria decision analysis (MCDA) framework to help patients and physicians clarify their values and preferences, consider each decision criterion, and support communication and shared decision-making.The framework was adapted from a generic MCDA framework (EVIDEM) with patient and clinician input. During a workshop, patients and clinicians expressed their individual values and preferences (criteria weights) and, on the basis of two scenarios (treatment vs WW; SSA-1 [lanreotide] vs SSA-2 [octreotide]) with evidence from a literature review, expressed how consideration of each criterion would impact their decision in favor of either option (score), and shared their knowledge and insights verbally and in writing.The framework included benefit-risk criteria and modulating factors, such as disease severity, quality of evidence, costs, and constraints. Overall and progression-free survival being most important, criteria weights ranged widely, highlighting variations in individual values and the need to share them. Scoring and considering each criterion prompted a rich exchange of perspectives and uncovered individual assumptions and interpretations. At the group level, type of benefit, disease severity, effectiveness, and quality of evidence favored treatment; cost aspects favored WW (scenario 1). For scenario 2, most criteria did not favor either option.Patients and clinicians consider many aspects in decision-making. The MCDA framework provided a common interpretive frame to structure this complexity, support individual reflection, and share perspectives.Ipsen Pharma.
Project description:The objective of the study was to reveal through pragmatic MCDA (EVIDEM) the contribution of a broad range of criteria to the value of the orphan drug lenvatinib for radioiodine refractory differentiated thyroid cancer (RR-DTC) in country-specific contexts.The study was designed to enable comprehensive appraisal (12 quantitative, 7 qualitative criteria) in the current disease context (watchful waiting, sorafenib) of France, Italy and Spain. Data on the value of lenvatinib was collected from diverse stakeholders during country-specific panels and included: criteria weights (individual and social values); performance scores (judgments on evidence-collected through MCDA systematic review); qualitative impacts of contextual criteria; and verbal and written insights structured by criteria. The value contribution of each criterion was calculated and uncertainty explored.Comparative effectiveness, Quality of evidence (Spain and Italy) and Disease severity (France) received the greatest weights. Four criteria contributed most to the value of lenvatinib, reflecting its superior Comparative effectiveness (16-22% of value), the severity of RR-DTC (16-22%), significant unmet needs (14-21%) and robust evidence (14-20%). Contributions varied by comparator, country and individuals, highlighting the importance of context and consultation. Results were reproducible at the group level. Impacts of contextual criteria varied across countries reflecting different health systems and cultural backgrounds. The MCDA process promoted sharing stakeholders' knowledge on lenvatinib and insights on context.The value of lenvatinib was consistently positive across diverse therapeutic contexts. MCDA identified the aspects contributing most to value, revealed rich contextual insights, and helped participants express and explicitly tackle ethical trade-offs inherent to balanced appraisal and decisionmaking.
Project description:A Multi Criteria Decision Analysis (MCDA) technique was adopted to reveal the preferences of the Appraisal Body of the Polish HTA agency towards orphan drugs (OMPs).There were 34 positive and 23 negative HTA recommendations out of 54 distinctive drug-indication pairs. The MCDA matrix consisted of 13 criteria, seven of which made the most impact on the HTA process. Appraisal of clinical evidence, cost of therapy, and safety considerations were the main contributors to the HTA guidance, whilst advancement of technology and manufacturing costs made the least impact.MCDA can be regarded as a valuable tool for revealing decision makers' preferences in the healthcare sector. Given that only roughly half of all criteria included in the MCDA matrix were deemed to make an impact on the HTA process, there is certainly some room for improvement with respect to the adaptation of a new approach towards the value assessment of OMPs in Poland.
Project description:Healthcare decision makers need to make trade-offs between different elements of value of new treatments. Multicriteria decision analysis (MCDA) provides a framework that can help decision makers to understand stakeholders' preferences and be explicit about the trade-offs that are being made.The objective of this study was to use MCDA to obtain preferences and views on decision criteria across three stakeholder groups (patients, clinicians and payers) in Italy and to use these to assess the performance of obinutuzumab for rituximab-refractory indolent non-Hodgkin lymphoma (iNHL).We used EVIDEM V3.0, an MCDA framework, and collected participants' preferences via an online survey and structured meetings.Patients and clinicians expressed a preference for interventions targeting severe conditions. Payers expressed preference for treatments targeting areas with an unmet need, which are cheaper than the comparator, and with high-quality evidence. Obinutuzumab in combination with bendamustine, compared with bendamustine alone, received high positive scores for the criteria 'disease severity' and 'type of therapeutic benefit' by all three groups, and negative scores on the economic-related criteria, according to all stakeholder groups.MCDA can be used to elicit the views of different stakeholder groups and has the potential to structure and inform reimbursement decisions.
Project description:BACKGROUND: Consistent healthcare decision making requires systematic consideration of decision criteria and evidence available to inform them. This can be tackled by combining multicriteria decision analysis (MCDA) and Health Technology Assessment (HTA). The objective of this study was to field-test a decision support framework (EVIDEM), explore its utility to a drug advisory committee and test its reliability over time. METHODS: Tramadol for chronic non-cancer pain was selected by the health plan as a case study relevant to their context. Based on extensive literature review, a by-criterion HTA report was developed to provide synthesized evidence for each criterion of the framework (14 criteria for the MCDA Core Model and 6 qualitative criteria for the Contextual Tool). During workshop sessions, committee members tested the framework in three steps by assigning: 1) weights to each criterion of the MCDA Core Model representing individual perspective; 2) scores for tramadol for each criterion of the MCDA Core Model using synthesized data; and 3) qualitative impacts of criteria of the Contextual Tool on the appraisal. Utility and reliability of the approach were explored through discussion, survey and test-retest. Agreement between test and retest data was analyzed by calculating intra-rater correlation coefficients (ICCs) for weights, scores and MCDA value estimates. RESULTS: The framework was found useful by the drug advisory committee in supporting systematic consideration of a broad range of criteria to promote a consistent approach to appraising healthcare interventions. Directly integrated in the framework as a "by-criterion" HTA report, synthesized evidence for each criterion facilitated its consideration, although this was sometimes limited by lack of relevant data. Test-retest analysis showed fair to good consistency of weights, scores and MCDA value estimates at the individual level (ICC ranging from 0.676 to 0.698), thus lending some support for the reliability of the approach. Overall, committee members endorsed the inclusion of most framework criteria and revealed important areas of discussion, clarification and adaptation of the framework to the needs of the committee. CONCLUSIONS: By promoting systematic consideration of all decision criteria and the underlying evidence, the framework allows a consistent approach to appraising healthcare interventions. Further testing and validation are needed to advance MCDA approaches in healthcare decisionmaking.
Project description:European League Against Rheumatism and are jointly supporting multiphase development of systemic lupus erythematosus (SLE) classification criteria based on weighted criteria and a continuous probability scale. Prior steps included item generation, item reduction and hierarchical organisation of candidate criteria using an evidence-based approach. Our objectives were to determine relative weights using multicriteria decision analysis (MCDA) and to set a provisional threshold score for SLE classification. An SLE Expert Panel (8 European, 9 North American) submitted 164 real, unique cases with a wide range of SLE probability in a standardised format. Using the candidate criteria, experts scored and rank-ordered 20 representative cases. At an in-person meeting, experts reviewed inter-rater reliability of scoring, further refined criteria definitions and participated in an MCDA exercise. Based on expert consensus decisions on pairwise comparisons of criteria, 1000minds software calculated criteria weights and rank-ordered the remaining 144 cases based on their additive scores. The score of the lowest-ranked case for which complete expert consensus was achieved defined the provisional threshold classification score. Inter-rater reliability of scoring cases with the candidate criteria was good. MCDA involved 74 pairwise decisions and was repeated for the arthritis and mucocutaneous domains when the initial ranking of some cases did not match expert opinion. After criteria weights and additive scores were recalculated once, experts reached consensus for SLE classification for all cases scoring>83. Using an iterative process, the candidate criteria definitions were refined, preliminary weights were calculated and a provisional threshold score for SLE classification was determined.
Project description:River rehabilitation aims at alleviating negative effects of human impacts such as loss of biodiversity and reduction of ecosystem services. Such interventions entail difficult trade-offs between different ecological and often socio-economic objectives. Multi-Criteria Decision Analysis (MCDA) is a very suitable approach that helps assessing the current ecological state and prioritizing river rehabilitation measures in a standardized way, based on stakeholder or expert preferences. Applications of MCDA in river rehabilitation projects are often simplified, i.e. using a limited number of objectives and indicators, assuming linear value functions, aggregating individual indicator assessments additively, and/or assuming risk neutrality of experts. Here, we demonstrate an implementation of MCDA expert preference assessments to river rehabilitation and provide ample material for other applications. To test whether the above simplifications reflect common expert opinion, we carried out very detailed interviews with five river ecologists and a hydraulic engineer. We defined essential objectives and measurable quality indicators (attributes), elicited the experts´ preferences for objectives on a standardized scale (value functions) and their risk attitude, and identified suitable aggregation methods. The experts recommended an extensive objectives hierarchy including between 54 and 93 essential objectives and between 37 to 61 essential attributes. For 81% of these, they defined non-linear value functions and in 76% recommended multiplicative aggregation. The experts were risk averse or risk prone (but never risk neutral), depending on the current ecological state of the river, and the experts´ personal importance of objectives. We conclude that the four commonly applied simplifications clearly do not reflect the opinion of river rehabilitation experts. The optimal level of model complexity, however, remains highly case-study specific depending on data and resource availability, the context, and the complexity of the decision problem.
Project description:OBJECTIVE:To use Multi-Criteria Decision Analysis (MCDA) to determine weights for eleven criteria in order to prioritize COVID-19 non-critical patients for admission to hospital in healthcare settings with limited resources. METHODS:The MCDA was applied in two main steps: specification of criteria for prioritizing COVID-19 patients (and levels within each criterion); and determination of weights for the criteria based on experts' knowledge and experience in managing COVID-19 patients, via an online survey. Criteria were selected based on available COVID-19 evidence with a focus on low- and middle-income countries (LMICs). RESULTS:The most important criteria (mean weights, summing to 100%) are: PaO2 (16.3%); peripheral O2 saturation (15.9%); chest X-ray (14.1%); Modified Early Warning Score-MEWS (11.4%); respiratory rate (9.5%); comorbidities (6.5%); living with vulnerable people (6.4%); body mass index (5.6%); duration of symptoms before hospital evaluation (5.4%); CRP (5.1%); and age (3.8%). CONCLUSIONS:At the beginning of a new pandemic, when evidence for disease predictors is limited or unavailable and effective national contingency plans are difficult to establish, the MCDA prioritization model could play a pivotal role in improving the response of health systems.
Project description:Evaluation of integrated care programmes for individuals with multi-morbidity requires a broader evaluation framework and a broader definition of added value than is common in cost-utility analysis. This is possible through the use of Multi-Criteria Decision Analysis (MCDA).This paper presents the seven steps of an MCDA to evaluate 17 different integrated care programmes for individuals with multi-morbidity in 8 European countries participating in the 4-year, EU-funded SELFIE project. In step one, qualitative research was undertaken to better understand the decision-context of these programmes. The programmes faced decisions related to their sustainability in terms of reimbursement, continuation, extension, and/or wider implementation. In step two, a uniform set of decision criteria was defined in terms of outcomes measured across the 17 programmes: physical functioning, psychological well-being, social relationships and participation, enjoyment of life, resilience, person-centeredness, continuity of care, and total health and social care costs. These were supplemented by programme-type specific outcomes. Step three presents the quasi-experimental studies designed to measure the performance of the programmes on the decision criteria. Step four gives details of the methods (Discrete Choice Experiment, Swing Weighting) to determine the relative importance of the decision criteria among five stakeholder groups per country. An example in step five illustrates the value-based method of MCDA by which the performance of the programmes on each decision criterion is combined with the weight of the respective criterion to derive an overall value score. Step six describes how we deal with uncertainty and introduces the Conditional Multi-Attribute Acceptability Curve. Step seven addresses the interpretation of results in stakeholder workshops.By discussing our solutions to the challenges involved in creating a uniform MCDA approach for the evaluation of different programmes, this paper provides guidance to future evaluations and stimulates debate on how to evaluate integrated care for multi-morbidity.
Project description:OBJECTIVE:This study is aimed toward establishing a decision-making model with multiple criteria for appraisal and reimbursement to compare the attitudes of different stakeholders toward various dimensions and criteria and to evaluate the five targeted therapies (bevacizumab, cetuximab, panitumumab, aflibercept, and regorafenib) for metastatic colorectal cancer. METHOD:This study is a multi-criteria decision analysis (MCDA) using a model that includes three dimensions and nine criteria. Both the overall and individual scores of the respective targeted therapies in different dimensions and criteria were calculated. A sensitivity analysis was carried out in order to evaluate the robustness of the research results. An interview-based questionnaire survey was applied to obtain the performance information for the targeted therapies and the weights of the dimensions and criteria. RESULTS:Overall, the clinical dimension had the highest weight, followed by the economic dimension, and finally, the social dimension. In the clinical dimension, the "comparative efficacy" criterion had the highest weight; in the economic dimension, the "cost-effectiveness" criterion" was given the greatest importance; in the social dimension, the "social concern and patient needs" criterion was given more emphasis. The overall values ranked from high to low as follows: cetuximab (overall score 3.3666), bevacizumab (3.3043), panitumumab (3.2030), aflibercept (2.8923) and regorafenib (2.8366). CONCLUSIONS:A comprehensive value assessment system combining "multi-dimensional criteria," "multi-perspectives," and an "integrative assessment" is necessary to evaluate the value of medicines. The results showed not only the order of weights of different dimensions or criteria, but also the rankings of the value of the targeted therapies.