Statistical analyses of ordinal outcomes in randomised controlled trials: a scoping review
Ontology highlight
ABSTRACT:
SUBMITTER: Selman C
PROVIDER: S-EPMC10998402 | biostudies-literature | 2024 Jan
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:IntroductionRandomised controlled trials (RCTs) aim to assess the effect of one (or more) unproven health interventions relative to other reference interventions. RCTs sometimes use an ordinal outcome, which is an endpoint that comprises of multiple, monotonically ordered categories that are not necessarily separated by a quantifiable distance. Ordinal outcomes are appealing in clinical settings as specific disease states can represent meaningful categories that may be of clinical importance to researchers. Ordinal outcomes can also retain information and increase statistical power compared to dichotomised outcomes and can allow multiple clinical outcomes to be comprised in a single endpoint. Target parameters for ordinal outcomes in RCTs may vary depending on the nature of the research question, the modelling assumptions and the expertise of the data analyst. The aim of this scoping review is to systematically describe the use of ordinal outcomes in contemporary RCTs. Specifically, we aim to: [Formula: see text] Identify which target parameters are of interest in trials that use an ordinal outcome, and whether these parameters are explicitly defined. [Formula: see text] Describe how ordinal outcomes are analysed in RCTs to estimate a treatment effect. [Formula: see text] Describe whether RCTs that use an ordinal outcome adequately report key methodological aspects specific to the analysis of the ordinal outcome. Results from this review will outline the current state of practice of the use of ordinal outcomes in RCTs. Ways to improve the analysis and reporting of ordinal outcomes in RCTs will be discussed.Methods and analysisWe will review RCTs that are published in the top four medical journals (British Medical Journal, New England Journal of Medicine, The Lancet and the Journal of the American Medical Association) between 1 January 2012 and 31 July 2022 that use an ordinal outcome as either a primary or a secondary outcome. The review will identify articles through a PubMed-specific search strategy. Our review will adhere to guidelines for scoping reviews as described in the PRISMA-ScR checklist. The study characteristics and details of the study design and analysis, including the target parameter(s) and statistical methods used to analyse the ordinal outcome, will be extracted from eligible studies. The screening, review and data extraction will be conducted using Covidence, a web-based tool for managing systematic reviews. The data will be summarised using descriptive statistics.
| S-EPMC10119829 | biostudies-literature
Project description:ObjectivesTo identify the evidence gaps that exist regarding the efficacy or effectiveness of hand surgery.SettingA scoping review. We systematically searched MEDLINE, Embase and CENTRAL databases to identify all hand surgical randomised controlled trials from inception to 7 November 2020.ResultsOf the 220 identified randomised controlled trials, none were fundamental efficacy trials, that is, compared surgery with placebo surgery. 172 (78%) trials compared the outcomes of different surgical techniques, and 143 (65%) trials were trauma related. We identified only 47 (21%) trials comparing surgery with non-operative care or injection.ConclusionThe evidence supporting use of surgery especially for chronic hand conditions is scarce. To determine optimal care for people with hand conditions, more resources should be aimed at placebo-controlled trials and pragmatic effectiveness trials comparing hand surgery with non-operative care.Prospero registration numberCRD42019122710.
| S-EPMC9557326 | biostudies-literature
Project description:BackgroundStatistical methods for the analysis of harm outcomes in randomised controlled trials (RCTs) are rarely used, and there is a reliance on simple approaches to display information such as in frequency tables. We aimed to identify whether any statistical methods had been specifically developed to analyse prespecified secondary harm outcomes and non-specific emerging adverse events (AEs).MethodsA scoping review was undertaken to identify articles that proposed original methods or the original application of existing methods for the analysis of AEs that aimed to detect potential adverse drug reactions (ADRs) in phase II-IV parallel controlled group trials. Methods where harm outcomes were the (co)-primary outcome were excluded. Information was extracted on methodological characteristics such as: whether the method required the event to be prespecified or could be used to screen emerging events; and whether it was applied to individual events or the overall AE profile. Each statistical method was appraised and a taxonomy was developed for classification.ResultsForty-four eligible articles proposing 73 individual methods were included. A taxonomy was developed and articles were categorised as: visual summary methods (8 articles proposing 20 methods); hypothesis testing methods (11 articles proposing 16 methods); estimation methods (15 articles proposing 24 methods); or methods that provide decision-making probabilities (10 articles proposing 13 methods). Methods were further classified according to whether they required a prespecified event (9 articles proposing 12 methods), or could be applied to emerging events (35 articles proposing 61 methods); and if they were (group) sequential methods (10 articles proposing 12 methods) or methods to perform final/one analyses (34 articles proposing 61 methods).ConclusionsThis review highlighted that a broad range of methods exist for AE analysis. Immediate implementation of some of these could lead to improved inference for AE data in RCTs. For example, a well-designed graphic can be an effective means to communicate complex AE data and methods appropriate for counts, time-to-event data and that avoid dichotomising continuous outcomes can improve efficiencies in analysis. Previous research has shown that adoption of such methods in the scientific press is limited and that strategies to support change are needed.Trial registrationPROSPERO registration: https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=97442.
| S-EPMC7708917 | biostudies-literature
Project description:ObjectivesWe compared calculations of relative risks of cancer death in Swedish mammography trials and in other cancer screening trials.ParticipantsMen and women from 30 to 74 years of age.SettingRandomised trials on cancer screening.DesignFor each trial, we identified the intervention period, when screening was offered to screening groups and not to control groups, and the post-intervention period, when screening (or absence of screening) was the same in screening and control groups. We then examined which cancer deaths had been used for the computation of relative risk of cancer death.Main outcome measuresRelative risk of cancer death.ResultsIn 17 non-breast screening trials, deaths due to cancers diagnosed during the intervention and post-intervention periods were used for relative risk calculations. In the five Swedish trials, relative risk calculations used deaths due to breast cancers found during intervention periods, but deaths due to breast cancer found at first screening of control groups were added to these groups. After reallocation of the added breast cancer deaths to post-intervention periods of control groups, relative risks of 0.86 (0.76; 0.97) were obtained for cancers found during intervention periods and 0.83 (0.71; 0.97) for cancers found during post-intervention periods, indicating constant reduction in the risk of breast cancer death during follow-up, irrespective of screening.ConclusionsThe use of unconventional statistical methods in Swedish trials has led to overestimation of risk reduction in breast cancer death attributable to mammography screening. The constant risk reduction observed in screening groups was probably due to the trial design that optimised awareness and medical management of women allocated to screening groups.
| S-EPMC4672251 | biostudies-literature
Project description:BackgroundWith rising obesity rates worldwide, clinical trials focused on identifying effective treatments are increasing. While guidelines exist for pharmaceutical drugs targeting obesity, there are none for herbal medicine clinical trials for anti-obesity. Both industries refer to the same guidelines for clinical trials.ObjectivesThis scoping review aimed to gather information from herbal medicine anti-obesity randomised controlled trials (RCTs), analyse the methodologies and assess their alignment with international guidelines.Eligibility criteriaThis review included RCTs of participants of all ages with obesity utilising herbal medicine with any comparators and focusing on various outcome measures.Sources of evidence: Only published journal articles were included.Charting methodsArticles were extracted from MEDLINE, CENTRAL and EMBASE using predetermined keywords. Relevant data, such as the study characteristics, types of herbal interventions and controls, treatment durations, outcome measures and safety monitoring methods were recorded in a table format for comparative analysis.ResultsWe included 99 RCTs that showed participant sample sizes ranging from 8 to 182, ages 18 to 80 years and body mass indexes (BMIs) between 25 and 49.9 kg/m2. Herbal interventions used single herbs (n = 57) and mixtures (n = 42), given for 14 days to 56 weeks. Studies implementing diet modifications include restricted calorie diets (n = 35), food-portion controlled diets (n = 7) and fixed calorie diets (n = 7). Of the 28 studies implementing exercise, most were of moderate intensity (n = 22). All studies collected BMI and weight as primary outcomes. Body fat composition was measured in over 50% of studies using a body analyser (n = 57). Waist, hip and abdominal circumferences were infrequently measured. Radiological tools used include dual-energy X-ray absorptiometry (n = 16), computed tomography scans (n = 10) and ultrasound (n = 2). Safety monitoring methods were reported in most studies (n = 76).ConclusionIn conclusion, almost 50% of the studies adhered to international pharmaceutical clinical trial guidelines, addressing dietary, lifestyle, physical activity and cardiovascular risk factors. Nonetheless, more herbal anti-obesity studies need to consider the assessment of weight maintenance.
| S-EPMC11795625 | biostudies-literature
Project description:BackgroundChoosing or altering the planned statistical analysis approach after examination of trial data (often referred to as 'p-hacking') can bias the results of randomised trials. However, the extent of this issue in practice is currently unclear. We conducted a review of published randomised trials to evaluate how often a pre-specified analysis approach is publicly available, and how often the planned analysis is changed.MethodsA review of randomised trials published between January and April 2018 in six leading general medical journals. For each trial, we established whether a pre-specified analysis approach was publicly available in a protocol or statistical analysis plan and compared this to the trial publication.ResultsOverall, 89 of 101 eligible trials (88%) had a publicly available pre-specified analysis approach. Only 22/89 trials (25%) had no unexplained discrepancies between the pre-specified and conducted analysis. Fifty-four trials (61%) had one or more unexplained discrepancies, and in 13 trials (15%), it was impossible to ascertain whether any unexplained discrepancies occurred due to incomplete reporting of the statistical methods. Unexplained discrepancies were most common for the analysis model (n = 31, 35%) and analysis population (n = 28, 31%), followed by the use of covariates (n = 23, 26%) and the approach for handling missing data (n = 16, 18%). Many protocols or statistical analysis plans were dated after the trial had begun, so earlier discrepancies may have been missed.ConclusionsUnexplained discrepancies in the statistical methods of randomised trials are common. Increased transparency is required for proper evaluation of results.
| S-EPMC7257229 | biostudies-literature
Project description:OBJECTIVE:Randomised controlled trials (RCT) are the gold standard to provide unbiased data. However, when patients have a treatment preference, randomisation may influence participation and outcomes (eg, external and internal validity). The aim of this study was to assess the influence of patients' preference in RCTs by analysing partially randomised patient preference trials (RPPT); an RCT and preference cohort combined. DESIGN:Systematic review and meta-analyses. DATA SOURCES:MEDLINE, Embase, PsycINFO and the Cochrane Library. ELIGIBILITY CRITERIA FOR SELECTING STUDIES:RPPTs published between January 2005 and October 2018 reporting on allocation of patients to randomised and preference cohorts were included. DATA EXTRACTION AND SYNTHESIS:Two independent reviewers extracted data. The main outcomes were the difference in external validity (participation and baseline characteristics) and internal validity (lost to follow-up, crossover and the primary outcome) between the randomised and the preference cohort within each RPPT, compared in a meta-regression using a Wald test. Risk of bias was not assessed, as no quality assessment for RPPTs has yet been developed. RESULTS:In total, 117 of 3734 identified articles met screening criteria and 44 were eligible (24 873 patients). The participation rate in RPPTs was >95% in 14 trials (range: 48%-100%) and the randomisation refusal rate was >50% in 26 trials (range: 19%-99%). Higher education, female, older age, race and prior experience with one treatment arm were characteristics of patients declining randomisation. The lost to follow-up and cross-over rate were significantly higher in the randomised cohort compared with the preference cohort. Following the meta-analysis, the reported primary outcomes were comparable between both cohorts of the RPPTs, mean difference 0.093 (95% CI -0.178 to 0.364, p=0.502). CONCLUSIONS:Patients' preference led to a substantial proportion of a specific patient group refusing randomisation, while it did not influence the primary outcome within an RPPT. Therefore, RPPTs could increase external validity without compromising the internal validity compared with RCTs. PROSPERO REGISTRATION NUMBER:CRD42019094438.
| S-EPMC6797441 | biostudies-literature
Project description: Not available
| S-EPMC10945168 | biostudies-literature
Project description:BackgroundIn individually randomised trials we might expect interventions delivered in groups or by care providers to result in clustering of outcomes for participants treated in the same group or by the same care provider. In partially nested randomised controlled trials (pnRCTs) this clustering only occurs in one trial arm, commonly the intervention arm. It is important to measure and account for between-cluster variability in trial design and analysis. We compare analysis approaches for pnRCTs with continuous outcomes, investigating the impact on statistical inference of cluster sizes, coding of the non-clustered arm, intracluster correlation coefficient (ICCs), and differential variance between intervention and control arm, and provide recommendations for analysis.MethodsWe performed a simulation study assessing the performance of six analysis approaches for a two-arm pnRCT with a continuous outcome. These include: linear regression model; fully clustered mixed-effects model with singleton clusters in control arm; fully clustered mixed-effects model with one large cluster in control arm; fully clustered mixed-effects model with pseudo clusters in control arm; partially nested homoscedastic mixed effects model, and partially nested heteroscedastic mixed effects model. We varied the cluster size, number of clusters, ICC, and individual variance between the two trial arms.ResultsAll models provided unbiased intervention effect estimates. In the partially nested mixed-effects models, methods for classifying the non-clustered control arm had negligible impact. Failure to account for even small ICCs resulted in inflated Type I error rates and over-coverage of confidence intervals. Fully clustered mixed effects models provided poor control of the Type I error rates and biased ICC estimates. The heteroscedastic partially nested mixed-effects model maintained relatively good control of Type I error rates, unbiased ICC estimation, and did not noticeably reduce power even with homoscedastic individual variances across arms.ConclusionsIn general, we recommend the use of a heteroscedastic partially nested mixed-effects model, which models the clustering in only one arm, for continuous outcomes similar to those generated under the scenarios of our simulations study. However, with few clusters (3-6), small cluster sizes (5-10), and small ICC (≤0.05) this model underestimates Type I error rates and there is no optimal model.
| S-EPMC6186141 | biostudies-literature
Project description:BACKGROUND:Selective reporting of outcomes in clinical trials is a serious problem. We aimed to investigate the influence of the peer review process within biomedical journals on reporting of primary outcome(s) and statistical analyses within reports of randomised trials. METHODS:Each month, PubMed (May 2014 to April 2015) was searched to identify primary reports of randomised trials published in six high-impact general and 12 high-impact specialty journals. The corresponding author of each trial was invited to complete an online survey asking authors about changes made to their manuscript as part of the peer review process. Our main outcomes were to assess: (1) the nature and extent of changes as part of the peer review process, in relation to reporting of the primary outcome(s) and/or primary statistical analysis; (2) how often authors followed these requests; and (3) whether this was related to specific journal or trial characteristics. RESULTS:Of 893 corresponding authors who were invited to take part in the online survey 258 (29%) responded. The majority of trials were multicentre (n = 191; 74%); median sample size 325 (IQR 138 to 1010). The primary outcome was clearly defined in 92% (n = 238), of which the direction of treatment effect was statistically significant in 49%. The majority responded (1-10 Likert scale) they were satisfied with the overall handling (mean 8.6, SD 1.5) and quality of peer review (mean 8.5, SD 1.5) of their manuscript. Only 3% (n = 8) said that the editor or peer reviewers had asked them to change or clarify the trial's primary outcome. However, 27% (n = 69) reported they were asked to change or clarify the statistical analysis of the primary outcome; most had fulfilled the request, the main motivation being to improve the statistical methods (n = 38; 55%) or avoid rejection (n = 30; 44%). Overall, there was little association between authors being asked to make this change and the type of journal, intervention, significance of the primary outcome, or funding source. Thirty-six percent (n = 94) of authors had been asked to include additional analyses that had not been included in the original manuscript; in 77% (n = 72) these were not pre-specified in the protocol. Twenty-three percent (n = 60) had been asked to modify their overall conclusion, usually (n = 53; 88%) to provide a more cautious conclusion. CONCLUSION:Overall, most changes, as a result of the peer review process, resulted in improvements to the published manuscript; there was little evidence of a negative impact in terms of post hoc changes of the primary outcome. However, some suggested changes might be considered inappropriate, such as unplanned additional analyses, and should be discouraged.
| S-EPMC5765661 | biostudies-literature