Undisclosed changes in outcomes in randomized controlled trials: an observational study.
ABSTRACT: We wanted to investigate the frequency of undisclosed changes in the outcomes of randomized controlled trials (RCTs) between trial registration and publication.Using a retrospective, nonrandom, cross-sectional study design, we investigated RCTs published in consecutive issues of 5 major medical journals during a 6-month period and their associated trials registry entries. Articles were excluded if they did not have an available trial registry entry, did not have analyzable outcomes, or were secondary publications. The primary outcome was the proportion of publications in which the primary outcome of the trial was, without disclosure, changed between that recorded in the trial registry and that reported in the final publication. The secondary outcome was the proportion of publications in which the secondary outcome was changed without disclosure.We reviewed 158 reports of RCTs and included 110 in the analysis. In 34 (31%), a primary outcome had been changed, and in 77 (70%), a secondary outcome had been changed.There are substantial and important undisclosed changes made to the outcomes of published RCTs between trial registration and publication. This finding has important implications for the interpretation of trial results. Disclosure and discussion of changes would improve transparency in the performance and reporting of trials.
Project description:OBJECTIVE:To determine (i) the difference in the frequency of serious adverse events (SAEs) reported in trial registrations and their respective primary publications and (ii) the effect of adding SAE data from registries to a network meta-analysis (NMA) in changing the surface under the cumulative ranking (SUCRA) curve values of interventions. DESIGN:Secondary analysis of primary publications from two NMAs. ELIGIBILITY CRITERIA FOR SELECTING STUDIES:We included randomised trials published in English after 2005 that were included in two NMAs of pharmacological interventions for Alzheimer's disease and chronic obstructive pulmonary disease. DATA EXTRACTION:Two reviewers independently searched multiple international trial registries for registration status and abstracted data from the included study publications and ClinicalTrials.gov. RESULTS:Of the 203 randomised trials included, 140 (69.0%) were registered with a trial registry and 72 (35.5%) posted results in the registry. The proportion of registered trials increased over time (38.5% in 2005 vs 78.6% in 2014). Of the publications with results posted in a trial registry, 14 (19.4%) had inconsistent reporting of overall SAEs; 7 (10.4%) studies did not report SAEs in the publication but did in the registry. In the 134 randomised trials with a prespecified primary outcome in the registry, 19 studies (9.4%) had a change in the primary outcome in the publication. Adding SAEs reported in registries to the NMAs did not affect the ranking of interventions. CONCLUSION:We identified inconsistent reporting of SAEs in randomised trials that were included in two NMAs. Findings highlight the importance of including trial registries in the grey literature search and verifying safety data before incorporating it into NMAs. STUDY REGISTRATION:osf.io/mk6dr.
Project description:Recent literature hints that outcomes of clinical trials in medicine are selectively reported. If applicable to psychotic disorders, such bias would jeopardize the reliability of randomized clinical trials (RCTs) investigating antipsychotics and thus their extrapolation to clinical practice. We therefore comprehensively examined outcome reporting bias in RCTs of antipsychotic drugs by a systematic review of prespecified outcomes on ClinicalTrials.gov records of RCTs investigating antipsychotic drugs in schizophrenia and schizoaffective disorder between 1 January 2006 and 31 December 2013. These outcomes were compared with outcomes published in scientific journals. Our primary outcome measure was concordance between prespecified and published outcomes; secondary outcome measures included outcome modifications on ClinicalTrials.gov after trial inception and the effects of funding source and directionality of results on record adherence. Of the 48 RCTs, 85% did not fully adhere to the prespecified outcomes. Discrepancies between prespecified and published outcomes were found in 23% of RCTs for primary outcomes, whereas 81% of RCTs had at least one secondary outcome non-reported, newly introduced, or changed to a primary outcome in the respective publication. In total, 14% of primary and 44% of secondary prespecified outcomes were modified after trial initiation. Neither funding source (P=0.60) nor directionality of the RCT results (P=0.10) impacted ClinicalTrials.gov record adherence. Finally, the number of published safety endpoints (N=335) exceeded the number of prespecified safety outcomes by 5.5 fold. We conclude that RCTs investigating antipsychotic drugs suffer from substantial outcome reporting bias and offer suggestions to both monitor and limit such bias in the future.
Project description:BACKGROUND: Although randomized clinical trials (RCTs) are considered the gold standard of evidence, their reporting is often suboptimal. Trial registries have the potential to contribute important methodologic information for critical appraisal of study results. METHODS AND FINDINGS: The objective of the study was to evaluate the reporting of key methodologic study characteristics in trial registries. We identified a random sample (n = 265) of actively recruiting RCTs using the World Health Organization International Clinical Trials Registry Platform (ICTRP) search portal in 2008. We assessed the reporting of relevant domains from the Cochrane Collaboration's 'Risk of bias' tool and other key methodological aspects. Our primary outcomes were the proportion of registry records with adequate reporting of random sequence generation, allocation concealment, blinding, and trial outcomes. Two reviewers independently assessed each record. Weighted overall proportions in the ICTRP search portal for adequate reporting of sequence generation, allocation concealment, blinding (including and excluding open label RCT) and primary outcomes were 5.7% (95% CI 3.0-8.4%), 1.4% (0-2.8%), 41% (35-47%), 8.4% (4.1-13%), and 66% (60-72%), respectively. The proportion of adequately reported RCTs was higher for registries that used specific methodological fields for describing methods of randomization and allocation concealment compared to registries that did not. Concerning other key methodological aspects, weighted overall proportions of RCTs with adequately reported items were as follows: eligibility criteria (81%), secondary outcomes (46%), harm (5%) follow-up duration (62%), description of the interventions (53%) and sample size calculation (1%). CONCLUSIONS: Trial registries currently contain limited methodologic information about registered RCTs. In order to permit adequate critical appraisal of trial results reported in journals and registries, trial registries should consider requesting details on key RCT methods to complement journal publications. Full protocols remain the most comprehensive source of methodologic information and should be made publicly available.
Project description:OBJECTIVE:We assessed the adequacy of randomized controlled trial (RCT) registration, changes to registration data and reporting completeness for articles in ICMJE journals during 2.5 years after registration requirement policy. METHODS:For a set of 149 reports of 152 RCTs with ClinicalTrials.gov registration number, published from September 2005 to April 2008, we evaluated the completeness of 9 items from WHO 20-item Minimum Data Set relevant for assessing trial quality. We also assessed changes to the registration elements at the Archive site of ClinicalTrials.gov and compared published and registry data. RESULTS:RCTs were mostly registered before 13 September 2005 deadline (n?=?101, 66.4%); 118 (77.6%) started recruitment before and 31 (20.4%) after registration. At the time of registration, 152 RCTs had a total of 224 missing registry fields, most commonly 'Key secondary outcomes' (44.1% RCTs) and 'Primary outcome' (38.8%). More RCTs with post-registration recruitment had missing Minimum Data Set items than RCTs with pre-registration recruitment: 57/118 (48.3%) vs. 24/31 (77.4%) (?(2) (1)?=?7.255, P?=?0.007). Major changes in the data entries were found for 31 (25.2%) RCTs. The number of RCTs with differences between registered and published data ranged from 21 (13.8%) for Study type to 118 (77.6%) for Target sample size. CONCLUSIONS:ICMJE journals published RCTs with proper registration but the registration data were often not adequate, underwent substantial changes in the registry over time and differed in registered and published data. Editors need to establish quality control procedures in the journals so that they continue to contribute to the increased transparency of clinical trials.
Project description:OBJECTIVES:To analyse the completeness of reporting of blinding in randomised controlled trials (RCTs) of interventions in anaesthesiology, the actual blinding status of various persons associated with an RCT and trial authors' interpretation of blinding terminology related to RCTs. METHODS:This was a methodological study and a cross-sectional survey. We analysed reporting related to blinding in published RCTs of interventions published in seven highly cited anaesthesiology journals from 2014 to 2016 and registered protocols in ClinicalTrials.gov. We surveyed corresponding authors of included RCTs about their definitions of blinding. The primary outcome was the number of RCTs that explicitly described who was blinded in a trial. Secondary outcomes were definitions of blinding terminology in the trials; trial authors' interpretation of blinding terminology; discrepancies in the blinding description within registered protocols and between registered protocols and publications. RESULTS:Out of 622 analysed RCTs, 38% were not explicitly described as either open label or blinded studies and 10% did not report any information about blinding or lack of blinding. Only one manuscript fully reported the status of blinding for various individuals that may be involved with a trial. The most common descriptor was that a trial was double-blind. We found discrepant information regarding blinding in the majority of registered protocols. Even when there were no discrepancies in the registration, we found discrepancies in the reporting of blinding between the majority of registered protocols and published manuscripts. The survey of authors (40 responses from 231 eligible authors; 17% response rate) of analysed RCTs showed that they differed in how they defined different levels of blinding in trials. CONCLUSIONS:Reporting of the blinding status of key individuals involved in analysed anaesthesiology RCTs was insufficient. For reporting guidelines, peer reviewers and editors should insist on clear information on who was blinded in a trial instead of using the term 'double-blind' for different blinding practices.
Project description:To address the bias occurring in the medical literature associated with selective outcome reporting, in 2005, the International Committee of Medical Journal Editors (ICMJE) introduced mandatory trial registration guidelines and member journals required prospective registration of trials prior to patient enrolment as a condition of publication. No research has examined whether these guidelines are impacting psychiatry publications. Our objectives were to determine the extent to which articles published in psychiatry journals adhering to ICMJE guidelines were correctly prospectively registered, whether there was evidence of selective outcome reporting and changes to participant numbers, and whether there was a relationship between registration status and source of funding.Any clinical trial (as defined by ICMJE) published between 1 January 2009 and 31 July 2013 in the top five psychiatry journals adhering to ICMJE guidelines (The American Journal of Psychiatry, Archives of General Psychiatry/JAMA Psychiatry, Biological Psychiatry, Journal of the American Academy of Child and Adolescent Psychiatry, and The Journal of Clinical Psychiatry) and conducted after July 2005 (or 2007 for two journals) was included. For each identified trial, where possible we extracted trial registration information, changes to POMs between publication and registry to assess selective outcome reporting, changes to participant numbers, and funding type.Out of 3305 articles, 181 studies were identified as clinical trials requiring registration: 21 (11.6%) were deemed unregistered, 61 (33.7%) were retrospectively registered, 37 (20.4%) had unclear POMs either in the article or the registry and 2 (1.1%) were registered in an inaccessible trial registry. Only 60 (33.1%) studies were prospectively registered with clearly defined POMs; 17 of these 60 (28.3%) showed evidence of selective outcome reporting and 16 (26.7%) demonstrated a change in participant numbers of 20% or more; only 26 (14.4%) of the 181 the trials were prospectively registered and did not alter their POMs or the time frames at which they were measured. Prospective registration with no changes in POMs occurred more frequently with pharmaceutical funding.Although standards are in place to improve prospective registration and transparency in clinical trials, less than 15% of psychiatry trials were prospectively registered with no changes in POMs. Most trials were either not prospectively registered, changed POMs or the timeframes at some point after registration or changed participant numbers. Authors, journal editors and reviewers need to further efforts to highlight the value of prospective trial registration.
Project description:BACKGROUND:A randomized controlled trial (RCT) remains the pinnacle of clinical research design. However, RCTs in neurosurgery, especially those comparing surgery to non-operative treatment, are rare and their relevance and applicability have been questioned. This study set out to assess trial design and quality and identify their influence on outcomes in recent neurosurgical trials that compare surgery to non-operative treatment. METHODS:From 2000 to 2017, PubMed and Embase databases and four trial registries were searched. RCTs were evaluated for study design, funding, adjustments to reported outcome measures, accrual of patients, and academic impact. RESULTS:Eighty-two neurosurgical RCTs were identified, 40 in spine disorders, 19 neurovascular and neurotrauma, 11 functional neurosurgery, ten peripheral nerve, and two pituitary surgery. Eighty-four RCTs were registered, of which some are ongoing. Trial registration rate differed per subspecialty. Funding was mostly from non-industry institutions (58.5%), but 25.6% of RCTs did not report funding sources. 36.4% of RCTs did not report a difference between surgical and non-operative treatment, 3.7% favored non-operative management. Primary and secondary outcome measures were changed in 13.2% and 34.2% of RCTs respectively and varied by subspecialty. 41.9% of RCTs subtracted ≥ 10% of the anticipated accrual and 12.9% of RCTs added ≥ 10%. 7.3% of registered RCTs were terminated, mostly due to too slow recruitment. Subspecialty, registration, funding, masking, population size, and changing outcome measures were not significantly associated with a reported benefit of surgery. High Jadad scores (≥ 4) were negatively associated with a demonstration of surgical benefit (P < 0.05). CONCLUSIONS:Neurosurgical RCTs comparing surgical to non-operative treatment often find a benefit for surgical treatment. Changes to outcome measurements and anticipated accrual are common and funding sources are not always reported.
Project description:BACKGROUND:Process evaluations are increasingly conducted within pragmatic randomised controlled trials (RCTs) of health services interventions and provide vital information to enhance understanding of RCT findings. However, issues pertaining to process evaluation in this specific context have been little discussed. We aimed to describe the frequency, characteristics, labelling, value, practical conduct issues, and accessibility of published process evaluations within pragmatic RCTs in health services research. METHODS:We used a 2-phase systematic search process to (1) identify an index sample of journal articles reporting primary outcome results of pragmatic RCTs published in 2015 and then (2) identify all associated publications. We used an operational definition of process evaluation based on the Medical Research Council's process evaluation framework to identify both process evaluations reported separately and process data reported in the trial results papers. We extracted and analysed quantitative and qualitative data to answer review objectives. RESULTS:From an index sample of 31 pragmatic RCTs, we identified 17 separate process evaluation studies. These had varied characteristics and only three were labelled 'process evaluation'. Each of the 31 trial results papers also reported process data, with a median of five different process evaluation components per trial. Reported barriers and facilitators related to real-world collection of process data, recruitment of participants to process evaluations, and health services research regulations. We synthesised a wide range of reported benefits of process evaluations to interventions, trials, and wider knowledge. Visibility was often poor, with 13/17 process evaluations not mentioned in the trial results paper and 12/16 process evaluation journal articles not appearing in the trial registry. CONCLUSIONS:In our sample of reviewed pragmatic RCTs, the meaning of the label 'process evaluation' appears uncertain, and the scope and significance of the term warrant further research and clarification. Although there were many ways in which the process evaluations added value, they often had poor visibility. Our findings suggest approaches that could enhance the planning and utility of process evaluations in the context of pragmatic RCTs. TRIAL REGISTRATION:Not applicable for PROSPERO registration.
Project description:To compare the characteristics, quality and treatment effects of randomised clinical trials (RCTs) by individual patient data (IPD) availability, in trials eligible for 18 IPD meta-analyses (MA).Trial characteristics, risk of bias (RoB) and hazard ratio (HR) for overall survival were extracted from IPD-MA publications and/or RCTs publications. Data for the RoB assessment were extracted for a subset of 73 RCTs. Two investigators blinded to whether IPD was available or not evaluated the RoB for these trials. Treatment effects were compared using ratios of global HRs (RHRs) of IPD-unavailable trials and IPD-available trials. RHR were pooled using a fixed-effect model.We examined the IPD availability for each trial eligible for each IPD-MA; when the IPD was not available for a trial, we used information from published sources.We selected all published IPD-MAs conducted at Gustave Roussy and the RCTs eligible for each.349 RCTs (73?018 patients) from 18 MAs were eligible: 60 RCTs (5890 patients) had unavailable IPD and 289 RCTs (67?128 patients) had available IPD. The main reason for IPD unavailability was data loss by investigators. IPD-unavailable trials were smaller (p<0.001), more often monocentric (p<0.001) and non-international (p=0.0004) than IPD-available trials. Geographical areas differed (p=0.054) between IPD-unavailable IPD-available trials. RoB was higher in IPD-unavailable RCTs for random sequence generation (p=0.007) and allocation concealment (p=0.006). The HR and 95% confidence interval (CI) for overall survival were extractable from publications in 23/60 IPD-unavailable trials included in 10 different MAs. Treatment effects were significantly greater for IPD-unavailable trials compared with IPD-available trials (RHR=0.86 (95% CI 0.75 to 0.98)).IPD-unavailable RCTs were significantly different from IPD-available RCTs in terms of trial characteristics and were at greater RoB. IPD-unavailable RCTs had a significantly greater treatment effect.
Project description:Conflicts of interest may lead to biased trial designs and unbalanced interpretation of study results. We aimed to evaluate the reporting of potential conflicts of interest in full publications of surgical randomised controlled trials (RCTs). A systematic literature search was performed in CENTRAL, MEDLINE and EMBASE (1985-2014) to find all surgical RCTs of medical devices and perioperative pharmacological or nutritional interventions. The information on conflicts of interest was evaluated both quantitatively and qualitatively, and the development of stated conflicts over time was studied. Of 7934 articles, 444 met the inclusion criteria. In 93 of 444 trials (20.9%), conflicts of interest were disclosed. In half of the cases, the information provided was insufficient to permit conclusions regarding possible influence on the trials. Information about conflicts of interest has increased continuously during the last decades (1985-1994: 0%, 1995-2004: 2.8% and 2005-2014: 33.0%; p<0.001). Among the 115 industry-funded trials, industry participation was considered as a potential conflict of interest in 24 cases (20.9%). Over the past three decades, only every 10th trial has provided appropriate information on conflicts of interest. However, transparency is crucial for the reliability of evidence-based medicine. There is an urgent need for the full disclosure of all conflicts of interest in surgical publishing and for transparency regarding cooperation between academia and industry.