Dataset Information

Subgroup analyses in randomised controlled trials: cohort study on trial protocols and journal publications.

ABSTRACT:

Objective

To investigate the planning of subgroup analyses in protocols of randomised controlled trials and the agreement with corresponding full journal publications.

Design

Cohort of protocols of randomised controlled trial and subsequent full journal publications.

Setting

Six research ethics committees in Switzerland, Germany, and Canada.

Data sources

894 protocols of randomised controlled trial involving patients approved by participating research ethics committees between 2000 and 2003 and 515 subsequent full journal publications.

Results

Of 894 protocols of randomised controlled trials, 252 (28.2%) included one or more planned subgroup analyses. Of those, 17 (6.7%) provided a clear hypothesis for at least one subgroup analysis, 10 (4.0%) anticipated the direction of a subgroup effect, and 87 (34.5%) planned a statistical test for interaction. Industry sponsored trials more often planned subgroup analyses compared with investigator sponsored trials (195/551 (35.4%) v 57/343 (16.6%), P<0.001). Of 515 identified journal publications, 246 (47.8%) reported at least one subgroup analysis. In 81 (32.9%) of the 246 publications reporting subgroup analyses, authors stated that subgroup analyses were prespecified, but this was not supported by 28 (34.6%) corresponding protocols. In 86 publications, authors claimed a subgroup effect, but only 36 (41.9%) corresponding protocols reported a planned subgroup analysis.

Conclusions

Subgroup analyses are insufficiently described in the protocols of randomised controlled trials submitted to research ethics committees, and investigators rarely specify the anticipated direction of subgroup effects. More than one third of statements in publications of randomised controlled trials about subgroup prespecification had no documentation in the corresponding protocols. Definitive judgments regarding credibility of claimed subgroup effects are not possible without access to protocols and analysis plans of randomised controlled trials.

SUBMITTER: Kasenda B

PROVIDER: S-EPMC4100616 | biostudies-literature | 2014 Jul

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Subgroup analyses in randomised controlled trials: cohort study on trial protocols and journal publications.

Kasenda Benjamin B Schandelmaier Stefan S Sun Xin X von Elm Erik E You John J Blümle Anette A Tomonaga Yuki Y Saccilotto Ramon R Amstutz Alain A Bengough Theresa T Meerpohl Joerg J JJ Stegert Mihaela M Olu Kelechi K KK Tikkinen Kari A O KA Neumann Ignacio I Carrasco-Labra Alonso A Faulhaber Markus M Mulla Sohail M SM Mertz Dominik D Akl Elie A EA Bassler Dirk D Busse Jason W JW Ferreira-González Ignacio I Lamontagne Francois F Nordmann Alain A Gloy Viktoria V Raatz Heike H Moja Lorenzo L Rosenthal Rachel R Ebrahim Shanil S Vandvik Per O PO Johnston Bradley C BC Walter Martin A MA Burnand Bernard B Schwenkglenks Matthias M Hemkens Lars G LG Bucher Heiner C HC Guyatt Gordon H GH Briel Matthias M

BMJ (Clinical research ed.) 20140716

<h4>Objective</h4>To investigate the planning of subgroup analyses in protocols of randomised controlled trials and the agreement with corresponding full journal publications.<h4>Design</h4>Cohort of protocols of randomised controlled trial and subsequent full journal publications.<h4>Setting</h4>Six research ethics committees in Switzerland, Germany, and Canada.<h4>Data sources</h4>894 protocols of randomised controlled trial involving patients approved by participating research ethics committe ...[more]

PMID: 25030633

Similar Datasets

Project description:ObjectiveTo investigate to what extent evidence from previous similar trials or systematic reviews was considered before conducting new trials.DesignCohort study of contemporary protocols for trials with ethical approval.MethodsAll protocols for randomised trials approved by the five ethical committees in Denmark between January 2012 and March 2013 were screened for eligibility. Included protocols were read in full to determine whether a systematic search had been conducted and references were checked to evaluate whether trial rationale and design could be challenged for not adequately considering previous evidence. To investigate whether protocols cited relevant trials, we used simple search strategies that could easily be conducted by researchers without experience with literature searches.ResultsSixty-seven protocols were included. Only two (3%) of the protocols explicitly stated to have conducted a literature search and only one (1%) provided information that allowed the search to be replicated. Eleven (16%) of the protocols described trials where we found the information insufficient to judge if the trial was ethically justified, either due to a comparator that was not supported by the presented evidence (six protocols), because they did not present a rationale for conducting the trial (two protocols), or for both reasons (three protocols). For eight (12%) of the protocols, our search identified trials that could have been relevant to cite as justification.ConclusionsWhile most protocols seem to adequately consider existing evidence, a substantial minority of trials might lack a sufficient evidence base. Very few trials seemed to have been based on a literature search which makes it impossible to know whether all relevant previous trials had been considered. Rules for ethical approval should include requirements for systematic literature searches to ensure that research participants are not exposed to sub-optimal treatments or unnecessary harms as well as to reduce research waste.

Project description:ObjectiveTo investigate the impact of industry funding on reporting of subgroup analyses in randomised controlled trials.DesignSystematic review.Data sourcesMedline.Study selectionRandomised controlled trials published in 118 core clinical journals (defined by the National Library of Medicine) in 2007. 1140 study reports in a 1:1 ratio by high (five general medicine journals with largest number of total citations in 2007) versus lower impact journals, were randomly sampled. Two reviewers, independently and in duplicate, used standardised, piloted forms to screen study reports for eligibility and to extract data. They also used explicit criteria to determine whether a randomised controlled trial reported subgroup analyses. Logistic regression was used to examine the association of prespecified study characteristics with reporting versus not reporting of subgroup analyses.Results469 randomised controlled trials were included, of which 207 (44%) reported subgroup analyses. High impact journals (adjusted odds ratio 2.64, 95% confidence interval 1.62 to 4.33), non-surgical (versus surgical) trials (2.10, 1.26 to 3.50), and larger sample size (3.38, 1.64 to 6.99) were associated with more frequent reporting of subgroup analyses. The strength of association between trial funding and reporting of subgroups differed in trials with and without statistically significant primary outcomes (interaction P=0.02). In trials without statistically significant results for the primary outcome, industry funded trials were more likely to report subgroup analyses (2.29, 1.30 to 4.72) than non-industry funded trials. This was not true for trials with a statistically significant primary outcome (0.79, 0.46 to 1.36). Industry funded trials were associated with less frequent prespecification of subgroup hypotheses (31.3% v 38.0%, adjusted odds ratio 0.49, 0.26 to 0.94), and less use of the interaction test for analyses of subgroup effects (41.4% v 49.1%, 0.52, 0.28 to 0.97) than non-industry funded trials.ConclusionIndustry funded randomised controlled trials, in the absence of statistically significant primary outcomes, are more likely to report subgroup analyses than non-industry funded trials. Industry funded trials less frequently prespecify subgroup hypotheses and less frequently test for interaction than non-industry funded trials. Subgroup analyses from industry funded trials with negative results for the primary outcome should be viewed with caution.

Project description:ObjectiveTo improve communication of harm in publications of randomised controlled trials via the development of recommendations for visually presenting harm outcomes.DesignConsensus study.Setting15 clinical trials units registered with the UK Clinical Research Collaboration, an academic population health department, Roche Products, and The BMJ.ParticipantsExperts in clinical trials: 20 academic statisticians, one industry statistician, one academic health economist, one data graphics designer, and two clinicians.Main outcomemeasures A methodological review of statistical methods identified visualisations along with those recommended by consensus group members. Consensus on visual recommendations was achieved (at least 60% of the available votes) over a series of three meetings with participants. The participants reviewed and critically appraised candidate visualisations against an agreed framework and voted on whether to endorse each visualisation. Scores marginally below this threshold (50-60%) were revisited for further discussions and votes retaken until consensus was reached.Results28 visualisations were considered, of which 10 are recommended for researchers to consider in publications of main research findings. The choice of visualisations to present will depend on outcome type (eg, binary, count, time-to-event, or continuous), and the scenario (eg, summarising multiple emerging events or one event of interest). A decision tree is presented to assist trialists in deciding which visualisations to use. Examples are provided of each endorsed visualisation, along with an example interpretation, potential limitations, and signposting to code for implementation across a range of standard statistical software. Clinician feedback was incorporated into the explanatory information provided in the recommendations to aid understanding and interpretation.ConclusionsVisualisations provide a powerful tool to communicate harms in clinical trials, offering an alternative perspective to the traditional frequency tables. Increasing the use of visualisations for harm outcomes in clinical trial manuscripts and reports will provide clearer presentation of information and enable more informative interpretations. The limitations of each visualisation are discussed and examples of where their use would be inappropriate are given. Although the decision tree aids the choice of visualisation, the statistician and clinical trial team must ultimately decide the most appropriate visualisations for their data and objectives. Trialists should continue to examine crude numbers alongside visualisations to fully understand harm profiles.

Project description:BackgroundClinical trials are one of the most important sources of evidence for guiding evidence-based practice and the design of new trials. However, most of this information is available only in free text - e.g., in journal publications - which is labour intensive to process for systematic reviews, meta-analyses, and other evidence synthesis studies. This paper presents an automatic information extraction system, called ExaCT, that assists users with locating and extracting key trial characteristics (e.g., eligibility criteria, sample size, drug dosage, primary outcomes) from full-text journal articles reporting on randomized controlled trials (RCTs).MethodsExaCT consists of two parts: an information extraction (IE) engine that searches the article for text fragments that best describe the trial characteristics, and a web browser-based user interface that allows human reviewers to assess and modify the suggested selections. The IE engine uses a statistical text classifier to locate those sentences that have the highest probability of describing a trial characteristic. Then, the IE engine's second stage applies simple rules to these sentences to extract text fragments containing the target answer. The same approach is used for all 21 trial characteristics selected for this study.ResultsWe evaluated ExaCT using 50 previously unseen articles describing RCTs. The text classifier (first stage) was able to recover 88% of relevant sentences among its top five candidates (top5 recall) with the topmost candidate being relevant in 80% of cases (top1 precision). Precision and recall of the extraction rules (second stage) were 93% and 91%, respectively. Together, the two stages of the extraction engine were able to provide (partially) correct solutions in 992 out of 1050 test tasks (94%), with a majority of these (696) representing fully correct and complete answers.ConclusionsOur experiments confirmed the applicability and efficacy of ExaCT. Furthermore, they demonstrated that combining a statistical method with 'weak' extraction rules can identify a variety of study characteristics. The system is flexible and can be extended to handle other characteristics and document types (e.g., study protocols).

Project description:BackgroundSubgroup analyses are frequently used to assess heterogeneity of treatment effects in randomised clinical trials. Inconsistent, improper and incomplete implementation, reporting and interpretation have been identified as ongoing challenges. Further, subgroup analyses were frequently criticised because of unreliable or potentially misleading results. More recently, recommendations and guidelines have been provided to improve the reporting of data in this regard.MethodsThis systematic review was based on a literature search within the digital archives of three selected medical journals, The New England Journal of Medicine, The Lancet and Circulation. We reviewed articles of randomised clinical trials in the domain of cardiovascular disease which were published in 2015 and 2016. We screened and evaluated the selected articles for the mode of implementation and reporting of subgroup analyses.ResultsWe were able to identify a total of 130 eligible publications of randomised clinical trials. In 89/130 (68%) articles, results of at least one subgroup analysis were presented. This was dependent on the considered journal (p < 0.001), the number of included patients (p < 0.001) and the lack of statistical significance of a trial's primary analysis (p < 0.001). The number of reported subgroup analyses ranged from 1 to 101 (median = 13). We were able to comprehend the specification time of reported subgroup analyses for 71/89 (80%) articles, with 55/89 (62%) articles presenting exclusively pre-specified analyses. This information was not always traceable on the basis of provided trial protocols and often did not include the pre-definition of cut-off values for the categorization of subgroups. The use of interaction tests was reported in 84/89 (94%) articles, with 36/89 (40%) articles reporting heterogeneity of the treatment effect for at least one primary or secondary trial outcome. Subgroup analyses were reported more frequently for larger randomised clinical trials, and if primary analyses did not reach statistical significance. Information about the implementation of subgroup analyses was reported most consistently for articles from The New England Journal of Medicine, since it was also traceable on the basis of provided trial protocols. We were able to comprehend whether subgroup analyses were pre-specified in a majority of the reviewed publications. Even though results of multiple subgroup analyses were reported for most published trials, a corresponding adjustment for multiple testing was rarely considered.ConclusionCompared to previous reviews in this context, we observed improvements in the reporting of subgroup analyses of cardiovascular randomised clinical trials. Nonetheless, critical shortcomings, such as inconsistent reporting of the implementation and insufficient pre-specification, persist.

Dataset Information

Subgroup analyses in randomised controlled trials: cohort study on trial protocols and journal publications.

Objective

Design

Setting

Data sources

Results

Conclusions

Publications

Subgroup analyses in randomised controlled trials: cohort study on trial protocols and journal publications.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets