Dataset Information

Expert validation of fit-for-purpose guidelines for designing programmes of assessment.

ABSTRACT:

Background

An assessment programme, a purposeful mix of assessment activities, is necessary to achieve a complete picture of assessee competence. High quality assessment programmes exist, however, design requirements for such programmes are still unclear. We developed guidelines for design based on an earlier developed framework which identified areas to be covered. A fitness-for-purpose approach defining quality was adopted to develop and validate guidelines.

Methods

First, in a brainstorm, ideas were generated, followed by structured interviews with 9 international assessment experts. Then, guidelines were fine-tuned through analysis of the interviews. Finally, validation was based on expert consensus via member checking.

Results

In total 72 guidelines were developed and in this paper the most salient guidelines are discussed. The guidelines are related and grouped per layer of the framework. Some guidelines were so generic that these are applicable in any design consideration. These are: the principle of proportionality, rationales should underpin each decisions, and requirement of expertise. Logically, many guidelines focus on practical aspects of assessment. Some guidelines were found to be clear and concrete, others were less straightforward and were phrased more as issues for contemplation.

Conclusions

The set of guidelines is comprehensive and not bound to a specific context or educational approach. From the fitness-for-purpose principle, guidelines are eclectic, requiring expertise judgement to use them appropriately in different contexts. Further validation studies to test practicality are required.

SUBMITTER: Dijkstra J

PROVIDER: S-EPMC3676146 | biostudies-literature | 2012 Apr

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Expert validation of fit-for-purpose guidelines for designing programmes of assessment.

Dijkstra Joost J Galbraith Robert R Hodges Brian D BD McAvoy Pauline A PA McCrorie Peter P Southgate Lesley J LJ Van der Vleuten Cees P M CP Wass Val V Schuwirth Lambert W T LW

BMC medical education 20120417

<h4>Background</h4>An assessment programme, a purposeful mix of assessment activities, is necessary to achieve a complete picture of assessee competence. High quality assessment programmes exist, however, design requirements for such programmes are still unclear. We developed guidelines for design based on an earlier developed framework which identified areas to be covered. A fitness-for-purpose approach defining quality was adopted to develop and validate guidelines.<h4>Methods</h4>First, in a ...[more]

PMID: 22510502

Similar Datasets

Project description:The organization and function of the institutional animal care and use committee (IACUC) is the key component of government regulation and oversight of necessary scientific research using live animals and of AAALAC - International accreditation of animal care and use programs in the United States. The regulations, roles, and responsibilities of IACUCs have evolved since their inception 35 years ago from a limited focus on animal welfare and specific animal procedures to embracing scientific quality, data reproducibility and translation, and animal welfare as inextricably interdependent and critical components of generation of new scientific knowledge and medical treatments. A current challenge for IACUCs is in evaluating whether benefits to be derived (eg, new knowledge or treatments) justify any unavoidable pain, stress, or injury associated with proposed research protocols, because the former are long-term and at best speculative outcomes, whereas the latter are immediate and tangible for the study animals. Scientific consensus is that research most likely to generate significant new knowledge and medical treatments is that conducted to high scientific, technical, and quality standards and reported with full transparency to facilitate reproducibility. As an alternative to current benefits evaluations included in risk benefit and harm benefit constructs, the authors propose that IACUCs assess the proposed research for scientific quality and alignment of study elements with the study purpose (e.g., Fit for Purpose [FfP]), including justifications for study design components, selection of primary endpoints and technologies, rationale for data and statistical analyses, and research communication plans. Fit for Purpose endpoints are objective, immediate, and impactful as are the potential risks for study animals, and at the same time they are the best predictors for achievement of longer-term benefits. We propose that IACUCs and any revision of The ILAR Guide consider FfP concepts in place of traditional benefits assessment to accelerate the generation of new knowledge and treatments benefiting medical and veterinary patients and the environment through better science and animal welfare rather than to continue to rely on speculative future outcomes.

Project description:BACKGROUND: Mass spectrometry (MS) is a very sensitive and specific method for protein identification, biomarker discovery, and biomarker validation. Protein identification is commonly carried out by comparing MS data with public databases. However, with the development of high throughput and accurate genomic sequencing technology, public databases are being overwhelmed with new entries from different species every day. The application of these databases can also be problematic due to factors such as size, specificity, and unharmonized annotation of the molecules of interest. Current databases representing liquid chromatography-tandem mass spectrometry (LC-MS/MS)-based searches focus on enzyme digestion patterns and sequence information and consequently, important functional information can be missed within the search output. Protein variants displaying similar sequence homology can interfere with database identification when only certain homologues are examined. In addition, recombinant DNA technology can result in products that may not be accurately annotated in public databases. Curated databases, which focus on the molecule of interest with clearer functional annotation and sequence information, are necessary for accurate protein identification and validation. Here, four cases of curated database application have been explored and summarized. FINDINGS: The four presented curated databases were constructed with clear goals regarding application and have proven very useful for targeted protein identification and biomarker application in different fields. They include a sheeppox virus database created for accurate identification of proteins with strong antigenicity, a custom database containing clearly annotated protein variants such as tau transcript variant 2 for accurate biomarker identification, a sheep-hamster chimeric prion protein (PrP) database constructed for assay development of prion diseases, and a custom Escherichia coli (E. coli) flagella (H antigen) database produced for MS-H, a new H-typing technique. Clearly annotating the proteins of interest was essential for highly accurate, specific, and sensitive sequence identification, and searching against public databases resulted in inaccurate identification of the sequence of interest, while combining the curated database with a public database reduced both the confidence and sequence coverage of the protein search. CONCLUSION: Curated protein sequence databases incorporating clear annotations are very useful for accurate protein identification and fit-for-purpose application through MS-based biomarker validation.

Project description:Background: The freezing process of tissue samples is crucial for the preservation of morphological and molecular features. Several biobanking guidelines describe freezing techniques for optimal outcomes. As the Vetbiobank standard freezing protocol does not comply with those recommendations in detail, a process validation was performed to demonstrate that samples are suitable for downstream applications. Here we give a formal example of a process validation in the biobanking setting, as required by the biobanking guideline ISO 20387 (2018). Methods: Three different freezing protocols, freezing in liquid nitrogen, freezing via isopentane precooled on dry ice and freezing via liquid nitrogen vapor, were assessed based on morphological integrity of mouse liver and muscle tissue samples. Samples were either frozen in cryotubes (without Optimal Cutting Temperature compound, OCT) or in cryomolds (with OCT). The protocol providing the best results was validated for reproducibility and robustness in terms of defined acceptance criteria for morphological evaluability, A260/A280 ratio, and RNA integrity number values (RIN). In addition, performance tests were run by gene expression analyzes of selected, tissue specific biomarkers to confirm that processed samples are fit for purpose. Results: From the three applied freezing protocols, freezing in liquid nitrogen generated best results. Reproducibility acceptance criteria were met for both, morphological integrity and RNA quality. The freezing method was robust for the tested tissue types and the application of OCT, with exception of liver tissue, where it led to a significant decrease of the RIN value. Gene expression analyzes showed good comparability of results regardless of the applied freezing method. Conclusion: Freezing of tissue samples in liquid nitrogen provides samples of adequate quality for subsequent RNA investigations. A negative impact of OCT on the RIN value of liver samples was observed, which was independent from the applied freezing protocol and showed no impact on subsequent gene expression analysis.

Dataset Information

Expert validation of fit-for-purpose guidelines for designing programmes of assessment.

Background

Methods

Results

Conclusions

Publications

Expert validation of fit-for-purpose guidelines for designing programmes of assessment.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets