Dataset Information

Very Short Answer Questions in Team-Based Learning: Limited Effect on Peer Elaboration and Memory.

ABSTRACT:

Introduction

In team-based learning (TBL), single best answer questions (SBAQs) are traditionally used because immediate computer-assisted feedback facilitates team discussions. Recent improvements in digital marking systems and criticisms of non-analytical strategies in SBAQs have prompted the consideration of very short answer questions (VSAQs) as an alternative to SBAQs. We aim to compare the effect of VSAQs and SBAQs on peer elaboration and knowledge retention in TBL.

Materials and methods

Twenty-four second-year students from the Lee Kong Chian School of Medicine participated in a mixed-methods study that included a randomised controlled crossover trial with two intervention arms (TBL-VSAQs and TBL-SBAQs). Two TBL sessions were conducted, with one topic covered each. Students were randomly allocated into six teams of four members. Individual and team scores and completion times were measured, and students were surveyed on their TBL experience. A follow-up quiz on the same topics was administered two weeks later to assess knowledge retention.

Results

Individuals scored lower for VSAQs than SBAQs in the second TBL topic (7.17 ± 1.52 versus 8.25 ± 1.48; p = 0.046), while findings in other metrics were non-significant. Follow-up quiz scores showed no significant difference in knowledge retention, although effect size and power were low. Students perceived VSAQs as more authentic and challenging, though most preferred the continued use of SBAQs for TBL.

Discussion

VSAQs have a limited impact on peer elaboration and knowledge retention versus SBAQs in TBL.

Conclusions

We should be circumspect about implementing VSAQs in TBL especially for medical students at the pre-clinical level.

Supplementary information

The online version contains supplementary material available at 10.1007/s40670-022-01716-5.

SUBMITTER: Lau KY

PROVIDER: S-EPMC9765373 | biostudies-literature | 2022 Dec

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Very Short Answer Questions in Team-Based Learning: Limited Effect on Peer Elaboration and Memory.

Lau Kai Yuan KY Ang Jessica Yang Huey JYH Rajalingam Preman P

Medical science educator 20221220 1

<h4>Introduction</h4>In team-based learning (TBL), single best answer questions (SBAQs) are traditionally used because immediate computer-assisted feedback facilitates team discussions. Recent improvements in digital marking systems and criticisms of non-analytical strategies in SBAQs have prompted the consideration of very short answer questions (VSAQs) as an alternative to SBAQs. We aim to compare the effect of VSAQs and SBAQs on peer elaboration and knowledge retention in TBL.<h4>Materials an ...[more]

PMID: 36569367

Similar Datasets

Project description:Multiple choice questions (MCQs) offer high reliability and easy machine-marking, but allow for cueing and stimulate recognition-based learning. Very short answer questions (VSAQs), which are open-ended questions requiring a very short answer, may circumvent these limitations. Although VSAQ use in medical assessment increases, almost all research on reliability and validity of VSAQs in medical education has been performed by a single research group with extensive experience in the development of VSAQs. Therefore, we aimed to validate previous findings about VSAQ reliability, discrimination, and acceptability in undergraduate medical students and teachers with limited experience in VSAQs development. To validate the results presented in previous studies, we partially replicated a previous study and extended results on student experiences. Dutch undergraduate medical students (n = 375) were randomized to VSAQs first and MCQs second or vice versa in a formative exam in two courses, to determine reliability, discrimination, and cueing. Acceptability for teachers (i.e., VSAQ review time) was determined in the summative exam. Reliability (Cronbach's α) was 0.74 for VSAQs and 0.57 for MCQs in one course. In the other course, Cronbach's α was 0.87 for VSAQs and 0.83 for MCQs. Discrimination (average Rir) was 0.27 vs. 0.17 and 0.43 vs. 0.39 for VSAQs vs. MCQs, respectively. Reviewing time of one VSAQ for the entire student cohort was ±2 minutes on average. Positive cueing occurred more in MCQs than in VSAQs (20% vs. 4% and 20.8% vs. 8.3% of questions per person in both courses). This study validates the positive results regarding VSAQs reliability, discrimination, and acceptability in undergraduate medical students. Furthermore, we demonstrate that VSAQ use is reliable among teachers with limited experience in writing and marking VSAQs. The short learning curve for teachers, favourable marking time and applicability regardless of the topic suggest that VSAQs might also be valuable beyond medical assessment.

Project description:OBJECTIVES:The study aimed to compare candidate performance between traditional best-of-five single-best-answer (SBA) questions and very-short-answer (VSA) questions, in which candidates must generate their own answers of between one and five words. The primary objective was to determine if the mean positive cue rate for SBAs exceeded the null hypothesis guessing rate of 20%. DESIGN:This was a cross-sectional study undertaken in 2018. SETTING:20 medical schools in the UK. PARTICIPANTS:1417 volunteer medical students preparing for their final undergraduate medicine examinations (total eligible population across all UK medical schools approximately 7500). INTERVENTIONS:Students completed a 50-question VSA test, followed immediately by the same test in SBA format, using a novel digital exam delivery platform which also facilitated rapid marking of VSAs. MAIN OUTCOME MEASURES:The main outcome measure was the mean positive cue rate across SBAs: the percentage of students getting the SBA format of the question correct after getting the VSA format incorrect. Internal consistency, item discrimination and the pass rate using Cohen standard setting for VSAs and SBAs were also evaluated, and a cost analysis in terms of marking the VSA was performed. RESULTS:The study was completed by 1417 students. Mean student scores were 21 percentage points higher for SBAs. The mean positive cue rate was 42.7% (95% CI 36.8% to 48.6%), one-sample t-test against ≤20%: t=7.53, p<0.001. Internal consistency was higher for VSAs than SBAs and the median item discrimination equivalent. The estimated marking cost was £2655 ($3500), with 24.5 hours of clinician time required (1.25 s per student per question). CONCLUSIONS:SBA questions can give a false impression of students' competence. VSAs appear to have greater authenticity and can provide useful information regarding students' cognitive errors, helping to improve learning as well as assessment. Electronic delivery and marking of VSAs is feasible and cost-effective.

Dataset Information

Very Short Answer Questions in Team-Based Learning: Limited Effect on Peer Elaboration and Memory.

Introduction

Materials and methods

Results

Discussion

Conclusions

Supplementary information

Publications

Very Short Answer Questions in Team-Based Learning: Limited Effect on Peer Elaboration and Memory.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets