Dataset Information

What should the standard be for passing and mastery on the Critical Thinking about Health Test? A consensus study.

ABSTRACT:

Objective

Most health literacy measures rely on subjective self-assessment. The Critical Thinking about Health Test is an objective measure that includes two multiple-choice questions (MCQs) for each of the nine Informed Health Choices Key Concepts included in the educational resources for secondary schools. The objective of this study was to determine cut-off scores for passing (the border between having and not having a basic understanding and the ability to apply the nine concepts) and mastery (the border between having mastered and not having mastered them).

Design

Using a combination of two widely used methods: Angoff's and Nedelsky's, a panel judged the likelihood that an individual on the border of passing and another on the border of having mastered the concepts would answer each MCQ correctly. The cut-off scores were determined by summing up the probability of answering each MCQ correctly. Their independent assessments were summarised and discussed. A nominal group technique was used to reach a consensus.

Setting

The study was conducted in secondary schools in East Africa.

Participants

The panel included eight individuals with 5 or more years' experience in the following areas: evaluation of critical thinking interventions, curriculum development, teaching of lower secondary school and evidence-informed decision-making.

Results

The panel agreed that for a passing score, students had to answer 9 of the 18 questions and for a mastery score, 14 out of 18 questions correctly.

Conclusion

There was wide variation in the judgements made by individual panel members for many of the questions, but they quickly reached a consensus on the cut-off scores after discussions.

SUBMITTER: Nsangi A

PROVIDER: S-EPMC9972413 | biostudies-literature | 2023 Feb

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

What should the standard be for passing and mastery on the Critical Thinking about Health Test? A consensus study.

Nsangi Allen A Aranza Diana D Asimwe Roger R Munaabi-Babigumira Susan Kyomuhendo SK Nantongo Judith J Nordheim Lena Victoria LV Ochieng Robert R Oyuga Cyril C Uwimana Innocent I Dahlgren Astrid A Oxman Andrew A

BMJ open 20230224 2

<h4>Objective</h4>Most health literacy measures rely on subjective self-assessment. The Critical Thinking about Health Test is an objective measure that includes two multiple-choice questions (MCQs) for each of the nine Informed Health Choices Key Concepts included in the educational resources for secondary schools. The objective of this study was to determine cut-off scores for passing (the border between having and not having a basic understanding and the ability to apply the nine concepts) an ...[more]

PMID: 36828652

Similar Datasets

Project description:IntroductionWe evaluated the Informed Health Choices secondary school intervention in a cluster randomized trial in Rwanda. The intervention was effective in helping students to think critically about health. In parallel to the trial, we conducted a process evaluation to assess factors affecting the implementation, impacts, and scale-up of the intervention.MethodsWe used a mixed methods approach that included quantitative and qualitative methods. We collected quantitative data from teachers to evaluate the teacher training and each lesson. We conducted focus group discussions with students (n=10) and their parents/guardians (n=5). We conducted lesson observations (n=16) and key informant interviews with teachers (n=10) and school administrators (n=10) from intervention schools and policymakers (n=2). We analyzed the quantitative data using descriptive statistics. We used framework analysis and thematic content analysis to analyze the qualitative data.ResultsTeachers noted that the teacher training supported their delivery of the intervention and that they made only small adaptations to fit student, teacher, or contextual needs. Students reported obtaining important skills, including recognizing health claims, understanding the need for research, and "thinking twice" before deciding. Participants saw the design of the intervention, students' and teachers' motivation, and school and home support as key facilitators for the implementation and impact of the intervention. Implementation barriers identified included the content of the lessons not being included in national examinations, competing priorities, and time constraints. Participants identified several factors that could facilitate intervention scale-up, including the need for the skills taught in the lessons and compatibility of the intervention with the national curriculum.ConclusionWe found that it was feasible to implement the intervention in Rwandan secondary schools and that students benefited from the intervention. Scaling up the intervention will likely require addressing the barriers identified in this study.

Project description:IntroductionWe evaluated the Informed Health Choices secondary school intervention to help students in Kenya think critically about health choices. We conducted this process evaluation to explore if the intervention was implemented as planned, identify factors that facilitated or hindered implementation, potential benefits of the intervention, and how to scale up the intervention beyond the trial.MethodsThis was a mixed methods process evaluation nested in a cluster-randomized trial of the Informed Health Choices intervention. We analyzed quantitative data from teacher training evaluation forms completed by 39 teachers, 10 lesson evaluation forms completed by 40 teachers allocated to the intervention, and 72 structured classroom observation forms. We conducted a framework analysis of qualitative data from 14 group interviews (with 96 students, 23 teachers, and 18 parents) and 22 individual interviews (with 8 teachers, 5 school principals, 6 curriculum developers, and 3 policymakers). We assessed confidence in our findings from the qualitative analysis using a modified version of Confidence in the Evidence from Reviews of Qualitative Research.ResultsLesson objectives were achieved with minimal adaptations. Factors that might have facilitated the implementation of the intervention include teacher training; perceived value of the intervention by students, teachers, and policymakers; and support from school administration. Time constraints, teachers' heavy workloads, and the lessons not being included in the curriculum or national examination are factors that might have impeded implementation. Both students and teachers demonstrated the ability to apply key concepts that were taught to health choices and other choices. However, they experienced difficulties with 2 of the lessons.ConclusionScale-up of this intervention in Kenyan schools is feasible but may depend on adjusting the time allocated to teaching the lessons, modifying the 2 lessons that teachers and students found difficult, and including the lesson objectives and assessment in the national curriculum.

Dataset Information

What should the standard be for passing and mastery on the Critical Thinking about Health Test? A consensus study.

Objective

Design

Setting

Participants

Results

Conclusion

Publications

What should the standard be for passing and mastery on the Critical Thinking about Health Test? A consensus study.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets