Grade Expectations: Rationality and Overconfidence.
ABSTRACT: Confidence and overconfidence are essential aspects of human nature, but measuring (over)confidence is not easy. Our approach is to consider students' forecasts of their exam grades. Part of a student's grade expectation is based on the student's previous academic achievements; what remains can be interpreted as (over)confidence. Our results are based on a sample of about 500 second-year undergraduate students enrolled in a statistics course in Moscow. The course contains three exams and each student produces a forecast for each of the three exams. Our models allow us to estimate overconfidence quantitatively. Using these models we find that students' expectations are not rational and that most students are overconfident, in agreement with the general literature. Less obvious is that overconfidence helps: given the same academic achievement students with larger confidence obtain higher exam grades. Female students are less overconfident than male students, their forecasts are more rational, and they are also faster learners in the sense that they adjust their expectations more rapidly.
Project description:<h4>Background</h4>To evaluate the scientific reasoning in basic science among undergraduate medical students, we established the National Medical Science Olympiad in Iran. In this Olympiad, the drawing of a concept map was used to evaluate a student's knowledge framework; students' ability in hypothesis generation and testing were also evaluated in four different steps. All medical students were invited to participate in this program. Finally, 133 undergraduate medical students with average grades ≥ 16/20 from 45 different medical schools in Iran were selected. The program took the form of four exams: drawing a concept map (Exam I), hypothesis generation (Exam II), choosing variables based on the hypothesis (Exam III), measuring scientific thought (Exam IV). The examinees were asked to complete all examination items in their own time without using textbooks, websites, or personal consultations. Data were presented as mean ± SE of each parameter. The correlation coefficient between students' scores in each exam with the total final score and average grade was calculated using the Spearman test.<h4>Results</h4>Out of a possible score of 200, the mean ± SE of each exam were as follows: 183.88 ± 5.590 for Exam I; 78.68 ± 9.168 for Exam II; 92.04 ± 2.503 for exam III; 106.13 ± 2.345 for Exam IV. The correlation of each exam score with the total final score was calculated, and there was a significant correlation between them (p < 0.001). The scatter plot of the data showed a linear correlation between the score for each exam and the total final score. This meant that students with a higher final score were able to perform better in each exam through having drawn up a meaningful concept map.The average grade was significantly correlated with the total final score (R = 0.770), (p < 0.001). There was also a significant correlation between each exam score and the average grade (p < 0.001). The highest correlation was observed between Exam I (R = 0.7708) and the average grade. This means students with higher average grades had better grades in each exam, especially in drawing the concept map.<h4>Conclusions</h4>We hope that this competition will encourage medical schools to integrate theory and practice, analyze data, and read research articles. Our findings relate to a selected population, and our data may not be applicable to all medical students. Therefore, further studies are required to validate our results.
Project description:Motivated by the self-determination theory of psychology, we investigate how simple school practices can forge students' engagement with the academic aspect of school life. We carried out a large-scale preregistered randomized field experiment with a crossover design, involving all the students of the University of Szeged in Hungary. Our intervention consisted of an automated encouragement message that praised students' past achievements and signaled trust in their success. The treated students received encouragement messages before their exam via two channels: e-mail and SMS message. The control students did not receive any encouragement. Our primary analysis compared the treated and control students' end-of-semester exam grades, obtained from the university's registry. Our secondary analysis explored the difference between the treated and control students' self-efficacy, motivation, and test anxiety, obtained from an online survey before students' exams. We did not find an average treatment effect on students' exam grades. However, in the subsample of those who answered the endline survey, the treated students reported higher self-efficacy than the control students. The treatment affected students' motivation before their first exam-but not before their second-and did not affect students' test anxiety. Our results indicate that automated encouragement messages sent shortly before exams do not boost students' exam grades, but they do increase self-efficacy. These results contribute to understanding the self-efficacy mechanism through which future encouragement campaigns might exert their effect. We conclude that encouraging students and raising their self-efficacy might create a school climate that better engages students with the academic aspect of school life.
Project description:<h4>Background</h4>Since the emergence of coronavirus disease 2019 (Covid-19), distance education has been extensively implemented in all educational institutes and remote electronic exams (E-exams) have been adopted as a primary mode of assessment.<h4>Objectives</h4>This cross-sectional study aimed to evaluate the experience of students at faculties of Medicine, Dentistry, Pharmacy, Nursing and Applied Medical Sciences at Jordan University of Science and Technology regarding remote E-exams preferences and academic dishonesty during the pandemic.<h4>Materials and methods</h4>The survey composed of 16 questions, prepared using Google forms and distributed through students' E-learning platforms. The survey explored factors affecting students' preference for remote E-exams, methods for course assessment/evaluation, factors related to students' exam dishonesty/misconduct during remote E-exams and measures that can be considered to reduce this behavior. Data were analyzed using descriptive, cross tabulation and Chi-square tests.<h4>Results</h4>Among 730 students, approximately only one third preferred remote E-exams. This was significantly (P < 0.05) associated with academic major, efforts/time for remote E-exam preparation, questions appropriateness with study material, and academic achievements (students Grade Point Average (GPA), curriculum objectives). Combining both exams and quizzes was the most preferred method of assessment (30%), while submission of reports or short written assignments were the least preferred ones. Exam dishonesty/misconduct appears as one of the major challenges with remote E-exams. The main measures considered by students to reduce exam dishonesty included substituting the exam with other forms of assessment, using different exam forms, the use of online proctoring solutions and considering compulsory pass/fail grades.<h4>Conclusion</h4>Results suggested less preference of remote E-exams among students at medical faculties. Findings from this study are highly valuable to plan for academic strategies to overcome difficulties and challenges of remote E-exams. These might include improvement for the distance teaching methodologies, rearrangement of assessment options, modification of the academic curriculum to fit the current situation, and adopting certain measures to prevent exam dishonesty and maintain academic integrity.
Project description:National calls for teaching transformation build on a constructivist learning theory and propose that students learn by actively engaging in course activities and interacting with other students. While interactive pedagogies can improve learning, they also have the potential to challenge traditional norms regarding class participation and learning strategies. To better understand the potential openness of students to interactive teaching practices, we administered a survey during the first week of two sections of an introductory biology course to characterize how students envisioned spending time during class as well as what activities they expected to complete outside of class during non-exam weeks and in preparation for exams. Additionally, we sought to test the hypothesis that the expectations of first-year students differed from those of non-first-year students. Analyses of closed-ended and open-ended questions revealed that students held a wide range of expectations and that most students expressed expectations consistent with some degree of transformed teaching. Furthermore, first-year students expected more active learning in class, more out-of-class coursework during non-exam weeks, and more social learning strategies than non-first-year students. We discuss how instructor awareness of incoming student expectations might be used to promote success in introductory science courses.
Project description:Adequate knowledge in pharmacology is crucial in many professions but a non-negligible proportion of students fail the exams and knowledge of underlying factors is largely lacking. This study was performed to evaluate to what extent various factors are related to student performance in pharmacology-related courses in higher education, linking administrative data to attendance at non-mandatory teaching sessions and questionnaire replies. A total of 596 students (median age: 22 years; 70% female) were included from eight courses which are part of either the medical, pharmacy, dentistry, nursing, or biomedical analyst degree programs at the Sahlgrenska Academy, Gothenburg, Sweden. In all, 380 (64%) students passed the regular program- and course-specific exam. Multivariate logistic regression analysis revealed that a high participation rate in non-mandatory teaching sessions, as well as a perceived great interest in pharmacology, was associated with students' passing of the exam; adjusted odds ratio (95% confidence interval): 1.30 (1.19 to 1.42; per 10 percentage unit increase in attendance) and 3.38 (1.86 to 6.12), respectively. Working for wages during the course weeks and pre-university grades used in the program application were significant factors in subgroups of students, negatively and positively associated with the exam results, respectively. Age, having Swedish as a second language, and time spent studying were only associated with the exam result in the univariate analyses. To conclude, both students and teachers can contribute significantly to successful education within pharmacology, students by participating in the teaching sessions and teachers by encouraging students to find the subject interesting.
Project description:BACKGROUND:Children in the UK go through rigorous teacher assessments and standardized exams throughout compulsory (elementary and secondary) education, culminating with the GCSE exams (General Certificate of Secondary Education) at the age of 16 and A-level exams (Advanced Certificate of Secondary Education) at the age of 18. These exams are a major tipping point directing young individuals towards different lifelong trajectories. However, little is known about the associations between teacher assessments and exam performance or how well these two measurement approaches predict educational outcomes at the end of compulsory education and beyond. METHODS:The current investigation used the UK-representative Twins Early Development Study (TEDS) sample of over 5,000 twin pairs studied longitudinally from childhood to young adulthood (age 7-18). We used teacher assessment and exam performance across development to investigate, using genetically sensitive designs, the associations between teacher assessment and standardized exam scores, as well as teacher assessments' prediction of exam scores at ages 16 and 18, and university enrolment. RESULTS:Teacher assessments of achievement are as reliable, stable and heritable (~60%) as test scores at every stage of the educational experience. Teacher and test scores correlate strongly phenotypically (r ~ .70) and genetically (genetic correlation ~.80) both contemporaneously and over time. Earlier exam performance accounts for additional variance in standardized exam results (~10%) at age 16, when controlling for teacher assessments. However, exam performance explains less additional variance in later academic success, ~5% for exam grades at 18, and ~3% for university entry, when controlling for teacher assessments. Teacher assessments also predict additional variance in later exam performance and university enrolment, when controlling for previous exam scores. CONCLUSIONS:Teachers can reliably and validly monitor students' progress, abilities and inclinations. High-stakes exams may shift educational experience away from learning towards exam performance. For these reasons, we suggest that teacher assessments could replace some, or all, high-stakes exams.
Project description:<h4>Background</h4>Misplaced or poorly calibrated confidence in healthcare professionals' judgments compromises the quality of health care. Using higher fidelity clinical simulations to elicit clinicians' confidence 'calibration' (i.e. overconfidence or underconfidence) in more realistic settings is a promising but underutilized tactic. In this study we examine nurses' calibration of confidence with judgment accuracy for critical event risk assessment judgments in a high fidelity simulated clinical environment. The study also explores the effects of clinical experience, task difficulty and time pressure on the relationship between confidence and accuracy.<h4>Methods</h4>63 student and 34 experienced nurses made dichotomous risk assessments on 25 scenarios simulated in a high fidelity clinical environment. Each nurse also assigned a score (0-100) reflecting the level of confidence in their judgments. Scenarios were derived from real patient cases and classified as easy or difficult judgment tasks. Nurses made half of their judgments under time pressure. Confidence calibration statistics were calculated and calibration curves generated.<h4>Results</h4>Nurse students were underconfident (mean over/underconfidence score -1.05) and experienced nurses overconfident (mean over/underconfidence score 6.56), P = 0.01. No significant differences in calibration and resolution were found between the two groups (P = 0.80 and P = 0.51, respectively). There was a significant interaction between time pressure and task difficulty on confidence (P = 0.008); time pressure increased confidence in easy cases but reduced confidence in difficult cases. Time pressure had no effect on confidence or accuracy. Judgment task difficulty impacted significantly on nurses' judgmental accuracy and confidence. A 'hard-easy' effect was observed: nurses were overconfident in difficult judgments and underconfident in easy judgments.<h4>Conclusion</h4>Nurses were poorly calibrated when making risk assessment judgments in a high fidelity simulated setting. Nurses with more experience tended toward overconfidence. Whilst time pressure had little effect on calibration, nurses' over/underconfidence varied significantly with the degree of task difficulty. More research is required to identify strategies to minimize such cognitive biases.
Project description:In college introductory science courses, students are challenged with mastering large amounts of disciplinary content while developing as autonomous and effective learners. Self-regulated learning (SRL) is the process of setting learning goals, monitoring progress toward them, and applying appropriate study strategies. SRL characterizes successful, "expert" learners, and develops with time and practice. In a large, undergraduate introductory biology course, we investigated: 1) what SRL strategies students reported using the most when studying for exams, 2) which strategies were associated with higher achievement and with grade improvement on exams, and 3) what study approaches students proposed to use for future exams. Higher-achieving students, and students whose exam grades improved in the first half of the semester, reported using specific cognitive and metacognitive strategies significantly more frequently than their lower-achieving peers. Lower-achieving students more frequently reported that they did not implement their planned strategies or, if they did, still did not improve their outcomes. These results suggest that many students entering introductory biology have limited knowledge of SRL strategies and/or limited ability to implement them, which can impact their achievement. Course-specific interventions that promote SRL development should be considered as integral pedagogical tools, aimed at fostering development of students' lifelong learning skills.
Project description:The introduction of computer-based testing in high-stakes examining in higher education is developing rather slowly due to institutional barriers (the need of extra facilities, ensuring test security) and teacher and student acceptance. From the existing literature it is unclear whether computer-based exams will result in similar results as paper-based exams and whether student acceptance can change as a result of administering computer-based exams. In this study, we compared results from a computer-based and paper-based exam in a sample of psychology students and found no differences in total scores across the two modes. Furthermore, we investigated student acceptance and change in acceptance of computer-based examining. After taking the computer-based exam, fifty percent of the students preferred paper-and-pencil exams over computer-based exams and about a quarter preferred a computer-based exam. We conclude that computer-based exam total scores are similar as paper-based exam scores, but that for the acceptance of high-stakes computer-based exams it is important that students practice and get familiar with this new mode of test administration.
Project description:Emergence of coronavirus disease 2019 (COVID-19) forced the worldwide higher educational institutes to adopt distance learning mode. Further, remote electronic exams (E-exams) were considered as mode of assessment. <b>Objectives</b>: This cross-sectional study evaluated the students' experience of remote E-exams during the COVID-19 pandemic among Medical Sciences students in Jordan. <b>Materials and Methods:</b> A survey of 29 questions was prepared on Google forms and distributed among students at Faculties of Medical Sciences (Medicine, Dentistry, Pharmacy, Nursing and Applied Medical Sciences) at Jordan University of Science and Technology. The questions include students' demographics, stress experience, and factors contributing to stress as well as behavioral changes related to remote E-exams. Responses were analyzed using descriptive, cross tabulation and Chi-square tests. <b>Results:</b> Among 1019 respondents, 32% reported more stress with remote E-exams. This was associated with academic major and gender. Among students with more stress during remote E-exams, the exam duration, mode of questions navigation and technical problems (exam platform and internet connectivity) appeared as the main factors related to stress in 78%, 76% and >60%, respectively. Other factors include concern regarding the teaching methods, exam environment and students' dishonesty. Remote E-exams had negative impact on students' dietary habits (increase consumption of caffeine and high energy drinks, high sugar food, fast food), sleep (reduction in sleeping hours, more consumption of insomnia medications), physical activity (less exercises) and smoking habits (increase). <b>Conclusion</b>: Results suggested a negative impact of E-exams on students within Medical Faculties. Robust exam platform and remote mock E-exams are recommended to reduce students' potential stress. A stress-free environment is very essential to encourage students to adopt remote E-exams, particularly if the pandemic will take longer. Various awareness programs about students' habits related to dietary, sleep quality, physical activity and smoking are highly valuable for students' health benefits. This is of particular importance since the current students at Faculties of Medical Sciences are the future health care providers.