Dataset Information

Giving away some of their powers! Towards learner agency in digital assessment and feedback.

ABSTRACT: Digital assessment and feedback have been a growing area of research and practice in the past decade in higher education. Within this theme, research has been published highlighting the importance of learner agency in the assessment and feedback process as a way to develop assessment literacy in contrast with the existing lecturer-led approach. In this research, we aimed to find out whether lecturers are willing to let go of some of the power they currently have in the digital assessment and feedback process and how they see opportunities for agency being developed in the digital assessment and feedback systems. We collected data from 10 sandpits with 58 lecturers in which, using a storytelling technique and one mock-up of a digital assessment and feedback system, we discussed and critiqued an assessment scenario intending to collect perceptions about digital assessment and feedback and the constraints felt by lecturers in their assessment practice. Based on these perceptions, we identify recommendations that may improve digital assessment and feedback systems and practices. We discuss the data and the recommendations based on three clusters of themes: (i) preparation for the assessment, (ii) formative feedback and (iii) feedback post-submission.

Supplementary information

The online version contains supplementary material available at 10.1186/s41039-021-00168-6.

SUBMITTER: Casanova D

PROVIDER: S-EPMC8261811 | biostudies-literature | 2021

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Giving away some of their powers! Towards learner agency in digital assessment and feedback.

Casanova Diogo D Alsop Graham G Huet Isabel I

Research and practice in technology enhanced learning 20210707 1

Digital assessment and feedback have been a growing area of research and practice in the past decade in higher education. Within this theme, research has been published highlighting the importance of learner agency in the assessment and feedback process as a way to develop assessment literacy in contrast with the existing lecturer-led approach. In this research, we aimed to find out whether lecturers are willing to let go of some of the power they currently have in the digital assessment and fee ...[more]

PMID: 34249154

Similar Datasets

Project description:ProblemWritten feedback is often overly positive, nonspecific, and difficult to interpret. Learner satisfaction with written feedback is low and obtaining written feedback that encourages self-reflection is challenging. Improving feedback quality is laborious and only modestly effective.ApproachThe authors developed the LEAF (Learner-Engaged Analysis of Feedback) method to improve learner satisfaction with, and reflection on, existing written feedback. The method pairs a learner and coach to methodically identify themes in the learner's written feedback. Themes occurring more frequently or less frequently than typical offer areas for reflection, as they may identify learners' relative strengths or weaknesses. The method was introduced at the Massachusetts General Hospital in 2017 during program director (PD) meetings with anesthesiology residents. In 2018, resident satisfaction was measured (1 to 5 Likert-type questions, 1 = "not at all satisfied," 5 = "extremely satisfied") for 4 feedback sources, 2 related to the LEAF method (PD meetings, written feedback) and 2 unrelated (verbal feedback, mentor feedback). Residents' comments were qualitatively assessed to explore the impact on self-reflection.OutcomesResidents who had participated in a LEAF session (n = 54), compared with those who had not (n = 11), reported higher satisfaction with written feedback (mean 3.1 versus 2.5, d = 0.53, P = .03) and PD meeting feedback (mean 3.8 versus 2.8, d = 0.80, P = .03). There were no significant differences between groups for satisfaction with feedback unrelated to the LEAF method. Qualitative analysis of comments suggested that residents found the method useful for providing holistic self-assessment, facilitating goal setting, uncovering blind spots, and improving feedback interpretation.Next stepsNext steps should include studies determining if the association between increased learner satisfaction with written feedback and the LEAF method is causal, and whether this feedback process changes learners' subsequent behaviors.

Project description:Digital mental health interventions (DMHI) have the potential to address barriers to face-to-face mental healthcare. In particular, digital mental health assessments offer the opportunity to increase access, reduce strain on services, and improve identification. Despite the potential of DMHIs there remains a high drop-out rate. Therefore, investigating user feedback may elucidate how to best design and deliver an engaging digital mental health assessment. The current study aimed to understand 1304 user perspectives of (1) a newly developed digital mental health assessment to determine which features users consider to be positive or negative and (2) the Composite International Diagnostic Interview (CIDI) employed in a previous large-scale pilot study. A thematic analysis method was employed to identify themes in feedback to three question prompts related to: (1) the questions included in the digital assessment, (2) the homepage design and reminders, and (3) the assessment results report. The largest proportion of the positive and negative feedback received regarding the questions included in the assessment (n = 706), focused on the quality of the assessment (n = 183, 25.92% and n = 284, 40.23%, respectively). Feedback for the homepage and reminders (n = 671) was overwhelmingly positive, with the largest two themes identified being positive usability (i.e., ease of use; n = 500, 74.52%) and functionality (i.e., reminders; n = 278, 41.43%). The most frequently identified negative theme in results report feedback (n = 794) was related to the report content (n = 309, 38.92%), with users stating it was lacking in-depth information. Nevertheless, the most frequent positive theme regarding the results report feedback was related to wellbeing outcomes (n = 145, 18.26%), with users stating the results report, albeit brief, encouraged them to seek professional support. Interestingly, despite some negative feedback, most users reported that completing the digital mental health assessment has been worthwhile (n = 1,017, 77.99%). Based on these findings, we offer recommendations to address potential barriers to user engagement with a digital mental health assessment. In summary, we recommend undertaking extensive co-design activities during the development of digital assessment tools, flexibility in answering modalities within digital assessment, customizable additional features such as reminders, transparency of diagnostic decision making, and an actionable results report with personalized mental health resources.

Project description:IntroductionIn the 21st century, digital devices have become integral to our daily lives. Still, practical assessments designed to evaluate an individual's digital tool competencies are absent. The present study introduces the "Digital Tools Test" ("DIGI"), specifically designed for the evaluation of one's proficiency in handling common applications and functions of smartphones and tablets. The DIGI assessment has been primarily tailored for prospective use among older adults and neurological patients with the latter frequently suffering from so-called apraxia, which potentially also affects the handling of digital tools. Similar to traditional tool use tests that assess tool-selection and tool-action processes, the DIGI assessment evaluates an individual's ability to select an appropriate application for a given task (e.g., creating a new contact), their capacity to navigate within the chosen application and their competence in executing precise and accurate movements, such as swiping.MethodsWe tested the implementation of the DIGI in a group of 16 healthy adults aged 18 to 28 years and 16 healthy adults aged 60 to 74 years. All participants were able to withstand the assessment and reported good acceptance.ResultsThe results revealed a significant performance disparity, with older adults displaying notably lower proficiency in the DIGI. The DIGI performance of older adults exhibited a correlation with their ability to employ a set of novel mechanical tools, but not with their ability to handle a set of familiar common tools. There was no such correlation for the younger group.ConclusionIn conclusion, this study introduces an innovative assessment tool aimed at evaluating common digital tool competencies. Our preliminary results demonstrate good acceptance and reveal expected group differences. For current cohorts of older adults, the results seem to indicate that the ability to use novel tools may aid digital tool use. In the next step, the psychometric properties of the DIGI assessment should be evaluated in larger and more diverse samples. The advancement of digital tool competency assessments and rehabilitation strategies is essential when we aim at facilitating societal inclusion and participation for individuals in affected populations.

Project description:BackgroundLearner agency involves students actively engaging in their learning process and shaping their educational experiences through autonomy, self-regulation, and decision-making. In professional education, particularly within health professions, learner agency is critical for fostering adaptability and lifelong learning. This scoping review explores how learner agency, alongside concepts such as self-regulated learning and self-directed learning, is addressed in undergraduate dental education, aiming to understand its implications and strategies for enhancing student agency in this context.MethodsThe scoping review examined literature on self-regulated learning, self-directed learning and learner agency in undergraduate dental education from 1994-April 2024 across five databases: PubMed, Scopus, Embase, ProQuest Central, and Web of Science. A manual search of the cited references was also conducted. Relevant studies were screened, and the findings were summarized to offer a comprehensive overview and identify research gaps.ResultsIn total, 33 studies were included in the review. The results revealed a strong interconnection between intrapersonal, behavioral, and contextual dimensions in shaping learner agency through self-regulated learning and self-directed learning among undergraduate dental students. The studies analyzed, predominantly quantitative, highlighted the multifaceted relationships among self-regulated learning and self-directed learning and learner agency, emphasizing its significance for educational practice and policy.ConclusionsSelf-regulated learning and self-directed learning are crucial for developing learner agency, aiding undergraduate students' transition into independent professionals and fostering lifelong learning behaviors. Educational strategies should prioritize empowering students to become independent learners, reducing their reliance on faculty. Further research is needed to identify effective methods for promoting learner agency development among dental students.

Dataset Information

Giving away some of their powers! Towards learner agency in digital assessment and feedback.

Supplementary information

Publications

Giving away some of their powers! Towards learner agency in digital assessment and feedback.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets