Dataset Information

Artificial intelligence-tutoring problem-based learning in ophthalmology clerkship.

ABSTRACT:

Background

Artificial intelligence (AI) is an increasingly popular tool in medical investigations. However, AI's potential of aiding medical teaching has not been explored. This study aimed to evaluate the effectiveness of AI-tutoring problem-based-learning (PBL) in ophthalmology clerkship and to assess the student evaluations of this module.

Methods

Thirty-eight Grade-two students in ophthalmology clerkship at Sun Yat-Sen University were randomly assigned to two groups. In Group A, students learned congenital cataracts through an AI-tutoring PBL module by exploring and operating an AI diagnosis platform. In Group B, students learned congenital cataracts through traditional lecture given with the same faculty. The improvement in student performance was evaluated by comparing the pre- and post-lecture scores of a specific designed test using paired-T tests. Student evaluations of AI-tutoring PBL were measured by a 17-item questionnaire.

Results

The post-lecture scores were significantly higher than the pre-lecture scores in both groups (Group A: P<0.0001, Group B: P<0.0001). The improvement of group A in the part of sign and diagnosis test (Part I) was more significant than that of group B (P=0.016). However, there was no difference in the improvement in the part of treatment plan test (Part II) between two groups (P=0.556). Overall, all respondents were satisfied and agreed that AI-tutoring PBL was helpful, effective, motive and beneficial to help develop critical and creative thinking.

Conclusions

The application of AI-tutoring PBL into ophthalmology clerkship improved students' performance and satisfaction. AI-tutoring PBL teaching showed advantage in promoting students' understanding of signs of diseases. The instructors play an indispensable role in AI-tutoring PBL curriculum.

SUBMITTER: Wu D

PROVIDER: S-EPMC7327320 | biostudies-literature | 2020 Jun

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Artificial intelligence-tutoring problem-based learning in ophthalmology clerkship.

Wu Dongxuan D Xiang Yifan Y Wu Xiaohang X Yu Tongyong T Huang Xiucheng X Zou Yuxian Y Liu Zhenzhen Z Lin Haotian H

Annals of translational medicine 20200601 11

<h4>Background</h4>Artificial intelligence (AI) is an increasingly popular tool in medical investigations. However, AI's potential of aiding medical teaching has not been explored. This study aimed to evaluate the effectiveness of AI-tutoring problem-based-learning (PBL) in ophthalmology clerkship and to assess the student evaluations of this module.<h4>Methods</h4>Thirty-eight Grade-two students in ophthalmology clerkship at Sun Yat-Sen University were randomly assigned to two groups. In Group ...[more]

PMID: 32617320

Similar Datasets

Project description:ObjectiveTo determine the appropriateness of ophthalmology recommendations from an online chat-based artificial intelligence model to ophthalmology questions.Patients and methodsCross-sectional qualitative study from April 1, 2023, to April 30, 2023. A total of 192 questions were generated spanning all ophthalmic subspecialties. Each question was posed to a large language model (LLM) 3 times. The responses were graded by appropriate subspecialists as appropriate, inappropriate, or unreliable in 2 grading contexts. The first grading context was if the information was presented on a patient information site. The second was an LLM-generated draft response to patient queries sent by the electronic medical record (EMR). Appropriate was defined as accurate and specific enough to serve as a surrogate for physician-approved information. Main outcome measure was percentage of appropriate responses per subspecialty.ResultsFor patient information site-related questions, the LLM provided an overall average of 79% appropriate responses. Variable rates of average appropriateness were observed across ophthalmic subspecialties for patient information site information ranging from 56% to 100%: cataract or refractive (92%), cornea (56%), glaucoma (72%), neuro-ophthalmology (67%), oculoplastic or orbital surgery (80%), ocular oncology (100%), pediatrics (89%), vitreoretinal diseases (86%), and uveitis (65%). For draft responses to patient questions via EMR, the LLM provided an overall average of 74% appropriate responses and varied by subspecialty: cataract or refractive (85%), cornea (54%), glaucoma (77%), neuro-ophthalmology (63%), oculoplastic or orbital surgery (62%), ocular oncology (90%), pediatrics (94%), vitreoretinal diseases (88%), and uveitis (55%). Stratifying grades across health information categories (disease and condition, risk and prevention, surgery-related, and treatment and management) showed notable but insignificant variations, with disease and condition often rated highest (72% and 69%) for appropriateness and surgery-related (55% and 51%) lowest, in both contexts.ConclusionThis LLM reported mostly appropriate responses across multiple ophthalmology subspecialties in the context of both patient information sites and EMR-related responses to patient questions. Current LLM offerings require optimization and improvement before widespread clinical use.

Project description:BackgroundThis review explores the bioethical implementation of artificial intelligence (AI) in medicine and in ophthalmology. AI, which was first introduced in the 1950s, is defined as "the machine simulation of human mental reasoning, decision making, and behavior". The increased power of computing, expansion of storage capacity, and compilation of medical big data helped the AI implementation surge in medical practice and research. Ophthalmology is a leading medical specialty in applying AI in screening, diagnosis, and treatment. The first Food and Drug Administration approved autonomous diagnostic system served to diagnose and classify diabetic retinopathy. Other ophthalmic conditions such as age-related macular degeneration, glaucoma, retinopathy of prematurity, and congenital cataract, among others, implemented AI too.PurposeTo review the contemporary literature of the bioethical issues of AI in medicine and ophthalmology, classify ethical issues in medical AI, and suggest possible standardizations of ethical frameworks for AI implementation.MethodsKeywords were searched on Google Scholar and PubMed between October 2019 and April 2020. The results were reviewed, cross-referenced, and summarized. A total of 284 references including articles, books, book chapters, and regulatory reports and statements were reviewed, and those that were relevant were cited in the paper.ResultsMost sources that studied the use of AI in medicine explored the ethical aspects. Bioethical challenges of AI implementation in medicine were categorized into 6 main categories. These include machine training ethics, machine accuracy ethics, patient-related ethics, physician-related ethics, shared ethics, and roles of regulators.ConclusionsThere are multiple stakeholders in the ethical issues surrounding AI in medicine and ophthalmology. Attention to the various aspects of ethics related to AI is important especially with the expanding use of AI. Solutions of ethical problems are envisioned to be multifactorial.

Project description:Background and objectiveAccurate retinal vessel segmentation is often considered to be a reliable biomarker of diagnosis and screening of various diseases, including cardiovascular diseases, diabetic, and ophthalmologic diseases. Recently, deep learning (DL) algorithms have demonstrated high performance in segmenting retinal images that may enable fast and lifesaving diagnoses. To our knowledge, there is no systematic review of the current work in this research area. Therefore, we performed a systematic review with a meta-analysis of relevant studies to quantify the performance of the DL algorithms in retinal vessel segmentation.MethodsA systematic search on EMBASE, PubMed, Google Scholar, Scopus, and Web of Science was conducted for studies that were published between 1 January 2000 and 15 January 2020. We followed the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) procedure. The DL-based study design was mandatory for a study's inclusion. Two authors independently screened all titles and abstracts against predefined inclusion and exclusion criteria. We used the Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) tool for assessing the risk of bias and applicability.ResultsThirty-one studies were included in the systematic review; however, only 23 studies met the inclusion criteria for the meta-analysis. DL showed high performance for four publicly available databases, achieving an average area under the ROC of 0.96, 0.97, 0.96, and 0.94 on the DRIVE, STARE, CHASE_DB1, and HRF databases, respectively. The pooled sensitivity for the DRIVE, STARE, CHASE_DB1, and HRF databases was 0.77, 0.79, 0.78, and 0.81, respectively. Moreover, the pooled specificity of the DRIVE, STARE, CHASE_DB1, and HRF databases was 0.97, 0.97, 0.97, and 0.92, respectively.ConclusionThe findings of our study showed the DL algorithms had high sensitivity and specificity for segmenting the retinal vessels from digital fundus images. The future role of DL algorithms in retinal vessel segmentation is promising, especially for those countries with limited access to healthcare. More compressive studies and global efforts are mandatory for evaluating the cost-effectiveness of DL-based tools for retinal disease screening worldwide.

Project description:ImportanceTo better understand the emerging role of artificial intelligence (AI) in surgical training, efficacy of AI tutoring systems, such as the Virtual Operative Assistant (VOA), must be tested and compared with conventional approaches.ObjectiveTo determine how VOA and remote expert instruction compare in learners' skill acquisition, affective, and cognitive outcomes during surgical simulation training.Design, setting, and participantsThis instructor-blinded randomized clinical trial included medical students (undergraduate years 0-2) from 4 institutions in Canada during a single simulation training at McGill Neurosurgical Simulation and Artificial Intelligence Learning Centre, Montreal, Canada. Cross-sectional data were collected from January to April 2021. Analysis was conducted based on intention-to-treat. Data were analyzed from April to June 2021.InterventionsThe interventions included 5 feedback sessions, 5 minutes each, during a single 75-minute training, including 5 practice sessions followed by 1 realistic virtual reality brain tumor resection. The 3 intervention arms included 2 treatment groups, AI audiovisual metric-based feedback (VOA group) and synchronous verbal scripted debriefing and instruction from a remote expert (instructor group), and a control group that received no feedback.Main outcomes and measuresThe coprimary outcomes were change in procedural performance, quantified as Expertise Score by a validated assessment algorithm (Intelligent Continuous Expertise Monitoring System [ICEMS]; range, -1.00 to 1.00) for each practice resection, and learning and retention, measured from performance in realistic resections by ICEMS and blinded Objective Structured Assessment of Technical Skills (OSATS; range 1-7). Secondary outcomes included strength of emotions before, during, and after the intervention and cognitive load after intervention, measured in self-reports.ResultsA total of 70 medical students (41 [59%] women and 29 [41%] men; mean [SD] age, 21.8 [2.3] years) from 4 institutions were randomized, including 23 students in the VOA group, 24 students in the instructor group, and 23 students in the control group. All participants were included in the final analysis. ICEMS assessed 350 practice resections, and ICEMS and OSATS evaluated 70 realistic resections. VOA significantly improved practice Expertise Scores by 0.66 (95% CI, 0.55 to 0.77) points compared with the instructor group and by 0.65 (95% CI, 0.54 to 0.77) points compared with the control group (P < .001). Realistic Expertise Scores were significantly higher for the VOA group compared with instructor (mean difference, 0.53 [95% CI, 0.40 to 0.67] points; P < .001) and control (mean difference. 0.49 [95% CI, 0.34 to 0.61] points; P < .001) groups. Mean global OSATS ratings were not statistically significant among the VOA (4.63 [95% CI, 4.06 to 5.20] points), instructor (4.40 [95% CI, 3.88-4.91] points), and control (3.86 [95% CI, 3.44 to 4.27] points) groups. However, on the OSATS subscores, VOA significantly enhanced the mean OSATS overall subscore compared with the control group (mean difference, 1.04 [95% CI, 0.13 to 1.96] points; P = .02), whereas expert instruction significantly improved OSATS subscores for instrument handling vs control (mean difference, 1.18 [95% CI, 0.22 to 2.14]; P = .01). No significant differences in cognitive load, positive activating, and negative emotions were found.Conclusions and relevanceIn this randomized clinical trial, VOA feedback demonstrated superior performance outcome and skill transfer, with equivalent OSATS ratings and cognitive and emotional responses compared with remote expert instruction, indicating advantages for its use in simulation training.Trial registrationClinicalTrials.gov Identifier: NCT04700384.

Project description:TopicThis scoping review summarizes artificial intelligence (AI) reporting in ophthalmology literature in respect to model development and validation. We characterize the state of transparency in reporting of studies prospectively validating models for disease classification.Clinical relevanceUnderstanding what elements authors currently describe regarding their AI models may aid in the future standardization of reporting. This review highlights the need for transparency to facilitate the critical appraisal of models prior to clinical implementation, to minimize bias and inappropriate use. Transparent reporting can improve effective and equitable use in clinical settings.MethodsEligible articles (as of January 2022) from PubMed, Embase, Web of Science, and CINAHL were independently screened by 2 reviewers. All observational and clinical trial studies evaluating the performance of an AI model for disease classification of ophthalmic conditions were included. Studies were evaluated for reporting of parameters derived from reporting guidelines (CONSORT-AI, MI-CLAIM) and our previously published editorial on model cards. The reporting of these factors, which included basic model and dataset details (source, demographics), and prospective validation outcomes, were summarized.ResultsThirty-seven prospective validation studies were included in the scoping review. Eleven additional associated training and/or retrospective validation studies were included if this information could not be determined from the primary articles. These 37 studies validated 27 unique AI models; multiple studies evaluated the same algorithms (EyeArt, IDx-DR, and Medios AI). Details of model development were variably reported; 18 of 27 models described training dataset annotation and 10 of 27 studies reported training data distribution. Demographic information of training data was rarely reported; 7 of the 27 unique models reported age and gender and only 2 reported race and/or ethnicity. At the level of prospective clinical validation, age and gender of populations was more consistently reported (29 and 28 of 37 studies, respectively), but only 9 studies reported race and/or ethnicity data. Scope of use was difficult to discern for the majority of models. Fifteen studies did not state or imply primary users.ConclusionOur scoping review demonstrates variable reporting of information related to both model development and validation. The intention of our study was not to assess the quality of the factors we examined, but to characterize what information is, and is not, regularly reported. Our results suggest the need for greater transparency in the reporting of information necessary to determine the appropriateness and fairness of these tools prior to clinical use.Financial disclosuresProprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

Dataset Information

Artificial intelligence-tutoring problem-based learning in ophthalmology clerkship.

Background

Methods

Results

Conclusions

Publications

Artificial intelligence-tutoring problem-based learning in ophthalmology clerkship.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets