biostudies-literatureUnknown12Takahashi H<h4>Background</h4>Generative artificial intelligence (AI) is increasingly used in medical education, including AI-based virtual patients to improve interview skills. However, how much AI-based assessment (ABA) differs from human-based assessment (HBA) remains unclear.<h4>Objective</h4>This study aimed to compare the quality of clinical interview assessments generated via an ABA (GPT-o1 Pro [ABA-o1] and GPT-5 Pro [ABA-5]) with those generated via an HBA conducted by clinical instructors in an AI-based virtual patient setting. We also examined whether AI reduced evaluation time and assessed agreement across participants with different levels of clinical experience.<h4>Methods</h4>A standardized case of leg weakness was implemented in an AI-based virtual patient. Seven participants (2 medicaJMIR medical educatione81673https://www.ebi.ac.uk/biostudies/studies/S-EPMC12912650biostudies-literatureAI- vs Human-Based Assessment of Medical Interview Transcripts in a Generative AI-Simulated Patient System: Cross-Sectional Validation Study.PMC12912650Aiyama YKishi MNagai SNaito TMatsuura TTomoda YKondo TShikino KShinohara TYamada YTokushima YSano FEnomoto AWatanabe RTakahashi HfalseAI- vs Human-Based Assessment of Medical Interview Transcripts in a Generative AI-Simulated Patient System: Cross-Sectional Validation Study.<h4>Background</h4>Generative artificial intelligence (AI) is increasingly used in medical education, including AI-based virtual patients to improve interview skills. However, how much AI-based assessment (ABA) differs from human-based assessment (HBA) remains unclear.<h4>Objective</h4>This study aimed to compare the quality of clinical interview assessments generated via an ABA (GPT-o1 Pro [ABA-o1] and GPT-5 Pro [ABA-5]) with those generated via an HBA conducted by clinical instructors in an AI-based virtual patient setting. We also examined whether AI reduced evaluation time and assessed agreement across participants with different levels of clinical experience.<h4>Methods</h4>A standardized case of leg weakness was implemented in an AI-based virtual patient. Seven participants (2 medica2026-01-01T00:00:00Z2026 Feb2026-07-16T00:38:30.916Z2026-07-09T10:26:28.154ZS-EPMC129126504170194610.2196/81673