biostudies-literatureUnknown649(8099)Center for AI SafetyBenchmarks are important tools for tracking the rapid advancements in large language model (LLM) capabilities. However, benchmarks are not keeping pace in difficulty: LLMs now achieve more than 90% accuracy on popular benchmarks such as Measuring Massive Multitask Language Understanding¹, limiting informed measurement of state-of-the-art LLM capabilities. Here, in response, we introduce Humanity's Last Exam (HLE), a multi-modal benchmark at the frontier of human knowledge, designed to be an expert-level closed-ended academic benchmark with broad subject coverage. HLE consists of 2,500 questions across dozens of subjects, including mathematics, humanities and the natural sciences. HLE is developed globally by subject-matter experts and consists of multiple-choice and short-answerNature1139-1146https://www.ebi.ac.uk/biostudies/studies/S-EPMC12851929biostudies-literatureA benchmark of expert-level academic questions to assess AI capabilities.PMC12851929Kratish YKazibwe ZDouville GRodrguez RAHMen QAlley WDias FMPantidis IBashmal LKlose IRank BChristof BMollaei SSperzel MSidhu JYuan QShahrtash HKean JYuan JMcInnis IMJJagota MSomrak MAvadhanam AShumar OAmaro BBailey JBerger ZSparrow CYu MCao HZhu ZYu SQi HMcAlister ICTawfeek ARQi TMcCarty BYu EShenoy AZhou YAYoon JMaksapetyan ARossbach VSayed MScotese CRZekry MWhite CYu ZRawal KWei BWei AGonzalez DEHernandez-Orallo JMishra RMishra SSemler APurohit SSHava DBBari MSRiblet RScale AIGulati SMikov AAlbanie SAbramovitch MStade JZhu KPoswiata RZang ADong WP RAgarwal KPRichmond MIvanov STerpin ARagoler ARao AXiong YSharma ABian SCeconello CAgarwal AChen ZOf Arc Xavier JChen YHoover AChen SLeventov RGoldfarb AIvanov AGundlach HChen QLu XHBharath VChen KPlassart AChen JChen MTwayana MMontecillo RGHarjadi CWehr DHoback SRosset SChen HChen BLaurendeau JPlecnik RGonzalez JCChen EMartinez DGoryachev VGalal ASakor AShiyu HNguyen THMa ZMa WNguyen TAFournier-Facio FNewbould JJohnson BHendrycks DRacz BVotava JMaximiano GTarver TBogdanik SDernoncourt FPhillips SMaghsoudimehrabani MPouriamanesh RBugas DMa BAnderson EPoulos JAnderson DPatramanis DSala SPreu TWillcocks CGShah NShen JBrenner MPShah SJindel KShah JFinke LShen ZVincze MShah TSalazar JKarwowski JDehghan ABateman GPu BHalevy KFeng SYMahmood MReidegeld FPeterson MMaung EDworakowska BWecker ADhuliawala SGehrunger THaggstrom OLucki JVenugopalan SHan XHan ZQuinn Liu HCole JVieira BHOller MMeril ENayak APhan LLindsey JFosin IFarhidi FSayous RApronti ASaha APlomecka MScipio EJBartlett CWPflugfelder RYap LMariji HSchneider KGerbicz RHallman EAgrawal ABrown EDLKolt NPerlitz YClarke DPeng CTran HAgrawal RLang LPatlan ASZubic NNunes MISIyer KAly RMendoza SSarti GSantos TCBFrancois PToukmaji CHernandez-Camara PEnyekwe ISingla VVinnikov VAntonenko DSManini DKretov EWeng EKansal YEicher JNguyen MCBattaglia MBizeul AAbdoli SHoyer DMarion PAmit AGimenez JMoat RDhingra SMorak WPatwardhan TChoi MZhidkovskaya ABGul SKrause MOlszewska KCho NGendron GGoswami LPYan MCheatom AMohamed MStanley RShen AChopra AGitter AMatsumoto KCoppola BPAlbani GKhan SMakarychev YLin XHabibi FJain RKDatta SZohar OLamparth MKhoja AYalam SBanik DHeilala VTaylor EDemian CSauers SQazi UStump CWang HKRannev IHe HHe MBodur OFStritecky VBezzi DGloor LJain EEbert SGoncalves DVPonkshe KMavroudis VGonzalez JSafdari MPai KLin HJain YTucker-Foltz JColino JAdesanya FChidambaram MYong ZXFruhauff TMalishev DJones AMedley GWist JMensah SFan HMaayan GFan JKarakoc ALi BStambaugh NKim DHSandlin JAi VWang JPPiperski APadlewski PSpelda PFan YVaquero LDe Maddalena RNevirkovets VHollom LChalstrey EKassahun HGeirhos RZheltonozhskii ELewark LSchut LYifei LSTordera DSegev BTurchin PGupta HAkov IGupta IZhelnov PStander DGeirhos CCunningham DMLi HLi JLi CReddy SPLi DLooi SZLi FLi RLi LPeristyy ATu HLi NStokes ACLi XLi YRodriguez-Romero ALi SMagoulas ISaxena PSchottdorf MLe VKDucey JDhole KMao MGrover RHalasyamani SZebaze ARLengler JRamirez-Trinidad AFriedrich AJiang TCozianu CShabani AJiang MJiang YJackson AJiang XMorris JNam JZheng GStendall RZheng EJia ZVerma SYin YBrianski MMalysheva YShi JWLSiddh SMaksimov APLuo GMalina SLuo EBurns NZhang WRaynor BLauff RAskew LGivre AZhang TNoye MLuo JHeimonen JPZhang ZZhang YZhang XYin MZhang GTelluri AZhang DParadise ONarayan HSunkari SZhang HMoulin AZhang OBouyamourn AZvonkine DZhang MZhang LBosca MCShinde DPZhang SZhang RMoyano AJZhang QDe Luca GBJin ASponselee KJin CCrisostomi DNi CJin JZhang AHa JPetruzella GKamdoum AGKElkhanany ARozhon VSenthilkumar SMukherjee KHauser JYucel ATMcGowan JClark RRissone PArnaboldi LSlen ARoth MHoerr RGBorbas BSantens TAbbondanza DDel Rio-Chanona MJeanplong FZhou AChu EMalusare ALe AZhou KGalgon GFerreira DXiao YDodonov DKelley ESoroko IXiao TNasim ZZhou YZhou XHaffenden HBatra HPottmaier DMundler NLuczyna GKarim LBrown ACarroll MSun CSchoelkopf HPronin AFraga NWAndreoletti JGopi SKumar PKazakov DNoever DGiordano PSun RSun DMoller SSun GWykowski JGolshani BLauer PFerret JMazeika MGustafsson NSun KSubramanian SYang HLe LYang JDrori JTan JKang TSun TBrooks MRadhakrishnan AHuber LSTaheri OSun XSun YSun ZYang XLabrador YCLShaaban MYang RYang TUsawasutsakorn SSundarapandiyan VSanker VBacho AMak JZweiger Ade Oliveira Santos EKoppel JPress OBacho KUzhou MDelaney EBacho RElSheikh ATee ARobinson JHausenloy JKassani PShukla ABrown ZWinter NDemircali AAYacar LDemircali AEBaranov AKumar HMourrat JCAji AFKazakov MShukla PBigi Fde la Rosa FCavalleri SShah IAKharlamova ASztyber-Betley ALoiseau GOverholt KJDuersch JCenter for AI SafetyManik TBalabanian GBarkan SChen PEAldous DVan Der Sypt LFossum CJSims BXu RCreighton SXu HXu JSouza VCarmi ARecchia GXu SScaramuzza DKvistad LXu ZBohdal OKopylov AYe ZPyda DRiis SImperial JMStehberger MBrussel LMcKellips GMeiburg AGangal VWydallis JBLi WDMostaghimi HLeser JNDoroshenko MKorbak TMirabadi AKYilmaz AYeadon WMuzhen JHeydari NSiedler PDDu ZChern SBar KEron MSalauddin SMMarjanovic SVDick AKPopescu MBarzowski SGreen SRKulkarni VAmbay JAIslam MSteinerberger SBradshaw PLai JLiu JLiu MSewuster GLeung JVargus FLopez-Miguel IDObikoya GDPoesia Reis E Silva GLiu YLiu ZLukas MEl-Wasif ZSonmez YNandor MSakal TCojoc DBrito-Santana LGuan JPoritski VDobarskyi KTire KGatti APokutnyi OMunro DPandey RAdoul ZElser VLoume GAMenshawy AKnights THarris CCrowson SJLai SHammon APetersen PZu YPlatnick JHulbert DKalinin MShulga NLiang YTodoran SHao WYaln YArrais AOHoang HPondaven ABalint BLiang KPonti EMMalod GBosio CVerkama ECrozier JChhablani GVergo EBasler LRobins JGusev IQuod Soler Bartomeu DThi Hoang HBosio ICartwright ASinger EStepanic SWeller ORadionov MRodrigo-Gines FJZhang CBCKazemi FRen YZsambok JUro JLShepherd MTanwie NMNaiya MMErmon SJackson JWTiryakioglu MQu XRen RRen QParcalabescu LDewerpe IXin LMontgomery KDendane AOvchynnikov MSingh ALing JChang YWong AAWLSanchez APang TCFu EJansen TSuresh SCosta MVajipey VBogdanov SRoberts JPerelkiewicz MAngquist IKamalov FSommerhage FMaheshwari RSiddiqi MRIskra OKirtland AGe JHossain ZSingh MMichael Pokorny RHausknecht KShamseldeen SSen AZe-An NSingh VYao JZenitani KRogers TRPiza MGurdogan HPopov SArsene PZampese JManolescu CVaz WSMilliron HThaman KEkstrom FFeng FGinis VJuefei-Xu FYakimchyk AFasiludeen AAKasamsetty RDieuleveut AWang EYShridhar KTony Lian LKrishna SJaeger JFish SFaraboschi PMerrill WVilchis VEGChung JWBran AMDe Oliveira Pena RPerrella DResman NFinocchio RArthornthurasuk SVilchis TGPan JAli SYuan MXPan RMatos JOGross JGrabb DPerry SSzlyk HPatel ARemy NFang JArdito CGHu YNagumalli LHu WDeng JDeng MChigurupati SAlgaba ADurand SPatel OPradeep RPatel SMaggetti MMofayezi MLynch JKostakos VLiakhovitskaia AMenezes AStolfo AThornley EDaans NRamakrishnan KPrabhu AReuel ASanz-Ros JHu JDo LGoshu GMMartinson SMohammadzadeh FRodriguez MOZouhar VMotwani SZhao XZhao SDhamane OZhao HCiobaca SCaetano DMZhao BCohen MKCohen Nde Witt CSJha KKuchkin ACheraku AMeer KSahu AKirchhof MKuchkin VElgnainy HBeger CPageler BLoader JAvagian DAlarab IDuc Ede Oliveira Junior BAKaddour JBurdzy KAziz MFBerkani MVetoshkin LPereira RSinghal RWolff HAgu CNattanmai AScaria KJCavanagh JMGiska FLass JFirsching MLux TCHStap DReddish JMehkary MDi Fratta CCosma AVerbeken BAggarwal DOutevsky DKondor DMuennighoff NClavier PMedved TCai MCai JSivarajan SZou AOrel DZou CXi HEcheazu DHeld WGuillod JKhanh TQOkutsu KRodola EKalpathi TPaek JLackner MWeber BLim DKreuer JWeber SAlapont XOliveira ESoori SCai WKreiman TShi STaamazyan VAaron AShapourian HShi TPortier JMilbauer JVillanueva JPMartin FCao PMJallon AZhuang CFriday COsbey TColeman SPham DTBartolo MFarina NMaar JPerez RHKaddar YPark HKHLE Contributors ConsortiumHeidinger HZhang DKYeo WLevin JCVan den Houte KChandok NCaciolai AEfremov DVendrow EAgnihotri SMyklebust BVendrow JPiccardo MChernyavsky JLee SChernyavsky IFranca AAnugraha DZhang CEKim HMa JJKim EQin JLee JTang CDemir GPeskoff DDai YMeding KCheng ZTang HTang MCheng VTang LTang NGarretson ACWheeler RGoertzen TKaparov NHebbar SAKamphuis TBriia GTam JAndrew Favre DOZsolnai-Feher KHartman TDSherman GRabern BTkadlec JBen-David SKim SKim RDai LKim WQin HLee YRodman ERedenti ALee KHYue SWu JSinha ALer DWang ASamal SWang CPatil DYue RWang DWu BWang GLo EJYWang JWang KWang MPatil VFoster MHogg TKirchner JHMinissi AMRWang SWang TSucholutsky IAchilleos AWu SHolmes DSridhar KWu TNhu ANWang XSamuele FWang YWu WWang ZHuy TDPark HPark IPark JDuclosel DHandokoPark NMahfoud MHo AInlow MHXian RSmucker LHo LPinto AAnantheswaran ULoh KYPinto FWu AWhitfill PVeith JPenaflor MArkhipov PSun HYLorena RSuhail AChamorro-Padial JPark CFGiboney ZKhoury SAnsarinejad BRosu PWilliamson DKrenek KGritsevskiy ARismanchian SSong ZNie YJoshi PCarauleanu MUtpala SNguyen TSong DTomasiello ACampbell RKon PTJNguyen JDegorre JDasouqi ANguyen LWinata GISchwaller PNguyen HSchmitt JCrispino NSileo DShahid SMMa MQBahaloohoreh MfalseA benchmark of expert-level academic questions to assess AI capabilities.Benchmarks are important tools for tracking the rapid advancements in large language model (LLM) capabilities. However, benchmarks are not keeping pace in difficulty: LLMs now achieve more than 90% accuracy on popular benchmarks such as Measuring Massive Multitask Language Understanding¹, limiting informed measurement of state-of-the-art LLM capabilities. Here, in response, we introduce Humanity's Last Exam (HLE), a multi-modal benchmark at the frontier of human knowledge, designed to be an expert-level closed-ended academic benchmark with broad subject coverage. HLE consists of 2,500 questions across dozens of subjects, including mathematics, humanities and the natural sciences. HLE is developed globally by subject-matter experts and consists of multiple-choice and short-answer2026-01-01T00:00:00Z2026 Jan2026-06-12T10:17:43.814Z2026-06-12T03:12:36.223ZS-EPMC128519294160615510.1038/s41586-025-09962-4