Dataset Information

Evaluating atypical language in autism using automated language measures.

ABSTRACT: Measurement of language atypicalities in Autism Spectrum Disorder (ASD) is cumbersome and costly. Better language outcome measures are needed. Using language transcripts, we generated Automated Language Measures (ALMs) and tested their validity. 169 participants (96 ASD, 28 TD, 45 ADHD) ages 7 to 17 were evaluated with the Autism Diagnostic Observation Schedule. Transcripts of one task were analyzed to generate seven ALMs: mean length of utterance in morphemes, number of different word roots (NDWR), um proportion, content maze proportion, unintelligible proportion, c-units per minute, and repetition proportion. With the exception of repetition proportion (p [Formula: see text]), nonparametric ANOVAs showed significant group differences (p[Formula: see text]). The TD and ADHD groups did not differ from each other in post-hoc analyses. With the exception of NDWR, the ASD group showed significantly (p[Formula: see text]) lower scores than both comparison groups. The ALMs were correlated with standardized clinical and language evaluations of ASD. In age- and IQ-adjusted logistic regression analyses, four ALMs significantly predicted ASD status with satisfactory accuracy (67.9-75.5%). When ALMs were combined together, accuracy improved to 82.4%. These ALMs offer a promising approach for generating novel outcome measures.

SUBMITTER: Salem AC

PROVIDER: S-EPMC8155086 | biostudies-literature | 2021 May

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Evaluating atypical language in autism using automated language measures.

Salem Alexandra C AC MacFarlane Heather H Adams Joel R JR Lawley Grace O GO Dolata Jill K JK Bedrick Steven S Fombonne Eric E

Scientific reports 20210526 1

Measurement of language atypicalities in Autism Spectrum Disorder (ASD) is cumbersome and costly. Better language outcome measures are needed. Using language transcripts, we generated Automated Language Measures (ALMs) and tested their validity. 169 participants (96 ASD, 28 TD, 45 ADHD) ages 7 to 17 were evaluated with the Autism Diagnostic Observation Schedule. Transcripts of one task were analyzed to generate seven ALMs: mean length of utterance in morphemes, number of different word roots (ND ...[more]

PMID: 34040042

Similar Datasets

Project description:Variability in expressive and receptive language, difficulty with pragmatic language, and prosodic difficulties are all features of autism spectrum disorder (ASD). Quantifying language and voice characteristics is an important step for measuring outcomes for autistic people, yet clinical measurement is cumbersome and costly. Using natural language processing (NLP) methods and a harmonic model of speech, we analyzed language transcripts and audio recordings to automatically classify individuals as ASD or non-ASD. One-hundred fifty-eight participants (88 ASD, 70 non-ASD) ages 7 to 17 were evaluated with the autism diagnostic observation schedule (ADOS-2), module 3. The ADOS-2 was transcribed following modified SALT guidelines. Seven automated language measures (ALMs) and 10 automated voice measures (AVMs) for each participant were generated from the transcripts and audio of one ADOS-2 task. The measures were analyzed using support vector machine (SVM; a binary classifier) and receiver operating characteristic (ROC). The AVM model resulted in an ROC area under the curve (AUC) of 0.7800, the ALM model an AUC of 0.8748, and the combined model a significantly improved AUC of 0.9205. The ALM model better detected ASD participants who were younger and had lower language skills and shorter activity time. ASD participants detected by the AVM model had better language profiles than those detected by the language model. In combination, automated measurement of language and voice characteristics successfully differentiated children with and without autism. This methodology could help design robust outcome measures for future research. LAY SUMMARY: People with autism often struggle with communication differences which traditional clinical measures and language tests cannot fully capture. Using language transcripts and audio recordings from 158 children ages 7 to 17, we showed that automated, objective language and voice measurements successfully predict the child's diagnosis. This methodology could help design improved outcome measures for research.

Project description:BackgroundEarly identification of autism spectrum disorder (ASD) provides an opportunity for early intervention and improved developmental outcomes. The use of electroencephalography (EEG) in infancy has shown promise in predicting later ASD diagnoses and in identifying neural mechanisms underlying the disorder. Given the high co-morbidity with language impairment, we and others have speculated that infants who are later diagnosed with ASD have altered language learning, including phoneme discrimination. Phoneme learning occurs rapidly in infancy, so altered neural substrates during the first year of life may serve as early, accurate indicators of later autism diagnosis.MethodsUsing EEG data collected at two different ages during a passive phoneme task in infants with high familial risk for ASD, we compared the predictive accuracy of a combination of feature selection and machine learning models at 6 months (during native phoneme learning) and 12 months (after native phoneme learning), and we identified a single model with strong predictive accuracy (100%) for both ages. Samples at both ages were matched in size and diagnoses (n = 14 with later ASD; n = 40 without ASD). Features included a combination of power and nonlinear measures across the 10‑20 montage electrodes and 6 frequency bands. Predictive features at each age were compared both by feature characteristics and EEG scalp location. Additional prediction analyses were performed on all EEGs collected at 12 months; this larger sample included 67 HR infants (27 HR-ASD, 40 HR-noASD).ResultsUsing a combination of Pearson correlation feature selection and support vector machine classifier, 100% predictive diagnostic accuracy was observed at both 6 and 12 months. Predictive features differed between the models trained on 6- versus 12-month data. At 6 months, predictive features were biased to measures from central electrodes, power measures, and frequencies in the alpha range. At 12 months, predictive features were more distributed between power and nonlinear measures, and biased toward frequencies in the beta range. However, diagnosis prediction accuracy substantially decreased in the larger, more behaviorally heterogeneous 12-month sample.ConclusionsThese results demonstrate that speech processing EEG measures can facilitate earlier identification of ASD but emphasize the need for age-specific predictive models with large sample sizes to develop clinically relevant classification algorithms.

Project description:BackgroundLarge language models show promise for improving radiology workflows, but their performance on structured radiological tasks such as Reporting and Data Systems (RADS) categorization remains unexplored.ObjectiveThis study aims to evaluate 3 large language model chatbots-Claude-2, GPT-3.5, and GPT-4-on assigning RADS categories to radiology reports and assess the impact of different prompting strategies.MethodsThis cross-sectional study compared 3 chatbots using 30 radiology reports (10 per RADS criteria), using a 3-level prompting strategy: zero-shot, few-shot, and guideline PDF-informed prompts. The cases were grounded in Liver Imaging Reporting & Data System (LI-RADS) version 2018, Lung CT (computed tomography) Screening Reporting & Data System (Lung-RADS) version 2022, and Ovarian-Adnexal Reporting & Data System (O-RADS) magnetic resonance imaging, meticulously prepared by board-certified radiologists. Each report underwent 6 assessments. Two blinded reviewers assessed the chatbots' response at patient-level RADS categorization and overall ratings. The agreement across repetitions was assessed using Fleiss κ.ResultsClaude-2 achieved the highest accuracy in overall ratings with few-shot prompts and guideline PDFs (prompt-2), attaining 57% (17/30) average accuracy over 6 runs and 50% (15/30) accuracy with k-pass voting. Without prompt engineering, all chatbots performed poorly. The introduction of a structured exemplar prompt (prompt-1) increased the accuracy of overall ratings for all chatbots. Providing prompt-2 further improved Claude-2's performance, an enhancement not replicated by GPT-4. The interrun agreement was substantial for Claude-2 (k=0.66 for overall rating and k=0.69 for RADS categorization), fair for GPT-4 (k=0.39 for both), and fair for GPT-3.5 (k=0.21 for overall rating and k=0.39 for RADS categorization). All chatbots showed significantly higher accuracy with LI-RADS version 2018 than with Lung-RADS version 2022 and O-RADS (P<.05); with prompt-2, Claude-2 achieved the highest overall rating accuracy of 75% (45/60) in LI-RADS version 2018.ConclusionsWhen equipped with structured prompts and guideline PDFs, Claude-2 demonstrated potential in assigning RADS categories to radiology cases according to established criteria such as LI-RADS version 2018. However, the current generation of chatbots lags in accurately categorizing cases based on more recent RADS criteria.

Dataset Information

Evaluating atypical language in autism using automated language measures.

Publications

Evaluating atypical language in autism using automated language measures.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets