{"database":"biostudies-literature","file_versions":[],"scores":null,"additional":{"omics_type":["Unknown"],"volume":["26"],"submitter":["Wang L"],"pubmed_abstract":["<h4>Background</h4>The study of disease progression relies on clinical data, including text data, and extracting valuable features from text data has been a research hot spot. With the rise of large language models (LLMs), semantic-based extraction pipelines are gaining acceptance in clinical research. However, the security and feature hallucination issues of LLMs require further attention.<h4>Objective</h4>This study aimed to introduce a novel modular LLM pipeline, which could semantically extract features from textual patient admission records.<h4>Methods</h4>The pipeline was designed to process a systematic succession of concept extraction, aggregation, question generation, corpus extraction, and question-and-answer scale extraction, which was tested via 2 low-parameter LLMs: Qwen-14B-Chat (QWEN) and Baichuan2-13B-Chat (BAICHUAN). A data set of 25,709 pregnancy cases from the People's Hospital of Guangxi Zhuang Autonomous Region, China, was used for evaluation with the help of a local expert's annotation. The pipeline was evaluated with the metrics of accuracy and precision, null ratio, and time consumption. Additionally, we evaluated its performance via a quantified version of Qwen-14B-Chat on a consumer-grade GPU.<h4>Results</h4>The pipeline demonstrates a high level of precision in feature extraction, as evidenced by the accuracy and precision results of Qwen-14B-Chat (95.52% and 92.93%, respectively) and Baichuan2-13B-Chat (95.86% and 90.08%, respectively). Furthermore, the pipeline exhibited low null ratios and variable time consumption. The INT4-quantified version of QWEN delivered an enhanced performance with 97.28% accuracy and a 0% null ratio.<h4>Conclusions</h4>The pipeline exhibited consistent performance across different LLMs and efficiently extracted clinical features from textual data. It also showed reliable performance on consumer-grade hardware. This approach offers a viable and effective solution for mining clinical research data from textual records."],"journal":["Journal of medical Internet research"],"pagination":["e54580"],"full_dataset_link":["https://www.ebi.ac.uk/biostudies/studies/S-EPMC11015372"],"repository":["biostudies-literature"],"pubmed_title":["An Entity Extraction Pipeline for Medical Text Records Using Large Language Models: Analytical Study."],"pmcid":["PMC11015372"],"pubmed_authors":["Bi W","Li Y","Lv H","Ma Y","Wang L"],"additional_accession":[]},"is_claimable":false,"name":"An Entity Extraction Pipeline for Medical Text Records Using Large Language Models: Analytical Study.","description":"<h4>Background</h4>The study of disease progression relies on clinical data, including text data, and extracting valuable features from text data has been a research hot spot. With the rise of large language models (LLMs), semantic-based extraction pipelines are gaining acceptance in clinical research. However, the security and feature hallucination issues of LLMs require further attention.<h4>Objective</h4>This study aimed to introduce a novel modular LLM pipeline, which could semantically extract features from textual patient admission records.<h4>Methods</h4>The pipeline was designed to process a systematic succession of concept extraction, aggregation, question generation, corpus extraction, and question-and-answer scale extraction, which was tested via 2 low-parameter LLMs: Qwen-14B-Chat (QWEN) and Baichuan2-13B-Chat (BAICHUAN). A data set of 25,709 pregnancy cases from the People's Hospital of Guangxi Zhuang Autonomous Region, China, was used for evaluation with the help of a local expert's annotation. The pipeline was evaluated with the metrics of accuracy and precision, null ratio, and time consumption. Additionally, we evaluated its performance via a quantified version of Qwen-14B-Chat on a consumer-grade GPU.<h4>Results</h4>The pipeline demonstrates a high level of precision in feature extraction, as evidenced by the accuracy and precision results of Qwen-14B-Chat (95.52% and 92.93%, respectively) and Baichuan2-13B-Chat (95.86% and 90.08%, respectively). Furthermore, the pipeline exhibited low null ratios and variable time consumption. The INT4-quantified version of QWEN delivered an enhanced performance with 97.28% accuracy and a 0% null ratio.<h4>Conclusions</h4>The pipeline exhibited consistent performance across different LLMs and efficiently extracted clinical features from textual data. It also showed reliable performance on consumer-grade hardware. This approach offers a viable and effective solution for mining clinical research data from textual records.","dates":{"release":"2024-01-01T00:00:00Z","publication":"2024 Mar","modification":"2025-04-19T20:25:20.307Z","creation":"2025-04-19T20:25:20.307Z"},"accession":"S-EPMC11015372","cross_references":{"pubmed":["38551633"],"doi":["10.2196/54580"]}}