Generative AI as a Tool for Environmental Health Research Translation.
Ontology highlight
ABSTRACT:
SUBMITTER: Anderson LB
PROVIDER: S-EPMC10369501 | biostudies-literature | 2023 Jul
REPOSITORIES: biostudies-literature
Publications
Generative AI as a Tool for Environmental Health Research Translation.
Anderson Lauren B LB Kanneganti Dhiraj D Houk Mary Bentley MB Holm Rochelle H RH Smith Ted T
GeoHealth 20230726 7
One valuable application for generative artificial intelligence (AI) is summarizing research studies for non-academic readers. We submitted five articles to Chat Generative Pre-trained Transformer (ChatGPT) for summarization, and asked the article's author to rate the summaries. Higher ratings were assigned to more insight-oriented activities, such as the production of eighth-grade reading level summaries, and summaries highlighting the most important findings and real-world applications. The ge ...[more]
Similar Datasets
Project description:Generative artificial intelligence, popularized by services like ChatGPT, has been the source of much recent popular attention for publishing health research. Another valuable application is in translating published research studies to readers in non-academic settings. These might include environmental justice communities, mainstream media outlets, and community science groups. Five recently published (2021-2022) open-access, peer-reviewed papers, authored by University of Louisville environmental health investigators and collaborators, were submitted to ChatGPT. The average rating of all summaries of all types across the five different studies ranged between 3 and 5, indicating good overall content quality. ChatGPT’s general summary request was consistently rated lower than all other summary types. Whereas higher ratings of 4 and 5 were assigned to the more synthetic, insight-oriented activities, such as the production of a plain language summaries suitable for an 8 th grade reading level and identifying the most important finding and real-world research applications. This is a case where artificial intelligence might help level the playing field, for example by creating accessible insights and enabling the large-scale production of high-quality plain language summaries which would truly bring open access to this scientific information. This possibility, combined with the increasing public policy trends encouraging and demanding free access for research supported with public funds, may alter the role journal publications play in communicating science in society. For the field of environmental health science, no-cost AI technology such as ChatGPT holds the promise to improve research translation, but it must continue to be improved (or improve itself) from its current capability.
| S-EPMC9980240 | biostudies-literature
Project description:As artificial intelligence (AI) technologies occupy a bigger role in psychiatric and psychological care and become the object of increased research attention, industry investment, and public scrutiny, tools for evaluating their clinical, ethical, and user-centricity standards have become essential. In this paper, we first review the history of rating systems used to evaluate AI mental health interventions. We then describe the recently introduced Framework for AI Tool Assessment in Mental Health (FAITA-Mental Health), whose scoring system allows users to grade AI mental health platforms on key domains, including credibility, user experience, crisis management, user agency, health equity, and transparency. Finally, we demonstrate the use of FAITA-Mental Health scale by systematically applying it to OCD Coach, a generative AI tool readily available on the ChatGPT store and designed to help manage the symptoms of obsessive-compulsive disorder. The results offer insights into the utility and limitations of FAITA-Mental Health when applied to "real-world" generative AI platforms in the mental health space, suggesting that the framework effectively identifies key strengths and gaps in AI-driven mental health tools, particularly in areas such as credibility, user experience, and acute crisis management. The results also highlight the need for stringent standards to guide AI integration into mental health care in a manner that is not only effective but also safe and protective of the users' rights and welfare.
| S-EPMC11530715 | biostudies-literature
Project description:BackgroundGenerative artificial intelligence has the potential to revolutionize health technology product development by improving coding quality, efficiency, documentation, quality assessment and review, and troubleshooting.ObjectiveThis paper explores the application of a commercially available generative artificial intelligence tool (ChatGPT) to the development of a digital health behavior change intervention designed to support patient engagement in a commercial digital diabetes prevention program.MethodsWe examined the capacity, advantages, and limitations of ChatGPT to support digital product idea conceptualization, intervention content development, and the software engineering process, including software requirement generation, software design, and code production. In total, 11 evaluators, each with at least 10 years of experience in fields of study ranging from medicine and implementation science to computer science, participated in the output review process (ChatGPT vs human-generated output). All had familiarity or prior exposure to the original personalized automatic messaging system intervention. The evaluators rated the ChatGPT-produced outputs in terms of understandability, usability, novelty, relevance, completeness, and efficiency.ResultsMost metrics received positive scores. We identified that ChatGPT can (1) support developers to achieve high-quality products faster and (2) facilitate nontechnical communication and system understanding between technical and nontechnical team members around the development goal of rapid and easy-to-build computational solutions for medical technologies.ConclusionsChatGPT can serve as a usable facilitator for researchers engaging in the software development life cycle, from product conceptualization to feature identification and user story development to code generation.Trial registrationClinicalTrials.gov NCT04049500; https://clinicaltrials.gov/ct2/show/NCT04049500.
| S-EPMC10955400 | biostudies-literature
Project description:BackgroundPatients find technology tools to be more approachable for seeking sensitive health-related information, such as reproductive health information. The inventive conversational ability of artificial intelligence (AI) chatbots, such as ChatGPT (OpenAI Inc), offers a potential means for patients to effectively locate answers to their health-related questions digitally.ObjectiveA pilot study was conducted to compare the novel ChatGPT with the existing Google Search technology for their ability to offer accurate, effective, and current information regarding proceeding action after missing a dose of oral contraceptive pill.MethodsA sequence of 11 questions, mimicking a patient inquiring about the action to take after missing a dose of an oral contraceptive pill, were input into ChatGPT as a cascade, given the conversational ability of ChatGPT. The questions were input into 4 different ChatGPT accounts, with the account holders being of various demographics, to evaluate potential differences and biases in the responses given to different account holders. The leading question, "what should I do if I missed a day of my oral contraception birth control?" alone was then input into Google Search, given its nonconversational nature. The results from the ChatGPT questions and the Google Search results for the leading question were evaluated on their readability, accuracy, and effective delivery of information.ResultsThe ChatGPT results were determined to be at an overall higher-grade reading level, with a longer reading duration, less accurate, less current, and with a less effective delivery of information. In contrast, the Google Search resulting answer box and snippets were at a lower-grade reading level, shorter reading duration, more current, able to reference the origin of the information (transparent), and provided the information in various formats in addition to text.ConclusionsChatGPT has room for improvement in accuracy, transparency, recency, and reliability before it can equitably be implemented into health care information delivery and provide the potential benefits it poses. However, AI may be used as a tool for providers to educate their patients in preferred, creative, and efficient ways, such as using AI to generate accessible short educational videos from health care provider-vetted information. Larger studies representing a diverse group of users are needed.
| S-EPMC11336497 | biostudies-literature
Project description:BackgroundChat Generative Pre-trained Transformer (ChatGPT); an open access artificial intelligence (AI) tool has been in the limelight with its ability to respond to prompts, analyse data information using algorithms to augment efficiency in day-to-day activities across a spectrum of human activities including MSK/Orthopaedic science.Purpose of the studyThe purpose of this cross-sectional survey has been to analyse the knowledge, understanding of the role of Chat Generative Pre-trained Transformer (ChatGPT) and its implications in clinical practice as well as research in medicine.Material & methodsAn online cross-sectional survey of 10 questions (multiple choice and free text) was circulated amongst orthopaedic surgeons, musculoskeletal radiologists and Rheumatologists in India and UK, to evaluate perception of Chat Generative Pre-trained Transformer (ChatGPT) AI Tool.ResultsWe had 125 responses with majority being aware of ChatGPT though a minority had used it. There was consensus that its going have detrimental effect on workforce with majority of the opinion that they would be used to create radiology reports. Mixed responses were noted regarding the quality of research and role of ChatGPT being an anonymous author.ConclusionThere is a considerable debate amongst clinicians of orthopaedic, radiology and rheumatology -specialities. The attitudes are mixed but mainly positive, although there are many concerns about the still-evolving new technology.Level of studyDiagnostic Study level 4.
| S-EPMC10562840 | biostudies-literature
Project description:BackgroundMental disorders have ranked among the top 10 prevalent causes of burden on a global scale. Generative artificial intelligence (GAI) has emerged as a promising and innovative technological advancement that has significant potential in the field of mental health care. Nevertheless, there is a scarcity of research dedicated to examining and understanding the application landscape of GAI within this domain.ObjectiveThis review aims to inform the current state of GAI knowledge and identify its key uses in the mental health domain by consolidating relevant literature.MethodsRecords were searched within 8 reputable sources including Web of Science, PubMed, IEEE Xplore, medRxiv, bioRxiv, Google Scholar, CNKI and Wanfang databases between 2013 and 2023. Our focus was on original, empirical research with either English or Chinese publications that use GAI technologies to benefit mental health. For an exhaustive search, we also checked the studies cited by relevant literature. Two reviewers were responsible for the data selection process, and all the extracted data were synthesized and summarized for brief and in-depth analyses depending on the GAI approaches used (traditional retrieval and rule-based techniques vs advanced GAI techniques).ResultsIn this review of 144 articles, 44 (30.6%) met the inclusion criteria for detailed analysis. Six key uses of advanced GAI emerged: mental disorder detection, counseling support, therapeutic application, clinical training, clinical decision-making support, and goal-driven optimization. Advanced GAI systems have been mainly focused on therapeutic applications (n=19, 43%) and counseling support (n=13, 30%), with clinical training being the least common. Most studies (n=28, 64%) focused broadly on mental health, while specific conditions such as anxiety (n=1, 2%), bipolar disorder (n=2, 5%), eating disorders (n=1, 2%), posttraumatic stress disorder (n=2, 5%), and schizophrenia (n=1, 2%) received limited attention. Despite prevalent use, the efficacy of ChatGPT in the detection of mental disorders remains insufficient. In addition, 100 articles on traditional GAI approaches were found, indicating diverse areas where advanced GAI could enhance mental health care.ConclusionsThis study provides a comprehensive overview of the use of GAI in mental health care, which serves as a valuable guide for future research, practical applications, and policy development in this domain. While GAI demonstrates promise in augmenting mental health care services, its inherent limitations emphasize its role as a supplementary tool rather than a replacement for trained mental health providers. A conscientious and ethical integration of GAI techniques is necessary, ensuring a balanced approach that maximizes benefits while mitigating potential challenges in mental health care practices.
| S-EPMC11347908 | biostudies-literature
Project description:ObjectiveThis study evaluates an AI assistant developed using OpenAI's GPT-4 for interpreting pharmacogenomic (PGx) testing results, aiming to improve decision-making and knowledge sharing in clinical genetics and to enhance patient care with equitable access.Materials and methodsThe AI assistant employs retrieval-augmented generation (RAG), which combines retrieval and generative techniques, by harnessing a knowledge base (KB) that comprises data from the Clinical Pharmacogenetics Implementation Consortium (CPIC). It uses context-aware GPT-4 to generate tailored responses to user queries from this KB, further refined through prompt engineering and guardrails.ResultsEvaluated against a specialized PGx question catalog, the AI assistant showed high efficacy in addressing user queries. Compared with OpenAI's ChatGPT 3.5, it demonstrated better performance, especially in provider-specific queries requiring specialized data and citations. Key areas for improvement include enhancing accuracy, relevancy, and representative language in responses.DiscussionThe integration of context-aware GPT-4 with RAG significantly enhanced the AI assistant's utility. RAG's ability to incorporate domain-specific CPIC data, including recent literature, proved beneficial. Challenges persist, such as the need for specialized genetic/PGx models to improve accuracy and relevancy and addressing ethical, regulatory, and safety concerns.ConclusionThis study underscores generative AI's potential for transforming healthcare provider support and patient accessibility to complex pharmacogenomic information. While careful implementation of large language models like GPT-4 is necessary, it is clear that they can substantially improve understanding of pharmacogenomic data. With further development, these tools could augment healthcare expertise, provider productivity, and the delivery of equitable, patient-centered healthcare services.
| S-EPMC11105140 | biostudies-literature
Project description:BackgroundBrief message interventions have demonstrated immense promise in health care, yet the development of these messages has suffered from a dearth of transparency and a scarcity of publicly accessible data sets. Moreover, the researcher-driven content creation process has raised resource allocation issues, necessitating a more efficient and transparent approach to content development.ObjectiveThis research sets out to address the challenges of content development for SMS interventions by showcasing the use of generative artificial intelligence (AI) as a tool for content creation, transparently explaining the prompt design and content generation process, and providing the largest publicly available data set of brief messages and source code for future replication of our process.MethodsLeveraging the pretrained large language model GPT-3.5 (OpenAI), we generate a collection of messages in the context of medication adherence for individuals with type 2 diabetes using evidence-derived behavior change techniques identified in a prior systematic review. We create an attributed prompt designed to adhere to content (readability and tone) and SMS (character count and encoder type) standards while encouraging message variability to reflect differences in behavior change techniques.ResultsWe deliver the most extensive repository of brief messages for a singular health care intervention and the first library of messages crafted with generative AI. In total, our method yields a data set comprising 1150 messages, with 89.91% (n=1034) meeting character length requirements and 80.7% (n=928) meeting readability requirements. Furthermore, our analysis reveals that all messages exhibit diversity comparable to an existing publicly available data set created under the same theoretical framework for a similar setting.ConclusionsThis research provides a novel approach to content creation for health care interventions using state-of-the-art generative AI tools. Future research is needed to assess the generated content for ethical, safety, and research standards, as well as to determine whether the intervention is successful in improving the target behaviors.
| S-EPMC11522651 | biostudies-literature
Project description:Generative artificial intelligence (AI) systems have performed well at many biomedical tasks, but few studies have assessed their performance directly compared to students in higher-education courses. We compared student knowledge-assessment scores with prompting of 6 large-language model (LLM) systems as they would be used by typical students in a large online introductory course in biomedical and health informatics that is taken by graduate, continuing education, and medical students. The state-of-the-art LLM systems were prompted to answer multiple-choice questions (MCQs) and final exam questions. We compared the scores for 139 students (30 graduate students, 85 continuing education students, and 24 medical students) to the LLM systems. All of the LLMs scored between the 50th and 75th percentiles of students for MCQ and final exam questions. The performance of LLMs raises questions about student assessment in higher education, especially in courses that are knowledge-based and online.
| S-EPMC11399285 | biostudies-literature
Project description:Computational pathology1,2 has witnessed considerable progress in the development of both task-specific predictive models and task-agnostic self-supervised vision encoders3,4. However, despite the explosive growth of generative artificial intelligence (AI), there have been few studies on building general-purpose multimodal AI assistants and copilots5 tailored to pathology. Here we present PathChat, a vision-language generalist AI assistant for human pathology. We built PathChat by adapting a foundational vision encoder for pathology, combining it with a pretrained large language model and fine-tuning the whole system on over 456,000 diverse visual-language instructions consisting of 999,202 question and answer turns. We compare PathChat with several multimodal vision-language AI assistants and GPT-4V, which powers the commercially available multimodal general-purpose AI assistant ChatGPT-4 (ref. 6). PathChat achieved state-of-the-art performance on multiple-choice diagnostic questions from cases with diverse tissue origins and disease models. Furthermore, using open-ended questions and human expert evaluation, we found that overall PathChat produced more accurate and pathologist-preferable responses to diverse queries related to pathology. As an interactive vision-language AI copilot that can flexibly handle both visual and natural language inputs, PathChat may potentially find impactful applications in pathology education, research and human-in-the-loop clinical decision-making.
| S-EPMC11464372 | biostudies-literature