Designing learning technology collaboratively: Analysis of a chatbot co-design.
Ontology highlight
ABSTRACT: Supplementary information
The online version contains supplementary material available at 10.1007/s10639-022-11162-w.
SUBMITTER: Durall Gazulla E
PROVIDER: S-EPMC9226288 | biostudies-literature | 2022 Jun
REPOSITORIES: biostudies-literature
Publications
Designing learning technology collaboratively: Analysis of a chatbot co-design.
Durall Gazulla Eva E Martins Ludmila L Fernández-Ferrer Maite M
Education and information technologies 20220624 1
Collaborative design approaches have been increasingly adopted in the design of learning technologies since they contribute to develop pedagogically inclusive and appropriate learning designs. Despite the positive reception of collaborative design strategies in technology-enhanced learning, little attention has been dedicated to analyzing the challenges faced in design processes using a collaborative approach. In this paper, we disclose the collaborative design of a chatbot for self-regulated le ...[more]
Similar Datasets
Project description:BackgroundOur prototype smoking cessation chatbot, Quin, provides evidence-based, personalized support delivered via a smartphone app to help people quit smoking. We developed Quin using a multiphase program of co-design research, part of which included focus group evaluation of Quin among stakeholders prior to clinical testing.ObjectiveThis study aimed to gather and compare feedback on the user experience of the Quin prototype from end users and smoking cessation professionals (SCPs) via a beta testing process to inform ongoing chatbot iterations and refinements.MethodsFollowing active and passive recruitment, we conducted web-based focus groups with SCPs and end users from Queensland, Australia. Participants tested the app for 1-2 weeks prior to focus group discussion and could also log conversation feedback within the app. Focus groups of SCPs were completed first to review the breadth and accuracy of information, and feedback was prioritized and implemented as major updates using Agile processes prior to end user focus groups. We categorized logged in-app feedback using content analysis and thematically analyzed focus group transcripts.ResultsIn total, 6 focus groups were completed between August 2022 and June 2023; 3 for SCPs (n=9 participants) and 3 for end users (n=7 participants). Four SCPs had previously smoked, and most end users currently smoked cigarettes (n=5), and 2 had quit smoking. The mean duration of focus groups was 58 (SD 10.9; range 46-74) minutes. We identified four major themes from focus group feedback: (1) conversation design, (2) functionality, (3) relationality and anthropomorphism, and (4) role as a smoking cessation support tool. In response to SCPs' feedback, we made two major updates to Quin between cohorts: (1) improvements to conversation flow and (2) addition of the "Moments of Crisis" conversation tree. Participant feedback also informed 17 recommendations for future smoking cessation chatbot developments.ConclusionsFeedback from end users and SCPs highlighted the importance of chatbot functionality, as this underpinned Quin's conversation design and relationality. The ready accessibility of accurate cessation information and impartial support that Quin provided was recognized as a key benefit for end users, the latter of which contributed to a feeling of accountability to the chatbot. Findings will inform the ongoing development of a mature prototype for clinical testing.
| S-EPMC11369547 | biostudies-literature
Project description:BackgroundThe digitalisation of healthcare has provided new ways to address disparities in sexual health outcomes that particularly affect ethnic and sexual minorities. Conversational artificial intelligence (AI) chatbots can provide personalised health education and refer users for appropriate medical consultations. We aimed to explore design principles of a chatbot-assisted culturally sensitive self-assessment intervention based on the disclosure of health-related information.MethodsIn 2022, an online survey was conducted among an ethnically diverse UK sample (N = 1,287) to identify the level and type of health-related information disclosure to sexual health chatbots, and reactions to chatbots' risk appraisal. Follow-up interviews (N = 41) further explored perceptions of chatbot-led health assessment to identify aspects related to acceptability and utilisation. Datasets were analysed using one-way ANOVAs, linear regression, and thematic analysis.ResultsParticipants had neutral-to-positive attitudes towards chatbots and were comfortable disclosing demographic and sensitive health information. Chatbot awareness, previous experience and positive attitudes towards chatbots predicted information disclosure. Qualitatively, four main themes were identified: "Chatbot as an artificial health advisor", "Disclosing information to a chatbot", "Ways to facilitate trust and disclosure", and "Acting on self-assessment".ConclusionChatbots were acceptable for health self-assessment among this sample of ethnically diverse individuals. Most users reported being comfortable disclosing sensitive and personal information, but user anonymity is key to engagement with chatbots. As this technology becomes more advanced and widely available, chatbots could potentially become supplementary tools for health education and screening eligibility assessment. Future research is needed to establish their impact on screening uptake and access to health services among minoritised communities.
| S-EPMC11824973 | biostudies-literature
Project description:Background and objectivesThere is a growing interest to involve older adults in the co-design of technology to maintain their well-being and independence. What remains unknown is whether the beneficial effects of co-designed solutions are greater than those reported for non co-designed solutions. The aim of this study was to evaluate the effects and experiences of co-designed technology that support older adults to age in place.Research design and methodsWe conducted a systematic review to (a) investigate the health and well-being outcomes of co-designed technology for older adults (≥60 years), (b) identify co-design approaches and contexts where they are applied, and (c) identify barriers and facilitators of the co-design process with older adults. Searches were conducted in MEDLINE, EMBASE, CINAHL, Science Citation Index (Web of Science), Scopus, OpenGrey, and Business Source Premiere.ResultsWe identified 14,649 articles and included 34 projects. Four projects reported health and well-being outcomes; the effects were inconsistent. Co-design processes varied greatly and in their intensity of older adult involvement. Common facilitators of and barriers to co-design included the building of relationships between stakeholders, stakeholder knowledge of problems and solutions, and expertise in the co-design methodology.Discussion and implicationsThe effect of co-designed technology on health and well-being was rarely studied and it was difficult to ascertain its impact. Future co-design efforts need to address barriers unique to older adults. Evaluation of the impact of co-designed technologies is needed and standardization of the definition of co-design would be helpful to researchers and designers.
| S-EPMC8437501 | biostudies-literature
Project description:BackgroundChatbots have the potential to increase people's access to quality health care. However, the implementation of chatbot technology in the health care system is unclear due to the scarce analysis of publications on the adoption of chatbot in health and medical settings.ObjectiveThis paper presents a protocol of a bibliometric analysis aimed at offering the public insights into the current state and emerging trends in research related to the use of chatbot technology for promoting health.MethodsIn this bibliometric analysis, we will select published papers from the databases of CINAHL, IEEE Xplore, PubMed, Scopus, and Web of Science that pertain to chatbot technology and its applications in health care. Our search strategy includes keywords such as "chatbot," "virtual agent," "virtual assistant," "conversational agent," "conversational AI," "interactive agent," "health," and "healthcare." Five researchers who are AI engineers and clinicians will independently review the titles and abstracts of selected papers to determine their eligibility for a full-text review. The corresponding author (ZN) will serve as a mediator to address any discrepancies and disputes among the 5 reviewers. Our analysis will encompass various publication patterns of chatbot research, including the number of annual publications, their geographic or institutional distribution, and the number of annual grants supporting chatbot research, and further summarize the methodologies used in the development of health-related chatbots, along with their features and applications in health care settings. Software tool VOSViewer (version 1.6.19; Leiden University) will be used to construct and visualize bibliometric networks.ResultsThe preparation for the bibliometric analysis began on December 3, 2021, when the research team started the process of familiarizing themselves with the software tools that may be used in this analysis, VOSViewer and CiteSpace, during which they consulted 3 librarians at the Yale University regarding search terms and tentative results. Tentative searches on the aforementioned databases yielded a total of 2340 papers. The official search phase started on July 27, 2023. Our goal is to complete the screening of papers and the analysis by February 15, 2024.ConclusionsArtificial intelligence chatbots, such as ChatGPT (OpenAI Inc), have sparked numerous discussions within the health care industry regarding their impact on human health. Chatbot technology holds substantial promise for advancing health care systems worldwide. However, developing a sophisticated chatbot capable of precise interaction with health care consumers, delivering personalized care, and providing accurate health-related information and knowledge remain considerable challenges. This bibliometric analysis seeks to fill the knowledge gap in the existing literature on health-related chatbots, entailing their applications, the software used in their development, and their preferred functionalities among users.International registered report identifier (irrid)PRR1-10.2196/54349.
| S-EPMC10905346 | biostudies-literature
Project description:Many retailers invest in artificial intelligence (AI) to improve operational efficiency or enhance customer experience. However, AI often disrupts employees' ways of working causing them to resist change, thus threatening the successful embedding and sustained usage of the technology. Using a longitudinal, multi-site ethnographic approach combining 74 stakeholder interviews and 14 on-site retail observations over a 5-year period, this article examines how employees' practices change when retailers invest in AI. Practice co-evolution is identified as the process that undergirds successful AI integration and enables retail employees' sustained usage of AI. Unlike product or practice diffusion, which may be organic or fortuitous, practice co-evolution is an orchestrated, collaborative process in which a practice is co-envisioned, co-adapted, and co-(re)aligned. To be sustained, practice co-evolution must be recursive and enabled via intentional knowledge transfers. This empirically-derived recursive phasic model provides a roadmap for successful retail AI embedding, and fruitful future research avenues.Supplementary informationThe online version contains supplementary material available at 10.1007/s11747-022-00896-1.
| S-EPMC9390956 | biostudies-literature
Project description:To better support creative software developers and music technologists' needs, and to empower them as machine learning users and innovators, the usability of and developer experience with machine learning tools must be considered and better understood. We review background research on the design and evaluation of application programming interfaces (APIs), with a focus on the domain of machine learning for music technology software development. We present the design rationale for the RAPID-MIX API, an easy-to-use API for rapid prototyping with interactive machine learning, and a usability evaluation study with software developers of music technology. A cognitive dimensions questionnaire was designed and delivered to a group of 12 participants who used the RAPID-MIX API in their software projects, including people who developed systems for personal use and professionals developing software products for music and creative technology companies. The results from questionnaire indicate that participants found the RAPID-MIX API a machine learning API which is easy to learn and use, fun, and good for rapid prototyping with interactive machine learning. Based on these findings, we present an analysis and characterization of the RAPID-MIX API based on the cognitive dimensions framework, and discuss its design trade-offs and usability issues. We use these insights and our design experience to provide design recommendations for ML APIs for rapid prototyping of music technology. We conclude with a summary of the main insights, a discussion of the merits and challenges of the application of the CDs framework to the evaluation of machine learning APIs, and directions to future work which our research deems valuable.
| S-EPMC7861239 | biostudies-literature
Project description:Objectives: Physicians with different professional backgrounds often diagnose a patients' problem collaboratively. In this article, we first introduce a process model for collaborative diagnosing (CDR model), describe the development of a simulation used to empirically examine the facilitation of collaborative diagnostic reasoning. Based on a contemporary validity framework [1], we further suggest indicators for validity and collect initial evidence with respect to the scoring, generalization, extrapolation, and implication inferences to assess the validity of the simulation when used to assess effects of learning interventions. Method: In a quasi-experimental study, we assessed objectivity and reliability of the simulation and compared medical students with low and advanced prior knowledge to practitioners with high prior knowledge with respect to their diagnostic accuracy, diagnostic efficiency, information sharing skills, and their intrinsic cognitive load. Additionally, we obtained authenticity ratings from practitioners with high prior knowledge. Results: The results yielded satisfying initial evidence for the validity of the scoring and the extrapolation inferences as ratings are objective, and the simulation and the collaborative process is perceived as rather authentic. Additionally, participants on different levels of prior knowledge differ with respect to their diagnostic accuracy, diagnostic efficiency, information sharing skills, and their reported intrinsic cognitive load. With one exception (information sharing skills), the generalization inference seems to be valid as well. Conclusions: We conclude that collecting validity evidence for the simulation was an important step towards a better interpretation of the simulation. We found that the simulation is an authentic and valid representation of the chosen collaborative situation and that the collected validity evidence offers sufficient evidence for an initial validation of the simulation. Nevertheless, the validation process highlighted some important gaps that need further consideration. We further conclude that applying a validation model to the context of empirical research is promising and encourage other researchers to follow the example.
| S-EPMC7499460 | biostudies-literature
Project description:BackgroundIn addition to addiction and substance abuse, motivational interviewing (MI) is increasingly being integrated in treating other clinical issues such as mental health problems. Most of the many technological adaptations of MI, however, have focused on delivering the action-oriented treatment, leaving its relational component unexplored or vaguely described. This study intended to design a conversational sequence that considers both technical and relational components of MI for a mental health concern.ObjectiveThis case study aimed to design a conversational sequence for a brief motivational interview to be delivered by a Web-based text messaging application (chatbot) and to investigate its conversational experience with graduate students in their coping with stress.MethodsA brief conversational sequence was designed with varied combinations of MI skills to follow the 4 processes of MI. A Web-based text messaging application, Bonobot, was built as a research prototype to deliver the sequence in a conversation. A total of 30 full-time graduate students who self-reported stress with regard to their school life were recruited for a survey of demographic information and perceived stress and a semistructured interview. Interviews were transcribed verbatim and analyzed by Braun and Clarke's thematic method. The themes that reflect the process of, impact of, and needs for the conversational experience are reported.ResultsParticipants had a high level of perceived stress (mean 22.5 [SD 5.0]). Our findings included the following themes: Evocative Questions and Clichéd Feedback; Self-Reflection and Potential Consolation; and Need for Information and Contextualized Feedback. Participants particularly favored the relay of evocative questions but were less satisfied with the agent-generated reflective and affirming feedback that filled in-between. Discussing the idea of change was a good means of reflecting on themselves, and some of Bonobot's encouragements related to graduate school life were appreciated. Participants suggested the conversation provide informational support, as well as more contextualized feedback.ConclusionsA conversational sequence for a brief motivational interview was presented in this case study. Participant feedback suggests sequencing questions and MI-adherent statements can facilitate a conversation for stress management, which may encourage a chance of self-reflection. More diversified sequences, along with more contextualized feedback, should follow to offer a better conversational experience and to confirm any empirical effect.
| S-EPMC6488959 | biostudies-literature
Project description:BackgroundRecent shifts to telemedicine and remote patient monitoring demonstrate the potential for new technology to transform health systems; yet, methods to design for inclusion and resilience are lacking.ObjectiveThe aim of this study is to design and implement a participatory framework to produce effective health care solutions through co-design with diverse stakeholders.MethodsWe developed a design framework to cocreate solutions to locally prioritized health and communication problems focused on cancer care. The framework is premised on the framing and discovery of problems through community engagement and lead-user innovation with the hypothesis that diversity and inclusion in the co-design process generate more innovative and resilient solutions. Discovery, design, and development were implemented through structured phases with design studios at various locations in urban and rural Kentucky, including Appalachia, each building from prior work. In the final design studio, working prototypes were developed and tested. Outputs were assessed using the System Usability Scale as well as semistructured user feedback.ResultsWe co-designed, developed, and tested a mobile app (myPath) and service model for distress surveillance and cancer care coordination following the LAUNCH (Linking and Amplifying User-Centered Networks through Connected Health) framework. The problem of awareness, navigation, and communication through cancer care was selected by the community after framing areas for opportunity based on significant geographic disparities in cancer and health burden resource and broadband access. The codeveloped digital myPath app showed the highest perceived combined usability (mean 81.9, SD 15.2) compared with the current gold standard of distress management for patients with cancer, the paper-based National Comprehensive Cancer Network Distress Thermometer (mean 74.2, SD 15.8). Testing of the System Usability Scale subscales showed that the myPath app had significantly better usability than the paper Distress Thermometer (t63=2.611; P=.01), whereas learnability did not differ between the instruments (t63=-0.311; P=.76). Notable differences by patient and provider scoring and feedback were found.ConclusionsParticipatory problem definition and community-based co-design, design-with methods, may produce more acceptable and effective solutions than traditional design-for approaches.
| S-EPMC9044168 | biostudies-literature
Project description:IntroductionThe homeless population faces a "cliff edge of inequality" when trying to access essential dental services. There are several additional barriers to accessing dental care in comparison to the general population and the heterogeneous nature of patients presents a significant challenge when designing dental services to meet their needs. Following the Smile4Life report in 2009, there is limited up-to-date and population-specific evidence available for the optimal model of service delivery.AimThis study aimed to co-design principles for a prospective dental service for adults experiencing homelessness.MethodsA qualitative methodology was used to incorporate experts-by-experience into elements of co-designing a dental service for adults experiencing homelessness. The study combined elements of an experienced-based co-design framework for healthcare innovation with community-based participatory research. Focus groups with people experiencing homelessness and healthcare practitioners were conducted to identify principles for any prospective dental service, as well as several barriers and enablers to establishing a homeless dental service. The findings were transcribed and analysed using thematic analysis on Nvivo software.ResultsFrom the qualitative analysis five key themes emerged: (1) Impact and expectations of oral health while experiencing homelessness, (2) Barriers to accessing dental care; (3) Practitioner's views about homelessness and access to care; (4) Barriers to designing a homeless service and (5) Enablers for co-designing a new model of dental care delivery for the homeless population.Five key principles for a new model of homeless dental service were identified: (i) Services designed to address the needs of patients; (ii) Services delivered in a safe and welcoming environment (iii) Training and consistency of staff; (iv) Focus on dental education (v) Developing peer mentoring and peer support.ConclusionWhile the barriers to accessing dental care while homeless are well established and understood by healthcare practitioners, more work is required to gain consensus on the most effective way to deliver an innovative a sustainable dental service for patients experiencing homelessness. Previous negative experiences, lack of readily available information on services and barriers to access in the current system could be addressed by developing peer mentors within the homeless community, empowering individuals to share their knowledge and skills to support others in improving their oral health.
| S-EPMC11381413 | biostudies-literature