Project description:BackgroundMany research domains still heavily rely on paper-based data collection procedures, despite numerous associated drawbacks. The QuestionSys framework is intended to empower researchers as well as clinicians without programming skills to develop their own smart mobile apps in order to collect data for their specific scenarios.ObjectiveIn order to validate the feasibility of this model-driven, end-user programming approach, we conducted a study with 80 participants.MethodsAcross 2 sessions (7 days between Session 1 and Session 2), participants had to model 10 data collection instruments (5 at each session) with the developed configurator component of the framework. In this context, performance measures like the time and operations needed as well as the resulting errors were evaluated. Participants were separated into two groups (ie, novices vs experts) based on prior knowledge in process modeling, which is one fundamental pillar of the QuestionSys framework.ResultsStatistical analysis (t tests) revealed that novices showed significant learning effects for errors (P=.04), operations (P<.001), and time (P<.001) from the first to the last use of the configurator. Experts showed significant learning effects for operations (P=.001) and time (P<.001), but not for errors as the experts' errors were already very low at the first modeling of the data collection instrument. Moreover, regarding the time and operations needed, novices got significantly better at the third modeling task than experts were at the first one (t tests; P<.001 for time and P=.002 for operations). Regarding errors, novices did not get significantly better at working with any of the 10 data collection instruments than experts were at the first modeling task, but novices' error rates for all 5 data collection instruments at Session 2 were not significantly different anymore from those of experts at the first modeling task. After 7 days of not using the configurator (from Session 1 to Session 2), the experts' learning effect at the end of Session 1 remained stable at the beginning of Session 2, but the novices' learning effect at the end of Session 1 showed a significant decay at the beginning of Session 2 regarding time and operations (t tests; P<.001 for time and P=.03 for operations).ConclusionsIn conclusion, novices were able to use the configurator properly and showed fast (but unstable) learning effects, resulting in their performances becoming as good as those of experts (which were already good) after having little experience with the configurator. Following this, researchers and clinicians can use the QuestionSys configurator to develop data collection apps for smart mobile devices on their own.

Project description:BackgroundImplementation strategies have flourished in an effort to increase integration of research evidence into clinical practice. Most strategies are complex, socially mediated processes. Many are complicated, expensive, and ultimately impractical to deliver in real-world settings. The field lacks methods to assess the extent to which strategies are usable and aligned with the needs and constraints of the individuals and contexts who will deliver or receive them. Drawn from the field of human-centered design, cognitive walkthroughs are an efficient assessment method with potential to identify aspects of strategies that may inhibit their usability and, ultimately, effectiveness. This article presents a novel walkthrough methodology for evaluating strategy usability as well as an example application to a post-training consultation strategy to support school mental health clinicians to adopt measurement-based care.MethodThe Cognitive Walkthrough for Implementation Strategies (CWIS) is a pragmatic, mixed-methods approach for evaluating complex, socially mediated implementation strategies. CWIS includes six steps: (1) determine preconditions; (2) hierarchical task analysis; (3) task prioritization; (4) convert tasks to scenarios; (5) pragmatic group testing; and (6) usability issue identification, classification, and prioritization. A facilitator conducted two group testing sessions with clinician users (N = 10), guiding participants through 6 scenarios and 11 associated subtasks. Clinicians reported their anticipated likelihood of completing each subtask and provided qualitative justifications during group discussion. Following the walkthrough sessions, users completed an adapted quantitative assessment of strategy usability.ResultsAverage anticipated success ratings indicated substantial variability across participants and subtasks. Usability ratings (scale 0-100) of the consultation protocol averaged 71.3 (SD = 10.6). Twenty-one usability problems were identified via qualitative content analysis with consensus coding, and classified by severity and problem type. High-severity problems included potential misalignment between consultation and clinical service timelines as well as digressions during consultation processes.ConclusionsCWIS quantitative usability ratings indicated that the consultation protocol was at the low end of the "acceptable" range (based on norms from the unadapted scale). Collectively, the 21 resulting usability issues explained the quantitative usability data and provided specific direction for usability enhancements. The current study provides preliminary evidence for the utility of CWIS to assess strategy usability and generate a blueprint for redesign.

Project description:BackgroundThere are differences of opinion regarding the selection of the most practical usability evaluation method among different methods. The present study aimed to compare two expert-based evaluation methods in order to assess a nursing module as the most widely used module of a Hospital Information System (HIS).MethodsFive independent evaluators used the Heuristic Evaluation (HE) and Cognitive Walkthrough (CW) methods to evaluate the nursing module of Shafa HIS. In this regard, the number and severity of the recognized problems according to the usability attributes were compared using two evaluation methods.ResultsThe HE and CW evaluation methods resulted in the identification of 104 and 24 unique problems, respectively, of which 33.3% of recognized problems in the CW evaluation method overlapped with the HE method. The average severity of the recognized problems was considered to be minor (2.34) in the HE method and major (2.77) in the CW evaluation method. There was a significant difference in terms of the total number and average severity of the recognized problems by these methods (P < 0.001). Based on the usability attribute, the HE method identified a larger number of problems concerning all usability attributes, and a significant difference was observed in terms of the number of recognized problems in both methods for all attributes except 'memorability'. Also, there was a significant difference between the two methods based on the average severity of recognized problems only in terms of 'learnability'.ConclusionThe HE method identified more problems with lower average severity while the CW was able to recognize fewer problems with higher average severity. Regarding the evaluation goal, the HE method was able to be used to improve the effectiveness and satisfaction of the HIS. Furthermore, the CW evaluation method is recommended to identify usability problems with the highest average severity, especially in terms of 'learnability'.

Project description:Objective/aimGood design of cancer registry systems makes them easy to use, while poor design of their user interfaces leads to user dissatisfaction and resistance. The objective of this study was to evaluate the usability of a cancer registry system using Cognitive Walkthrough (CW) and to assess users' agreement with its usability problems.MethodsCW was used to evaluate the registry system. We developed a checklist to help evaluators speed up the evaluation process, a problems form to collect the usability issues identified by the evaluators, and a problems severity form to determine the severity of problems by the evaluators. The problems were classified into two categories according to the CW questions and the system tasks. The agreement of the users with the system problems was examined by an online questionnaire. Users' agreement with the problems was then analyzed using the Interclass Correlation Coefficient in the SPSS 22 (Statistical Package for Social Science).ResultsIn this study, 114 problems were identified. In the categorization of problems based on the CW questions, 41% (n = 47) of the problems concerned the issue of "users do not know what to do at each stage of working with the system", 24% (n = 27) were classified as "users cannot link what they intend to do with system controls", and 22% (n = 25) were related to "user's lack of understanding of the system processes". Based on user tasks, about 36% (n = 41) of the problems were related to "removing patient duplication" and 33% (n = 38) were related to "registration of patient identification information". User agreement with the problems was high (CI 95% = 0.9 (0.96, 0.98)).ConclusionSystem problems often originate from user ignorance about what to do at each stage of using the system. Also, half of the system problems concern a mismatch between what users want to do and the system controls, or a lack of understanding about what the system does at different stages. Therefore, to avoid user confusion, designers should use clues and guides on the screen for users, design controls consistent with the user model of thinking, and provide appropriate feedback after each user action to help users understand what the system is doing. The high agreement of users with the problems showed that in the absence of users system designers can use CW to identify the problems that users face in the real environment.

Project description:Asynchronous telerehabilitation in which computer-based interventions are remotely monitored and adapted offline is an emerging service delivery model in the rehabilitation of communication disorders. The asynchronous nature of this model may hold a benefit over its synchronous counterpart by eliminating scheduling issues and thus improving efficiency in a healthcare landscape of constrained resource allocation. The design of asynchronous telerehabilitation platforms should therefore ensure efficiency and flexibility. The authors have been engaged in a program of research to develop and evaluate an asynchronous telerehabilitation platform for use in speech-language pathology. eSALT is a novel asynchronous telerehabilitation platform in which clinicians design and individualize therapy tasks for transfer to a client's mobile device. An inbuilt telerehabilitation module allows for remote monitoring and updating of tasks. This paper introduces eSALT and reports outcomes from an usability study that considered the needs of two end-user groups, people with aphasia and clinicians, in the on-going refinement of eSALT. In the study participants with aphasia were paired with clinicians who used eSALT to design and customize therapy tasks. After training on the mobile device the participants engaged in therapy at home for a period of 3 weeks, while clinicians remotely monitored and updated tasks. Following the home trial, participants, and clinicians engaged in semi-structured interviews and completed surveys on the usability of eSALT and their satisfaction with the platform. Content analysis of data involving five participants and three clinicians revealed a number of usability themes including ease of use, user support, satisfaction, limitations, and potential improvements. These findings were translated into a number of refinements of the eSALT platform including the development of a client interface for use on the Apple iPad®, greater variety in feedback options to both the participant and clinician, automatic transfer of results to the clinician, and expansion of the task template list. This research highlights the importance of including end-users in the process of technology refinement, in order to ensure effective and efficient use of the technology. Future directions for research are discussed including clinical trials in which the effectiveness of and adherence to intervention protocols using asynchronous telerehabilitation are examined.

Project description:IntroductionIncreasing numbers of ambulance calls, vacant positions and growing workloads in Emergency Medicine (EM) are increasing the pressure to find adequate solutions. With telemedicine providing health-care services by bridging large distances, connecting remote providers and even patients while using modern communication technologies, such a technology seems beneficial. As the process of developing an optimal solution is challenging, a need to quantify involved processes could improve implementation. Existing models are based on qualitative studies although standardised questionnaires for factors such as Usability, Acceptability and Effectiveness exist.MethodsA survey was provided to participants within a German county. It was based on telemedical surveys, the System Usabilty Scale (SUS) and earlier works describing Usability, Acceptability and Effectiveness. Meanwhile a telemedical system was introduced in the investigated county. A comparison between user-groups aswell as an exploratory factor analysis (EFA) was performed.ResultsOf n = 91 included participants n = 73 (80,2%) were qualified as emergency medical staff (including paramedics n = 36 (39,56%), EMTs n = 28 (30,77%), call handlers n = 9 (9,89%)) and n = 18 (19,8%) as emergency physicians. Most participants approved that telemedicine positively impacts EM and improved treatment options with an overall Usabilty Score of 68,68. EFA provided a 3-factor solution involving Usability, Acceptability and Effectiveness.DiscussionWith our results being comparable to earlier studies but telemedicine only having being sparsely introduced, a positive attitude could still be attested. While our model describes 51,28% of the underlying factors, more research is needed to identify further influences. We showed that Usability is correlated with Acceptability (strong effect), Usability and Effectiveness with a medium effect, likewise Acceptability and Effectiveness. Therefore available systems need to improve. Our approach can be a guide for decision makers and developers, that a focus during implementation must be on improving usability and on a valid data driven implementation process.

Dataset Information

Active Involvement of End-Users in an EHR Procurement Process: a Usability Walkthrough Feasibility Case Study.

Publications

Active Involvement of End-Users in an EHR Procurement Process: a Usability Walkthrough Feasibility Case Study.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets