Development and Validation of the Open Matrices Item Bank.
Ontology highlight
ABSTRACT:
SUBMITTER: Koch M
PROVIDER: S-EPMC9326670 | biostudies-literature | 2022 Jul
REPOSITORIES: biostudies-literature
Publications
Development and Validation of the Open Matrices Item Bank.
Koch Marco M Spinath Frank M FM Greiff Samuel S Becker Nicolas N
Journal of Intelligence 20220713 3
Figural matrices tasks are one of the most prominent item formats used in intelligence tests, and their relevance for the assessment of cognitive abilities is unquestionable. However, despite endeavors of the open science movement to make scientific research accessible on all levels, there is a lack of royalty-free figural matrices tests. The Open Matrices Item Bank (OMIB) closes this gap by providing free and unlimited access (GPLv3 license) to a large set of empirically validated figural matri ...[more]
Similar Datasets
Project description:PurposeCurrent patient-reported outcomes for female urinary incontinence (UI) are limited by their inability to be tailored. Our objective is to describe the development and field testing of seven item banks designed to measure domains identified as important UI in females (UIf). We also describe the calibration and validation properties of the UIf-item banks, which allow for more efficient computerized adaptive testing (CAT) in the future.MethodsThe UIf-measures included 168 items covering seven domains: Stress UI (SUI), overactive bladder (OAB), urinary frequency, physical, social and emotional health impact and adaptation. Items underwent rigorous qualitative development and psychometric testing across two sites. Items were calibrated using item response theory and evaluated for internal consistency, construct validity and responsiveness.ResultsA total of 750 women (249 SUI, 249 OAB and 252 mixed UI) participated. Mean age was 55 ± 14 years, and 23 % were Hispanic and 80 % white. In addition to face and content validity, the measures demonstrated good internal consistency (coefficient alpha 0.92-0.98) and unidimensionality. There was evidence for construct validity with moderate-to-strong correlations with the UDI (r's ≥ 0.6) and IIQ (r's = ≥0.6) scales. The measures were responsive to change for SUI treatment (paired t test p < .001, ES range 1.3-2.9; SRM range 1.3-2.5) and OAB treatment (paired t test p < .05 for all domains except social health impact and adaptation, ES range 0.3-1.5, SRM range 0.4-1.0). The measures were responsive based on concurrent changes with the UDI and IIQ (p < 0.05). CAT versions were developed and pilot-tested.ConclusionsThe UIf-item banks demonstrate good psychometric characteristics and are a sufficiently valid set of customizable tools for measuring UI symptoms and life impact.
| S-EPMC4894000 | biostudies-literature
Project description:ObjectivesFunctional outcomes following cochlear implantation have traditionally been focused on word and sentence recognition, which, although important, do not capture the varied communication and other experiences of adult cochlear implant (CI) users. Although the inadequacies of speech recognition to quantify CI user benefits are widely acknowledged, rarely have adult CI user outcomes been comprehensively assessed beyond these conventional measures. An important limitation in addressing this knowledge gap is that patient-reported outcome measures have not been developed and validated in adult CI patients using rigorous scientific methods. The purpose of the present study is to build on our previous work and create an item bank that can be used to develop new patient-reported outcome measures that assess CI quality of life (QOL) in the adult CI population.DesignAn online questionnaire was made available to 500 adult CI users who represented the adult CI population and were recruited through a consortium of 20 CI centers in the United States. The questionnaire included the 101 question CIQOL item pool and additional questions related to demographics, hearing and CI history, and speech recognition scores. In accordance with the Patient-Reported Outcomes Measurement Information System, responses were psychometrically analyzed using confirmatory factor analysis and item response theory.ResultsOf the 500 questionnaires sent, 371 (74.2%) subjects completed the questionnaire. Subjects represented the full range of age, durations of CI use, speech recognition abilities, and listening modalities of the adult CI population; subjects were implanted with each of the three CI manufacturers' devices. The initial item pool consisted of the following domain constructs: communication, emotional, entertainment, environment, independence, listening effort, and social. Through psychometric analysis, after removing locally dependent and misfitting items, all of the domains were found to have sound psychometric properties, with the exception of the independence domain. This resulted in a final CIQOL item bank of 81 items in 6 domains with good psychometric properties.ConclusionsOur findings reveal that hypothesis-driven quantitative analyses result in a psychometrically sound CIQOL item bank, organized into unique domains comprised of independent items which measure the full ability range of the adult CI population. The final item bank will now be used to develop new instruments that evaluate and differentiate adult CIQOL across the patient ability spectrum.
| S-EPMC6749825 | biostudies-literature
Project description:BackgroundMedication adherence is important in managing the progression of chronic diseases. A promising approach to reduce cognitive burden when measuring medication adherence lies in the use of computer-adaptive tests (CATs) or in the development of shorter patient-reported outcome measures (PROMs). However, the lack of an item bank currently hampers this progress.ObjectiveWe aim to develop an item bank to measure general medication adherence.MethodsUsing the preferred reporting items for systematic review and meta-analysis (PRISMA), articles published before October 2019 were retrieved from PubMed, Embase, CINAHL, the Cochrane Library, and Web of Science. Items from existing PROMs were classified and selected ("binned" and "winnowed") according to standards published by the Patient-Reported Outcomes Measurement Information System (PROMIS) Cooperative Group.ResultsA total of 126 unique PROMs were identified from 213 studies in 48 countries. Items from the literature review (47 PROMs with 579 items for which permission has been obtained) underwent binning and winnowing. This resulted in 421 candidate items (77 extent of adherence and 344 reasons for adherence).ConclusionsWe developed an item bank for measuring general medication adherence using items from validated PROMs. This will allow researchers to create new PROMs from selected items and provide the foundation to develop CATs.
| S-EPMC7582150 | biostudies-literature
Project description:PURPOSE:A sense of meaning and purpose is important for people living with acute and chronic illness. It can buffer the effects of stress and facilitate adaptive coping. As part of the Patient-Reported Outcomes Measurement Information System (PROMIS), we developed and validated an item response theory (IRT)-based measure of meaning and purpose in life. METHODS:Informed by a literature review and patient and content-expert input, we wrote 52 items to assess meaning and purpose and administered them to a general population sample (n = 1000) along with the Meaning in Life Questionnaire-Presence of Meaning Subscale (MLQ-Presence) and the Life Engagement Test (LET). We split the sample in half for exploratory factor analysis (EFA) followed by confirmatory factor analysis (CFA). IRT analyses included assessments of differential item functioning (DIF). RESULTS:Participants had a mean age of 47.8 years and 50.3% were male. EFA revealed one dominant factor and CFA yielded a good fitting model for a 37-item bank (CFI = 0.962, TLI = 0.960, RMSEA = 0.085). All items were free of sex, age, education, and race DIF. Internal consistency reliability estimates ranged from α = 0.90 (4-item short form) to α = 0.98 (37-item bank). The 8-item Meaning and Purpose short form was correlated with the MLQ-Presence (r = 0.89), the LET (r = 0.79), and the full PROMIS Meaning and Purpose item bank (r = 0.98). CONCLUSIONS:The PROMIS Meaning and Purpose measures demonstrated sufficient unidimensionality and displayed good internal consistency, model fit, and convergent validity. Further psychometric testing of the PROMIS Meaning and Purpose item bank and short forms in people with chronic diseases will help evaluate the generalizability of this new tool.
| S-EPMC7367748 | biostudies-literature
Project description:ObjectiveSince the implementation of value-based healthcare, there has been a growing emphasis on utilizing patient-reported experience measures (PREMs) to enhance the quality of care. However, the current PREMs are primarily generic and static, whereas healthcare is constantly evolving and encompasses a wide variety of aspects that impact care quality. To continuously improve care requires a dynamic PREM. The aim of this study was to propose an item bank for the establishment of a dynamic and care-specific patient-reported evaluation.MethodsIn co-creation with patients, a mixed methods study was conducted involving: (1) an explorative review of the literature, (2) a focus group analysis with (ex-)patients, (3) qualitative analyses to formulate themes, and (4) a quantitative selection of items by patients and experts through prioritization.ResultsEight existing PREMs were evaluated. After removing duplicates, 141 items were identified. Through qualitative analyses of the focus group in which the patient journey was discussed, eight themes were formulated: "Organization of healthcare," "Competence of healthcare professionals," "Communication," "Information & services," "Patient empowerment," "Continuity & informal care," "Environment," and "Technology." Seven patients and eleven professionals were asked to prioritize what they considered the most important items. From this, an item bank with 76 items was proposed.ConclusionIn collaboration with patients and healthcare professionals, we have proposed a PREM-item bank to evaluate the experiences of patients' receiving cancer care in an outpatient clinic. This item bank is the first step to dynamically assess the quality of cancer care provided in an outpatient setting.
| S-EPMC10786971 | biostudies-literature
Project description:BackgroundPresbyopia is an age-related condition that causes a decreased ability to focus on nearby objects. Multifocal contact lenses are commonly used to address this issue. However, there seems to be a notable dissatisfaction among multifocal contact lens wearers. The absence of a reliable instrument to measure the patient's perspective, despite the widespread use of this method, highlights the need for further research in this area.ObjectiveThe objective of this study is to develop an item-bank integrating all domains necessary to assess the patient's perspective on multifocal contact lens performance, offering a comprehensive measure. The item-bank will ensure a high level of content validity, be self-administered, and will initially be available in Spanish. The aim of this tool is to serve as a valuable resource for research and optometric clinics, facilitating the follow-up of patients with presbyopia who wear multifocal contact lenses or those who are newly starting to use them.MethodologyThe MCL-PRO item bank, followed a systematic and step-wise inductive approach to gather information, following the recommendations outlined in the COSMIN guidelines and similar studies. The process involved the following steps: (1) Literature review and relevant existing items identification (2) Social media review, (3) Semi-structured focus groups, (4) performing qualitative analysis, (5) refining and revising the items, and (6) generating the content of the item bank.ResultsA total of 575 items were included in the item-bank hosted under 8 different domains that were found to be important for presbyopic population: visual symptoms (213), activity limitation (111), ocular symptoms (135), convenience (36), emotional well-being (33), general symptoms (16), cognitive issues (21) and economic issues (10).ConclusionThe item-bank created has followed standardised methodology for its development and encloses all the aspects for MCL performance evaluation from patients perspective.
| S-EPMC10853121 | biostudies-literature
Project description:Existing non-verbal ability tests are typically protected by copyright, preventing them from being freely adapted or computerized. Working towards an open science framework, we provide 80 novel, open-access abstract reasoning items, an online implementation and item-level data from 659 participants aged between 11 and 33 years: the matrix reasoning item bank (MaRs-IB). Each MaRs-IB item consists of an incomplete matrix containing abstract shapes. Participants complete the matrices by identifying relationships between the shapes. Our data demonstrate age differences in non-verbal reasoning accuracy, which increased during adolescence and stabilized in early adulthood. There was a slight linear increase in response times with age, resulting in a peak in efficiency (i.e. a measure combining speed and accuracy) in late adolescence. Overall, the data suggest that the MaRs-IB is sensitive to developmental differences in reasoning accuracy. Further psychometric validation is recommended.
| S-EPMC6837216 | biostudies-literature
Project description:BackgroundOrthoses are often prescribed to improve mobility of people with chronic health conditions that affect lower limb function. Patient-reported survey instruments can be used to measure aspects of mobility that cannot be easily assessed in clinical or research settings. A population-specific item bank could be designed to measure aspects of mobility that are most important to lower limb orthosis users and used to evaluate the effects of orthoses.ObjectiveTo develop items for a new survey instrument to measure mobility of lower limb orthosis users.DesignSurvey items were developed using a qualitative item review process.SettingFocus groups were held by video conferencing. Cognitive interviews were conducted by telephone.ParticipantsFocus group and cognitive interview participants were adults with at least 6 months of experience using a lower limb orthosis that extended from the foot to a level above the ankle.MethodsResearch methods included focus groups with lower limb orthosis users, an item generation and reduction process that involved a stakeholder advisory panel, and cognitive interviews with target respondents.ResultsA total of 1180 extant items were identified in a literature review. Focus group participants (n = 29) provided feedback that informed the suitability of a construct definition and conceptual model. An advisory panel contributed to the selection of 118 candidate items for measuring orthotic mobility. Feedback from cognitive interview participants (n = 30) informed removal or revision of problematic items, resulting in a candidate bank of 100 mobility items.ConclusionsThe rigorous qualitative methods applied here resulted in a large set of candidate items that spanned a range of situations relevant to moving with a lower limb orthosis. Next steps include administration of the candidate items to a large sample of lower limb orthosis users and calibration of the item bank.
| S-EPMC10119328 | biostudies-literature
Project description:ObjectiveTo improve the practical use of the short forms (SFs) developed from the item bank, we compared the measurement precision of the 4- and 8-item SFs generated from a motor item bank composed of the FIM and the Minimum Data Set (MDS).DesignThe FIM-MDS motor item bank allowed scores generated from different instruments to be co-calibrated. The 4- and 8-item SFs were developed based on Rasch analysis procedures. This article compared person strata, ceiling/floor effects, and test SE plots for each administration form and examined 95% confidence interval error bands of anchored person measures with the corresponding SFs. We used 0.3 SE as a criterion to reflect a reliability level of .90.SettingVeterans' inpatient rehabilitation facilities and community living centers.ParticipantsVeterans (N=2500) who had both FIM and the MDS data within 6 days during 2008 through 2010.InterventionsNot applicable.Main outcome measuresFour- and 8-item SFs of FIM, MDS, and FIM-MDS motor item bank.ResultsSix SFs were generated with 4 and 8 items across a range of difficulty levels from the FIM-MDS motor item bank. The three 8-item SFs all had higher correlations with the item bank (r=.82-.95), higher person strata, and less test error than the corresponding 4-item SFs (r=.80-.90). The three 4-item SFs did not meet the criteria of SE <0.3 for any theta values.ConclusionsEight-item SFs could improve clinical use of the item bank composed of existing instruments across the continuum of care in veterans. We also found that the number of items, not test specificity, determines the precision of the instrument.
| S-EPMC8895537 | biostudies-literature
Project description:Matrix reasoning tasks are among the most widely used measures of cognitive ability in the behavioral sciences, but the lack of matrix reasoning tests in the public domain complicates their use. Here, we present an extensive investigation and psychometric validation of the matrix reasoning item bank (MaRs-IB), an open-access set of matrix reasoning items. In a first study, we calibrate the psychometric functioning of the items in the MaRs-IB in a large sample of adult participants (N = 1501). Using additive multilevel item structure models, we establish that the MaRs-IB has many desirable psychometric properties: its items span a wide range of difficulty, possess medium-to-large levels of discrimination, and exhibit robust associations between item complexity and difficulty. However, we also find that item clones are not always psychometrically equivalent and cannot be assumed to be exchangeable. In a second study, we demonstrate how experimenters can use the estimated item parameters to design new matrix reasoning tests using optimal item assembly. Specifically, we design and validate two new sets of test forms in an independent sample of adults (N = 600). We find these new tests possess good reliability and convergent validity with an established measure of matrix reasoning. We hope that the materials and results made available here will encourage experimenters to use the MaRs-IB in their research.
| S-EPMC10551052 | biostudies-literature