Applying modern measurement approaches to constructs relevant to evidence-based practice among Canadian physical and occupational therapists.
ABSTRACT: BACKGROUND:Evidence-based practice (EBP) is a complex process. To quantify it, one has to also consider individual and contextual factors using multiple measures. Modern measurement approaches are available to optimize the measurement of complex constructs. This study aimed to develop a robust measurement approach for constructs around EBP including practice, individual (e.g. knowledge, attitudes, confidence, behaviours), and contextual factors (e.g. resources). METHODS:One hundred eighty-one items arising from 5 validated EBP measures were subjected to an item analysis. Nominal group technique was used to arrive at a consensus about the content relevance of each item. Baseline questionnaire responses from a longitudinal study of the evolution of EBP in 128 new graduates of Canadian physical and occupational therapy programmes were analysed. Principles of Rasch Measurement Theory were applied to identify challenges with threshold ordering, item and person fit to the Rasch model, unidimensionality, local independence, and differential item functioning (DIF). RESULTS:The nominal group technique identified 70/181 items, and modified Delphi approach identified 68 items that fit a formative model (2 related EBP domains: self-use of EBP (9 items) and EBP activities (7 items)) or a reflective model (4 related EBP domains: attitudes towards EBP (17 items), self-efficacy (9 items), knowledge (11 items) and resources (15 items)). Rasch analysis provided a single score for reflective construct. Among attitudes items, 65% (11/17) fit the Rasch model, item difficulties ranged from - 7.51 to logits (least difficult) to + 5.04 logits (most difficult), and person separation index (PSI) = 0.63. Among self-efficacy items, 89% (8/9) fit the Rasch model, item difficulties ranged from - 3.70 to + 4.91, and PSI = 0.80. Among knowledge items, 82% (9/11) fit the Rasch model, item difficulties ranged from - 7.85 to 4.50, and PSI = 0.81. Among resources items, 87% (13/15) fit the Rasch model, item difficulties ranged from - 3.38 to 2.86, and PSI = 0.86. DIF occurred in 2 constructs: attitudes (1 by profession and 2 by language) and knowledge (1 by language and 2 by profession) arising from poor wording in the original version leading to poor translation. CONCLUSIONS:Rasch Measurement Theory was applied to develop a valid and reliable measure of EBP. Further modifications to the items can be done for subsequent waves of the survey.
Project description:To evaluate the psychometric properties of the Adult Strabismus-20 (AS-20)- a health-related quality of life (HRQoL) questionnaire in adults with strabismus, and if flawed, to revise the AS-20 and its subscales creating valid measurement scales.584 adults (meanage, 27.5 years) with strabismus were recruited from an outpatient clinic at a South Indian tertiary eye care centre and were administered the AS-20 questionnaire.The AS-20 was translated and back translated into two Indian languages. The AS-20 and its two 10-item subscales - 'psychosocial' and 'function'were assessed separately for fit to the Rasch model, including an assessment of the rating scale, unidimensionality (by principal components analysis), measurement precision by person separation reliability, PSR, targeting, and differential item functioning (DIF; notable > 1.0 logits).Response categories were not used as intended, thereby, required re-organization and reducing their number from 5 to 3. The AS-20 had adequate measurement precision (PSR = 0.87) but lacked unidimensionality; however, deletion of the six multi-dimensionality causing items and an additional three misfitting items resulted in 11-item unidimensional questionnaire (AS-11). Two items failed to satisfy the model expectations in the 'psychosocial' subscale and were deleted - resulting in an 8-item unidimensional scale with adequate PSR (0.81) and targeting (0.23 logits). One item misfit in the 'function' subscale and was deleted-resulting in a 9 item Rasch-revised unidimensional subscale with acceptable PSR (0.80) and targeting (0.97 logits).None of the items displayed notable DIF by age, gender and level of education.The AS-11 and its two Rasch-revised subscales - 8-item psychosocial and 9-item function subscale may be more appropriate than the original AS-20 and its two 10-item subscales for use as unidimensional measures of HRQoL in adults with strabismus in India. Further work is required to establish the validity of the revised rating scale.
Project description:For patient undergoing cataract surgery in India, existing patient-reported outcome (PRO) measures are either not culturally relevant, have not been adequately validated, or are too long to be used in a busy clinical setting. We sought to develop and validate a brief and culturally relevant point-of-care PRO measure to address this need.Twelve items from the Indian Visual Functioning Questionnaire (IND-VFQ) were selected based on preliminary data. Patients 18 years and older were prospectively recruited at Aravind Eye Care System in Madurai, India. Clinical and sociodemographic data were collected and the 12-item short-form IND-VFQ (SF-IND-VFQ) was administered pre- and post-operatively to 225 patients; Factor analysis and Rasch modeling was performed to assess its psychometric properties.One item that did not fit a unidimensional scale and had poor fit with the Rasch model was eliminated from the questionnaire. The remaining 11 items represented a single construct (no residual correlations>?0.1) and were largely unaffected by differential item functioning. Five items had disordered thresholds resolved by collapsing the response scale from four to three categories. The survey had adequate reliability (0.80) and good construct (infit range, 0.77-1.29; outfit range, 0.56-1.30) and content (item separation index, 5.87 logits) validity. Measurement precision was fair (person separation index, 1.97). There was evidence that items were not optimally targeted to patients' visual ability (preoperatively, -?1.92 logits; overall, -?3.41 logits), though the survey measured a very large effect (Cohen's d 1.80). In a subset of patients, the average time to complete the questionnaire was 2 min 6.3 s.The SF-IND-VFQ is a valid, reliable, sensitive, and rapidly administered point-of-care PRO measure to assess changes in visual functioning in patients undergoing cataract surgery in India.
Project description:OBJECTIVE:To investigate the measurement properties of the Functional Test of the Hemiparetic Upper Extremity (FTHUE) and examine how its score may or may not inform design of a rehabilitation program. DESIGN:The FTHUE was recently used in the Interdisciplinary Comprehensive Arm Rehabilitation Evaluation randomized controlled trial. This circumstance provided the opportunity to examine the psychometric properties of the FTHUE as it pertains to contemporary poststroke rehabilitation and recovery models. SETTING:Outpatient rehabilitation clinic. PARTICIPANTS:Participants (N=109; mean age, 61.2±13.5y; mean days poststroke, 46±20.3) with resultant hemiparesis in the upper extremity. INTERVENTIONS:Not applicable. MAIN OUTCOME MEASURES:Dimensionality was examined with confirmatory factor analysis (CFA), and person and item measures were derived with Rasch item response analysis. Therapists' notes were also reviewed. RESULTS:The CFA results support unidimensionality, and 16 of 17 items fit the Rasch model. The Rasch person separation (2.17) and item separation (4.50) indices, ability strata (3.22), person reliability (.82), and item reliability (.95) indicate good measurement properties. Item difficulties ranked from -6.46 to 3.43 logits; however, there was a substantial ceiling effect of person measures. Post hoc examination of therapists' written observations indicated that the scoring criteria are not sensitive to the movement strategy used for task completion. CONCLUSIONS:The FTHUE's item difficulty hierarchy indicated that scores adequately distinguished the ability to perform simple versus complex motor movements of functional tasks. However, the FTHUE scoring method did not allow inclusion of the type of movement strategy used to accomplish task items. Therefore, we suggest modifications to the FTHUE that would allow it to be used for collaborative treatment planning and align well with more contemporary perspectives on treatment theory.
Project description:The Patient-Rated Wrist Evaluation (PRWE) was developed as a wrist joint specific measure of pain and disability and evidence of sound validity has been accumulated through classical psychometric methods. Rasch analysis (RA) has been endorsed as a newer method for analyzing the clinical measurement properties of self-report outcome measures. The purpose of this study was to evaluate the PRWE using Rasch modeling.We employed the Rasch model to assess overall fit, response scaling, individual item fit, differential item functioning (DIF), local dependency, unidimensionality and person separation index (PSI). A convenience sample of 382 patients with distal radius fracture was recruited from the hand and upper limb clinic at large academic healthcare organization, London, Ontario, Canada, 6-month post-injury scores of the PRWE was used. RA was conducted on the 3 subscales (pain, specific activities, and usual activities) of the PRWE separately.The pain subscale adequately fit the Rasch model when item 4 "Pain - When it is at its worst" was deleted to eliminate non-uniform DIF by age group, and item 5 "How often do you have pain" was rescored by collapsing into 8 intervals to eliminate disordered thresholds. Uniform DIF for "Use my affected hand to push up from the chair" (by work status) and "Use bathroom tissue with my affected hand" (by injured hand) was addressed by splitting the items for analysis. After background rescoring of 2 items in pain subscale, 2 items in specific activities and 3 items in usual activities, all three subscales of the PRWE were well targeted and had high reliability (PSI?=?0.86). These changes provided a unidimensional, interval-level scaled measure.Like a previous analysis of the Patient-Rated Wrist and Hand Evaluation, this study found the PRWE could be fit to the Rasch model with rescoring of multiple items. However, the modifications required to achieve fit were not the same across studies, our fit statistics also suggested one of the pain items should be deleted. This study adds to the pool of evidence supporting the PRWE, but cannot confidently provide a Rasch-based scoring algorithm.
Project description:Objectives:To evaluate application of the Barthel Index (BI) in assessing basic activities of daily living (ADL) of patients with dementia using Rasch analysis. Design:A multi-country cross-sectional study. Setting and Participants:Nineteen long-term care facilities located in China, Japan, South Korea, and Thailand. A total of 644 patients with dementia were included. Methods:Unidimensionality, global and item fit, local dependence, person-item targeting, threshold disordering, and differential item functioning (DIF) were examined. Negative correlations between scores for DIF items and Neuropsychiatric Inventory Nursing Home version (NPI-NH) were evaluated. Results:Item reliability (1.0) and person reliability (.88) were acceptable. The Rasch dimension explained 72.9% of the variance (Eigenvalue = 27), while the first contrast explained 6.6% (Eigenvalue = 2.4). The "mobility" was misfitting to the Rasch model (infit mean square = 1.86). The overall difficulty of the BI exceeded patients' ability (person location = -2.27 logits). The "stairs climbing" and "mobility" showed narrow category thresholds (< 1.4 logits). The location of "controlling bladder" and "toilet use" overlapped. Removing "stairs climbing", collapsing categories with narrow threshold widths in "mobility", and combining "controlling bowel" and "controlling bladder" into one item, improved unidimensionality, and item fit of the scale. Only three items ("grooming", "dressing", and "toilet use") were free from DIF across countries. The scores for "feeding" were negatively related to scores for "disinhibition" (r = -0.46, P < 0.01), and scores for "controlling bowel" were negatively related to scores for "disinhibition" (r = -0.44, P < 0.01), "agitation" (r = -0.32, P < 0.05), and "aggression" (r = -0.27, P < 0.01) in Japanese samples. Conclusions and Implications:The performance of the BI for assessing patients with dementia might be compromised by misfit items, person-item mistargeting, measurement gaps, redundant items, narrow threshold width, and item bias. Mobility ability might not be helpful for determining capability of basic ADL in the patients. Comparisons of BI scores between countries should be undertaken with caution due to item bias. Neuropsychiatric symptoms might interact with basic ADL abilities of the patients. We will not suggest using the instrument in patients with dementia, without future refining to improve its performance.
Project description:Parents of a child with disability must cope with greater demands than those living with a healthy child. Coping refers to a person's cognitive or behavioral efforts to manage the demands of a stressful situation. The Coping Health Inventory for Parents (CHIP) is a well-recognized measure of coping among parents of chronically ill children and assesses different coping patterns using its three subscales. The purpose of this study was to provide further insights into the psychometric properties of the CHIP subscales in a sample of parents of children with disabilities.In this cross-sectional study, 220 parents (mean age, 33.4 years; 85% mothers) caring for a child with disability enrolled in special schools as well as in mainstream schools completed the 45-item CHIP. Rasch analysis was applied to the CHIP data and the psychometric performance of each of the three subscales was tested. Subscale revision was performed in the context of Rasch analysis statistics.Response categories were not used as intended, necessitating combining categories, thereby reducing the number from 4 to 3. The subscale - 'maintaining social support' satisfied all the Rasch model expectations. Four item misfit the Rasch model in the subscale -maintaining family integration', but their deletion resulted in a 15-item scale with items that fit the Rasch model well. The remaining subscale - 'understanding the healthcare situation' lacked adequate measurement precision (<2.0 logits).The current Rasch analyses add to the evidence of measurement properties of the CHIP and show that the two of its subscales (one original and the other revised) have good psychometric properties and work well to measure coping patterns in parents of children with disabilities. However the third subscale is limited by its inadequate measurement precision and requires more items.
Project description:The Swallowing Quality-of-Life Questionnaire (SWAL-QoL) is considered the gold standard for assessing health-related QoL in oropharyngeal dysphagia. The Dutch translation (DSWAL-QoL) and its adjusted version (aDSWAL-QoL) have been validated using classical test theory (CTT). However, these scales have not been tested against the Rasch measurement model, which is required to establish the structural validity and objectivity of the total scale and subscale scores. Thus, the purpose of this study was to examine the psychometric properties of these scales using item analysis according to the Rasch model.Item analysis with the Rasch model was performed using RUMM2030 software with previously collected data from a validation study of 108 patients. The assessment included evaluations of overall model fit, reliability, unidimensionality, threshold ordering, individual item and person fits, differential item functioning (DIF), local item dependency (LID) and targeting.The analysis could not establish the psychometric properties of either of the scales or their subscales because they did not fit the Rasch model, and multidimensionality, disordered thresholds, DIF, and/or LID were found. The reliability and power of fit were high for the total scales (PSI?=?0.93) but low for most of the subscales (PSI?<?0.70). The targeting of persons and items was suboptimal. The main source of misfit was disordered thresholds for both the total scales and subscales. Based on the results of the analysis, adjustments to improve the scales were implemented as follows: disordered thresholds were rescaled, misfit items were removed and items were split for DIF. However, the multidimensionality and LID could not be resolved. The reliability and power of fit remained low for most of the subscales.This study represents the first analyses of the DSWAL-QoL and aDSWAL-QoL with the Rasch model. Relying on the DSWAL-QoL and aDSWAL-QoL total and subscale scores to make conclusions regarding dysphagia-related HRQoL should be treated with caution before the structural validity and objectivity of both scales have been established. A larger and well-targeted sample is recommended to derive definitive conclusions about the items and scales. Solutions for the psychometric weaknesses suggested by the model and practical implications are discussed.
Project description:Objectives. To derive a shorter version of the Motor Activity Log Quality-of-Movement Scale (MAL-28) with enhanced content and construct validity. Design. Validation cohort. Setting. Outpatient rehabilitation within an academic laboratory. Participants. Retrospective consecutive sample of 149 community-dwelling adults with chronic mild/moderate upper-extremity hemiparesis caused by stroke or multiple sclerosis (MS). Intervention. Not applicable. Methods. Participants received the MAL-28 at baseline and following upper-extremity rehabilitation. Rasch Measurement Theory informed threshold ordering of scoring categories, tests of fit, differential item functioning, targeting, response dependency, local dependency, and reliability (person separation index [PSI]). Seasoned examiners rated the content validity of each item. Test-retest reliability of the revised scale was calculated. Results. We established content and construct validity for 18 items. The resultant 18-item MAL fit the model (?2 = 77.93; df = 72; P = .30) and targeted the population-that is, minimal floor (12.08%) or ceiling effects (0%), with acceptable reliability (PSI = 0.84) and good test-retest reliability [ICC(1, 1) = 0.86]. The hierarchy of item difficulty was independent of sex, age, affected side, diagnosis, or intervention type used, and there was local dependency in 3 pairs of items. Responses from a subsequent testing session were dependent on the responses from prior testing, indicating response dependency, for which a correction was proposed. Once response dependency was neutralized, there was a 15% greater treatment response. Conclusions. Content and construct validity are established for Rasch-based MAL-18 for chronic stages of stroke and MS. A Rasch-based conversion table enables clinical use of the MAL-18.
Project description:BACKGROUND: The Nottingham Health Profile index of Distress (NHPD) has been proposed as a generic undimensional 24-item measure of illness-related distress that is embedded in the Nottingham Health Profile (NHP). Data indicate that the NHPD may have psychometric advantages to the 6-dimensional NHP profile scores. Detailed psychometric evaluations are, however, lacking. Furthermore, to support the validity of the generic property of outcome measures evidence that scores can be interpreted in the same manner in different diagnostic groups are needed. It is currently unknown if NHPD scores have the same meaning across patient populations. This study evaluated the measurement properties and cross-diagnostic validity of the NHPD as a survey instrument among people with Parkinson's disease (PD) and peripheral arterial disease (PAD). METHODS: Data from 215 (PD) and 258 (PAD) people were Rasch analyzed regarding model fit, reliability, differential item functioning (DIF), unidimensionality and targeting. In cases of cross-diagnostic DIF this was adjusted for and the impact of DIF on the total score and person measures was assessed. RESULTS: The NHPD was found to have good overall and individual item fit in both disorders as well as in the pooled sample, but seven items displayed signs of cross-diagnostic DIF. Following adjustment for DIF some aspects of model fit were slightly compromised, whereas others improved somewhat. DIF did not impact total NHPD scores or resulting person measures, but the unadjusted scale displayed minor signs of multidimensionality. Reliability was > 0.8 in all within- and cross-diagnostic analyses. Items tended to represent more distress (mean, 0 logits) than that experienced by the sample (mean, -1.6 logits). CONCLUSION: This study supports the within- and cross-diagnostic validity of the NHPD as a survey tool among people with PD and PAD. We encourage others to reassess available NHP data within the NHPD framework to further evaluate the strengths and weaknesses of this simple patient-reported index of illness-related distress.
Project description:To develop and psychometrically evaluate a visual functioning questionnaire (VFQ) in an ultra-low vision (ULV) population.Questionnaire items, based on visual activities self-reported by a ULV population, were categorized by functional visual domain (e.g., mobility) and visual aspect (e.g., contrast) to ensure a representative distribution. In Round 1, an initial set of 149 items was generated and administered to 90 participants with ULV (visual acuity [VA] ≤ 20/500; mean [SD] age 61  years), including six patients with a retinal implant. Psychometric properties were evaluated through Rasch analysis and a revised set (150 items) was administered to 80 participants in Round 2.In Round 1, the person measure distribution (range, 8.6 logits) was centered at -1.50 logits relative to the item measures. In Round 2, the person measure distribution (range, 9.5 logits) was centered at -0.86 relative to the item mean. The reliability index in both rounds was 0.97 for Items and 0.99 for Persons. Infit analysis showed four underfit items in Round 1, five underfit items in Round 2 with a z-score greater than 4 cutoff. Principal component analysis on the residuals found 69.9% explained variance; the largest component in the unexplained variance was less than 3%.The ULV-VFQ, developed with content generated from a ULV population, showed excellent psychometric properties as well as superior measurement validity in a ULV population.The ULV-VFQ, part of the Prosthetic Low Vision Rehabilitation (PLoVR) development program, is a new VFQ developed for assessment of functional vision in ULV populations.