Measures of organizational culture, organizational climate, and implementation climate in behavioral health: A systematic review.
Ontology highlight
ABSTRACT:
SUBMITTER: Powell BJ
PROVIDER: S-EPMC9978646 | biostudies-literature | 2021 Jan-Dec
REPOSITORIES: biostudies-literature
Publications
Measures of organizational culture, organizational climate, and implementation climate in behavioral health: A systematic review.
Powell Byron J BJ Mettert Kayne D KD Dorsey Caitlin N CN Weiner Bryan J BJ Stanick Cameo F CF Lengnick-Hall Rebecca R Ehrhart Mark G MG Aarons Gregory A GA Barwick Melanie A MA Damschroder Laura J LJ Lewis Cara C CC
Implementation research and practice 20210101
<h4>Background</h4>Organizational culture, organizational climate, and implementation climate are key organizational constructs that influence the implementation of evidence-based practices. However, there has been little systematic investigation of the availability of psychometrically strong measures that can be used to assess these constructs in behavioral health. This systematic review identified and assessed the psychometric properties of measures of organizational culture, organizational cl ...[more]
Similar Datasets
Project description:BackgroundPrimary care is increasingly contributing to improving the quality of patient care. This has imposed significant demands on clinicians with rising needs and limited resources. Organizational culture and climate have been found to be crucial in improving workforce well-being and hence quality of care. The objectives of this study are to identify organizational culture and climate measures used in primary care from 2008 to 2019 and evaluate their psychometric properties.MethodsData sources include PubMed, PsycINFO, HAPI, CINAHL, and Mental Measurements Yearbook. Bibliographies of relevant articles were reviewed and a cited reference search in Scopus was performed. Eligibility criteria include primary health care professionals, primary care settings, and use of measures representing the general concept of organizational culture and climate. Consensus-Based Standards for the selection of health Measurement Instruments (COSMIN) guidelines were followed to evaluate individual studies for methodological quality, rate results of measurement properties, qualitatively pool studies by measure, and grade evidence.ResultsOf 1745 initial studies, 42 studies met key study inclusion criteria, with 27 measures available for review (16 for organizational culture, 11 for organizational climate). There was considerable variability in measures, both conceptually and in psychometric quality. Many reported limited or no psychometric information.DiscussionNotable measures selected for frequent use and strength and applicability of measurement properties include the Culture Questionnaire adapted for health care settings, Practice Culture Assessment, and Medical Group Practice Culture Assessment for organizational culture. Notable climate measures include the Nurse Practitioner Primary Care Organizational Climate Questionnaire, Practice Climate Survey, and Task and Relational Climate Scale. This synthesis and appraisal of organizational culture and climate measures can help investigators make informed decisions in choosing a measure or deciding to develop a new one. In terms of limitations, ratings should be considered conservative due to adaptations of the COSMIN protocol for clinician populations.Prospero registration numberCRD 42019133117.
| S-EPMC7878641 | biostudies-literature
Project description:BackgroundTwo of the current methodological barriers to implementation science efforts are the lack of agreement regarding constructs hypothesized to affect implementation success and identifiable measures of these constructs. In order to address these gaps, the main goals of this paper were to identify a multi-level framework that captures the predominant factors that impact implementation outcomes, conduct a systematic review of available measures assessing constructs subsumed within these primary factors, and determine the criterion validity of these measures in the search articles.MethodWe conducted a systematic literature review to identify articles reporting the use or development of measures designed to assess constructs that predict the implementation of evidence-based health innovations. Articles published through 12 August 2012 were identified through MEDLINE, CINAHL, PsycINFO and the journal Implementation Science. We then utilized a modified five-factor framework in order to code whether each measure contained items that assess constructs representing structural, organizational, provider, patient, and innovation level factors. Further, we coded the criterion validity of each measure within the search articles obtained.ResultsOur review identified 62 measures. Results indicate that organization, provider, and innovation-level constructs have the greatest number of measures available for use, whereas structural and patient-level constructs have the least. Additionally, relatively few measures demonstrated criterion validity, or reliable association with an implementation outcome (e.g., fidelity).DiscussionIn light of these findings, our discussion centers on strategies that researchers can utilize in order to identify, adapt, and improve extant measures for use in their own implementation research. In total, our literature review and resulting measures compendium increases the capacity of researchers to conceptualize and measure implementation-related constructs in their ongoing and future research.
| S-EPMC3598720 | biostudies-literature
Project description:Both organizational culture and climate are associated with service quality and outcomes across youth-service settings. Increasing evidence indicates capacity of organizational interventions to promote a positive and effective culture and climate. Less is known about common intervention components across studies and service settings. The current systematic review reviewed 9223 citations and identified 31 studies, across six youth-service settings, measuring changes over time in organizational culture and climate following implementation of an organizational or workforce support intervention. Results highlight the promise of organizational interventions, a need for more comparison and randomized designs, and future directions for maximizing capacity of organizations to promote health for frontline providers and the children they serve.
| S-EPMC9872793 | biostudies-literature
Project description:BackgroundPublic policy has tremendous impacts on population health. While policy development has been extensively studied, policy implementation research is newer and relies largely on qualitative methods. Quantitative measures are needed to disentangle differential impacts of policy implementation determinants (i.e., barriers and facilitators) and outcomes to ensure intended benefits are realized. Implementation outcomes include acceptability, adoption, appropriateness, compliance/fidelity, feasibility, penetration, sustainability, and costs. This systematic review identified quantitative measures that are used to assess health policy implementation determinants and outcomes and evaluated the quality of these measures.MethodsThree frameworks guided the review: Implementation Outcomes Framework (Proctor et al.), Consolidated Framework for Implementation Research (Damschroder et al.), and Policy Implementation Determinants Framework (Bullock et al.). Six databases were searched: Medline, CINAHL Plus, PsycInfo, PAIS, ERIC, and Worldwide Political. Searches were limited to English language, peer-reviewed journal articles published January 1995 to April 2019. Search terms addressed four levels: health, public policy, implementation, and measurement. Empirical studies of public policies addressing physical or behavioral health with quantitative self-report or archival measures of policy implementation with at least two items assessing implementation outcomes or determinants were included. Consensus scoring of the Psychometric and Pragmatic Evidence Rating Scale assessed the quality of measures.ResultsDatabase searches yielded 8417 non-duplicate studies, with 870 (10.3%) undergoing full-text screening, yielding 66 studies. From the included studies, 70 unique measures were identified to quantitatively assess implementation outcomes and/or determinants. Acceptability, feasibility, appropriateness, and compliance were the most commonly measured implementation outcomes. Common determinants in the identified measures were organizational culture, implementation climate, and readiness for implementation, each aspects of the internal setting. Pragmatic quality ranged from adequate to good, with most measures freely available, brief, and at high school reading level. Few psychometric properties were reported.ConclusionsWell-tested quantitative measures of implementation internal settings were under-utilized in policy studies. Further development and testing of external context measures are warranted. This review is intended to stimulate measure development and high-quality assessment of health policy implementation outcomes and determinants to help practitioners and researchers spread evidence-informed policies to improve population health.RegistrationNot registered.
| S-EPMC7304175 | biostudies-literature
Project description:OBJECTIVE: To review and characterize existing health care efficiency measures in order to facilitate a common understanding about the adequacy of these methods. DATA SOURCES: Review of the MedLine and EconLit databases for articles published from 1990 to 2008, as well as search of the "gray" literature for additional measures developed by private organizations. STUDY DESIGN: We performed a systematic review for existing efficiency measures. We classified the efficiency measures by perspective, outputs, inputs, methods used, and reporting of scientific soundness. PRINCIPAL FINDINGS: We identified 265 measures in the peer-reviewed literature and eight measures in the gray literature, with little overlap between the two sets of measures. Almost all of the measures did not explicitly consider the quality of care. Thus, if quality varies substantially across groups, which is likely in some cases, the measures reflect only the costs of care, not efficiency. Evidence on the measures' scientific soundness was mostly lacking: evidence on reliability or validity was reported for six measures (2.3 percent) and sensitivity analyses were reported for 67 measures (25.3 percent). CONCLUSIONS: Efficiency measures have been subjected to few rigorous evaluations of reliability and validity, and methods of accounting for quality of care in efficiency measurement are not well developed at this time. Use of these measures without greater understanding of these issues is likely to engender resistance from providers and could lead to unintended consequences.
| S-EPMC2699907 | biostudies-other
Project description: Not available
| S-EPMC9924289 | biostudies-literature
Project description:BackgroundLearning health systems strive to continuously integrate data and evidence into practice to improve patient outcomes and ensure value-based healthcare. While the LHS concept is gaining traction, the operationalization of LHSs is underexplored.ObjectiveTo identify and synthesize the existing evidence on the implementation and evaluation of advancing learning health systems across international health care settings.MethodsA mixed methods systematic review was conducted. Six databases (CINAHL, Embase, Medline, PAIS, Scopus and Nursing at Allied Health Database) were searched up to July 2022 for terms related to learning health systems, implementation, and evaluation measures. Any study design, health care setting and population were considered for inclusion. No limitations were placed on language or date of publication. Two reviewers independently screened the titles, abstracts, and full texts of identified articles. Data were extracted and synthesized using a convergent integrated approach. Studies were critically appraised using relevant JBI critical appraisal checklists.ResultsThirty-five studies were included in the review. Most studies were conducted in the United States (n = 21) and published between 2019 and 2022 (n = 24). Digital data capture was the most common LHS characteristic reported across studies, while patient engagement, aligned governance and a culture of rapid learning and improvement were reported least often. We identified 33 unique strategies for implementing LHSs including: change record systems, conduct local consensus discussions and audit & provide feedback. A triangulation of quantitative and qualitative data revealed three integrated findings related to the implementation of LHSs: (1) The digital infrastructure of LHSs optimizes health service delivery; (2) LHSs have a positive impact on patient care and health outcomes; and (3) LHSs can influence health care providers and the health system.ConclusionThis paper provides a comprehensive overview of the implementation of LHSs in various healthcare settings. While this review identified key implementation strategies, potential outcome measures, and components of functioning LHSs, further research is needed to better understand the impact of LHSs on patient, provider and population outcomes, and health system costs. Health systems researchers should continue to apply the LHS concept in practice, with a stronger focus on evaluation.
| S-EPMC10680228 | biostudies-literature
Project description:IntroductionThe exponential growth in health information technology (HIT) presents an immense opportunity for facilitating the data-to-knowledge-to-performance loop which supports learning health systems. This scoping review addresses the gap in knowledge around HIT implementation contextual factors such as organizational culture and provides a current state assessment.MethodsA search of 13 databases guided by Arskey and O'Malley's framework identified content on HIT implementations and organizational culture. The Consolidated Framework for Implementation Research (CFIR) was used to assess culture and to develop review criteria. Culture stress, culture effort, implementation climate, learning climate, readiness for implementation, leadership engagement, and available resources were the constructs examined. Rayyan and Qualtrics were used for screening and data extraction.ResultsFifty two studies included were mainly conducted in Academic Health Centers (n = 18, 35%) and at urban locations (n = 50, 96%). Interviews frequently used for data collection (n = 26, 50%) and guided by multiple frameworks (n = 34). Studies mostly focused on EHR implementations (n = 23, 44%) followed by clinical decision support (n = 9, 17%). About two-thirds (n = 34, 65%) reflected culture stress theme and 62% (21 of 34) acknowledged it as a barrier. Culture effort identified in 27 studies and was a facilitator in most (78%, 21 of 27). Leadership engagement theme in majority studies (71%, n = 37), with 35% (n = 13) noting it as a facilitator. Eighty percent (42 studies) noted available resources, 12 of which identified this as barrier to successful implementation.ConclusionsIt is vital to determine the culture and other CFIR inner setting constructs that are significant to HIT implementation as facilitators or barriers. This scoping review presents a limited number of empirical studies in this topic highlighting the need for additional research to quantify the effects of culture. This will help build evidence and best practices that facilitate HIT implementations and hence serve as a platform to support robust learning health systems.
| S-EPMC9284926 | biostudies-literature
Project description:Organizational climate is a key determinant of successful adoption of innovations; however, its relation to medical home implementation is unknown. This study examined the association between primary care providers' (PCPs') perception of organization climate and medical home implementation in the Veterans Health Administration. Multivariate regression was used to test the hypothesis that organizational climate predicts medical home implementation. This analysis of 191 PCPs found that higher scores in 2 domains of organizational climate (communication and cooperation, and orientation to quality improvement) were associated with a statistically significantly higher percentage (from 7 to 10 percentage points) of PCPs implementing structural changes to support the medical home model. In addition, some aspects of a better organizational climate were associated with improved organizational processes of care, including a higher percentage of patients contacted within 2 days of hospital discharge (by 2 to 3 percentage points) and appointments made within 3 days of a patient request (by 2 percentage points).
| S-EPMC6345270 | biostudies-literature
Project description:BACKGROUND:Implementation science is the study of strategies used to integrate evidence-based practices into real-world settings (Eccles and Mittman, Implement Sci. 1(1):1, 2006). Central to the identification of replicable, feasible, and effective implementation strategies is the ability to assess the impact of contextual constructs and intervention characteristics that may influence implementation, but several measurement issues make this work quite difficult. For instance, it is unclear which constructs have no measures and which measures have any evidence of psychometric properties like reliability and validity. As part of a larger set of studies to advance implementation science measurement (Lewis et al., Implement Sci. 10:102, 2015), we will complete systematic reviews of measures that map onto the Consolidated Framework for Implementation Research (Damschroder et al., Implement Sci. 4:50, 2009) and the Implementation Outcomes Framework (Proctor et al., Adm Policy Ment Health. 38(2):65-76, 2011), the protocol for which is described in this manuscript. METHODS:Our primary databases will be PubMed and Embase. Our search strings will be comprised of five levels: (1) the outcome or construct term; (2) terms for measure; (3) terms for evidence-based practice; (4) terms for implementation; and (5) terms for mental health. Two trained research specialists will independently review all titles and abstracts followed by full-text review for inclusion. The research specialists will then conduct measure-forward searches using the "cited by" function to identify all published empirical studies using each measure. The measure and associated publications will be compiled in a packet for data extraction. Data relevant to our Psychometric and Pragmatic Evidence Rating Scale (PAPERS) will be independently extracted and then rated using a worst score counts methodology reflecting "poor" to "excellent" evidence. DISCUSSION:We will build a centralized, accessible, searchable repository through which researchers, practitioners, and other stakeholders can identify psychometrically and pragmatically strong measures of implementation contexts, processes, and outcomes. By facilitating the employment of psychometrically and pragmatically strong measures identified through this systematic review, the repository would enhance the cumulativeness, reproducibility, and applicability of research findings in the rapidly growing field of implementation science.
| S-EPMC5918558 | biostudies-literature