Assessing diagnostic tests: how to correct for the combined effects of interpretation and reference standard.
ABSTRACT: We describe a general solution to the problem of determining diagnostic accuracy without the use of a perfect reference standard and in the presence of interpreter variability. The accuracy of a diagnostic test is typically determined by comparing its outcomes with those of an established reference standard. But the accuracy of the standard itself and those of the interpreters strongly influence such assessments. We use our solution to examine the effects of the properties of the standard, the reliability of the interpreters, and the prevalence of abnormality on the measured sensitivity and specificity. Our results provide a method of systematically adjusting the measured sensitivity and specificity in order to estimate their true values. The results are validated by simulations and their detailed application to specific cases are described.
Project description:BACKGROUND:Facilitating access to professional interpretation services is key to equitable hospital care for migrants with limited language proficiency; however, interpreter underuse has been documented. The factors that potentially enable or hinder professional interpreter use are not well understood. We aimed to compare perceptions held by hospital managers and healthcare practitioners of the factors influencing the use of remote video interpretation and in-person interpretation. METHODS:This study employed a retrospective qualitative design. Two hospitals, located in Austria and Norway, with adequately similar baseline characteristics were purposively selected. Both hospitals used in-person interpreters, and the Austrian hospital had recently introduced remote video interpretation as an alternative and supplement. Fifteen managers and healthcare practitioners participated in focus groups and individual interviews. Data were thematically analysed with the aid of behavioural system theory. RESULTS:Across sites, the facilitators of interpreter use included individual factors (knowledge about interpreter services, skills to assess when/how to use an interpreter, beliefs about favourable consequences), as well as organisational factors (soft budget constraints). Barriers were identified at the individual level (lack of interpersonal skills to handle difficult provider-interpreter situations, lack of skills to persuade patients to accept interpreter use, lack of trust in service professionalism), and at the organisational level (limited interpreter availability, time constraints). The introduction of remote video interpretation services seemed to counteract the organisational barriers. Video interpretation was further perceived to enable patient confidentiality, which was regarded as a facilitator. However, video interpretation introduced specific barriers, including perceived communication deficiencies. CONCLUSION:This study has identified a range of factors that are perceived to influence the use of interpreters in hospitals. The research suggests that-implementing remote video interpretation services lessens the barriers to use and that such services should be introduced in hospital settings as an alternative or supplement to in-person interpreters. Further intervention functions should be considered to bring about change in the use of interpretation services, including developing guidelines for interpreter use, educating staff in the appropriate use of video technology, and training staff in communicating with interpreter and patients with limited language proficiency.
Project description:This literature review examined the effects of patients' limited English proficiency and use of professional and ad hoc interpreters on the quality of psychiatric care.PubMed, PsycINFO, and CINAHL (Cumulative Index to Nursing and Allied Health Literature) were systematically searched for English-language publications from inception of each database to April 2009. Reference lists were reviewed, and expert sources were consulted. Among the 321 articles identified, 26 met inclusion criteria: peer-reviewed articles reporting primary data on clinical care for psychiatric disorders among patients with limited proficiency in English or in the provider's language.Evaluation in a patient's nonprimary language can lead to incomplete or distorted mental status assessment. Although both untrained and trained interpreters may make errors, untrained interpreters' errors may have greater clinical impact, compromising diagnostic accuracy and clinicians' detection of disordered thought or delusional content. Use of professional interpreters may improve disclosure in patient-provider communications, referral to specialty care, and patient satisfaction.Little systematic research has addressed the impact of language proficiency or interpreter use on the quality of psychiatric care in contemporary U.S. settings. Findings are insufficient to inform evidence-based guidelines for improving quality of care among patients with limited English proficiency. Clinicians should be aware of the ways in which quality of care can be compromised when they evaluate patients in a nonprimary language or use an interpreter. Given U.S. demographic trends, future research should help guide practice and policy by addressing deficits in the evidence base.
Project description:Language barriers are a large and growing problem for patients in the US and around the world. Interpreter services are a standard solution for addressing language barriers and most research has focused on utilization of interpreter services and their effect on health outcomes for patients who do not speak the same language as their healthcare providers including nurses. However, there is limited research on patients' perceptions of these interpreter services.To examine Hmong- and Spanish-speaking patients' perceptions of interpreter service quality in the context of receiving cancer preventive services.Twenty limited English proficient Hmong (n=10) and Spanish-speaking participants (n=10) ranging in age from 33 to 75 years were interviewed by two bilingual researchers in a Midwestern state. Interviews were audio taped, transcribed verbatim, and translated into English. Analysis was done using conventional content analysis.The two groups shared perceptions about the quality of interpreter services as variable along three dimensions. Specifically, both groups evaluated quality of interpreters based on the interpreters' ability to provide: (a) literal interpretation, (b) cultural interpretation, and (c) emotional interpretation during the health care encounter. The groups differed, however, on how they described the consequences of poor interpretation quality. Hmong participants described how poor quality interpretation could lead to: (a) poor interpersonal relationships among patients, providers, and interpreters, (b) inability of patients to follow through with treatment plans, and (c) emotional distress for patients.Our study highlights the fact that patients are discerning consumers of interpreter services; and could be effective partners in efforts to reform and enhance interpreter services.
Project description:<h4>Background</h4>Electrocardiogram (ECG) with preparticipation evaluation (PPE) for athletes remains controversial in the United States and diagnostic accuracy of clinician ECG interpretation is unclear. This study aimed to assess reliability and validity of clinician ECG interpretation using expert-validated ECGs according to the 2010 European Society of Cardiology (ESC) interpretation criteria.<h4>Methods</h4>This is a blinded, prospective study of diagnostic accuracy of clinician ECG interpretation. Anonymized ECGs were validated for normal and abnormal patterns by blinded expert interpreters according to the ESC interpretation criteria from October 2011 through March 2012. Six pairs of clinician interpreters were recruited from relevant clinical specialties in an academic medical center in March 2012. Each clinician interpreted 85 ECGs according to the ESC interpretation guidelines. Cohen and Fleiss' kappa, sensitivity, and specificity were calculated within specialties and across primary care and cardiology specialty groups.<h4>Results</h4>Experts interpreted 189 ECGs yielding a kappa of 0.63, demonstrating "substantial" inter-rater agreement. A total of 85 validated ECGs, including 26 abnormals, were selected for clinician interpretation. The kappa across cardiology specialists was "substantial" and "moderate" across primary care (0.69 vs 0.52, respectively, P < 0.001). Sensitivity and specificity to detect abnormal patterns were similar between cardiology and primary care groups (sensitivity 93.3% vs 81.3%, respectively, P = 0.31; specificity 88.8% vs 89.8%, respectively, P = 0.91).<h4>Conclusions</h4>Clinician ECG interpretation according to the ESC interpretation criteria appears to demonstrate limited reliability and validity. Before widespread adoption of ECG for PPE of U.S. athletes, further research of training focused on improved reliability and validity of clinician ECG interpretation is warranted.
Project description:BACKGROUND: Interpreter services for medical care increase physician-patient communication and safety, yet a "formal certification" process to demonstrate interpreter competence does not exist. Testing and training is left to individual health care facilities nationwide. Bilingual staff are often used to interpret, without any assessment of their skills. Assessing interpreters' linguistic competence and setting standards for testing is a priority. OBJECTIVE: To assess dual-role staff interpreter linguistic competence in an integrated health care system to determine skill qualification to work as medical interpreters. DESIGN: Dual-role staff interpreters voluntarily completed a linguistic competency assessment using a test developed by a language school to measure comprehension, completeness, and vocabulary through written and oral assessment in English and the second language. Pass levels were predetermined by school as not passing, basic (limited ability to read, write, and speak English and the second language) and medical interpreter level. Five staff-interpreter focus groups discussed experiences as interpreters and with language test. RESULTS: A total of 840 dual-role staff interpreters were tested for Spanish (75%), Chinese (12%), and Russian (5%) language competence. Most dual-role interpreters serve as administrative assistants (39%), medical assistants (27%), and clinical staff (17%). Two percent did not pass, 21% passed at basic level, 77% passed at medical interpreter level. Staff that passed at the basic level was prone to interpretation errors, including omissions and word confusion. Focus groups revealed acceptance of exam process and feelings of increased validation in interpreter role. CONCLUSIONS: We found that about 1 in 5 dual-role staff interpreters at a large health care organization had insufficient bilingual skills to serve as interpreters in a medical encounter. Health care organizations that depend on dual-role staff interpreters should consider assessing staff English and second language skills.
Project description:BACKGROUND: The population of persons seeking medical care is linguistically diverse in the United States. Language barriers can adversely affect a patient's ability to explain their symptoms. Among hospitalized patients, these barriers may lead to higher readmission rates and longer hospitalizations. Trained interpreters help overcome communication barriers; however, interpreter usage among patients is suboptimal. OBJECTIVE: To investigate differences among patients with limited English proficiency (LEP) in their length of stay (LOS) and 30-day readmission rate associated with their receiving professional interpretation at admission or discharge. DESIGN: We analyzed the rates of interpretation at admission and discharge of all LEP patients admitted to a tertiary care hospital over a three-year period. We calculated length of stay in days and as log of LOS. We also examined 30-day readmission. Using multivariable regression models, we explored differences among patients who received interpretation at admission, discharge, or both, controlling for patient characteristics, including age, illness severity, language, and gender. PARTICIPANTS: All LEP patients admitted between May 1, 2004 and April 30, 2007. MAIN MEASURES: Length of hospital stay as related to use of professional interpreters; readmission to the hospital within 30 days. KEY RESULTS: Of the 3071 patients included in the study, 39 % received language interpretation on both admission and discharge date. Patients who did not receive professional interpretation at admission or both admission/discharge had an increase in their LOS of between 0.75 and 1.47 days, compared to patients who had an interpreter on both day of admission and discharge (P<0.02). Patients receiving interpretation at admission and/or discharge were less likely than patients receiving no interpretation to be readmitted with 30 days. CONCLUSIONS: The length of a hospital stay for LEP patients was significantly longer when professional interpreters were not used at admission or both admission/discharge.
Project description:Little is known about how cancer physicians communicate with limited English proficient (LEP) patients. We studied physician-reported use and availability of interpreters.A 2004 survey was fielded among physicians identified by a population-based sample of breast cancer patients. Three hundred and forty-eight physicians completed mailed surveys (response rate: 77 percent) regarding the structure and organization of care.We used logistic regression to analyze use and availability of interpreters.Most physicians reported treating LEP patients. Among physicians using interpreters within the last 12 months, 42 percent reported using trained medical interpreters, 21 percent telephone interpreter services, and 75 percent reported using untrained interpreters to communicate with LEP patients. Only one-third of physicians reported good availability of trained medical interpreters or telephone interpreter services when needed. Compared with HMO physicians, physicians in solo practice and single-specialty medical groups were less likely to report using trained medical interpreters or telephone interpreter services, and they were less likely to report good availability of these services.There were important practice setting differences predicting use and availability of trained medical interpreters and telephone interpretation services. These findings may have troubling implications for effective physician-patient communication critically needed during cancer treatment.
Project description:<h4>Objective</h4>To determine if professional medical interpreters have a positive impact on clinical care for limited English proficiency (LEP) patients.<h4>Data sources</h4>A systematic literature search, limited to the English language, in PubMed and PsycINFO for publications between 1966 and September 2005, and a search of the Cochrane Library.<h4>Study design</h4>Any peer-reviewed article which compared at least two language groups, and contained data about professional medical interpreters and addressed communication (errors and comprehension), utilization, clinical outcomes, or satisfaction were included. Of 3,698 references, 28 were found by multiple reviewers to meet inclusion criteria and, of these, 21 assessed professional interpreters separately from ad hoc interpreters. Data were abstracted from each article by two reviewers. Data were collected on the study design, size, comparison groups, analytic technique, interpreter training, and method of determining the participants' need for an interpreter. Each study was evaluated for the effect of interpreter use on four clinical topics that were most likely to either impact or reflect disparities in health and health care.<h4>Principal findings</h4>In all four areas examined, use of professional interpreters is associated with improved clinical care more than is use of ad hoc interpreters, and professional interpreters appear to raise the quality of clinical care for LEP patients to approach or equal that for patients without language barriers.<h4>Conclusions</h4>Published studies report positive benefits of professional interpreters on communication (errors and comprehension), utilization, clinical outcomes and satisfaction with care.
Project description:BACKGROUND: In our ever-increasingly multicultural, multilingual society, medical interpreters serve an important role in the provision of care. Though it is known that using untrained interpreters leads to decreased quality of care for limited English proficiency patients, because of a short supply of professionals and a lack of formalized, feasible education programs for volunteers, community health centers and internal medicine practices continue to rely on untrained interpreters. OBJECTIVE: To develop and formally evaluate a novel medical interpreter education program that encompasses major tenets of interpretation, tailored to the needs of volunteer medical interpreters. DESIGN: One-armed, quasi-experimental retro-pre-post study using survey ratings and feedback correlated by assessment scores to determine educational intervention effects. PARTICIPANTS: Thirty-eight students; 24 Spanish, nine Mandarin, and five Vietnamese. The majority had prior interpreting experience but no formal medical interpreter training. OUTCOME MEASURES: Students completed retrospective pre-test and post-test surveys measuring confidence in and perceived knowledge of key skills of interpretation. Primary outcome measures were a 10-point Likert scale for survey questions of knowledge, skills, and confidence, written and oral assessments of interpreter skills, and qualitative evidence of newfound knowledge in written reflections. RESULTS: Analyses showed a statistically significant (P <0.001) change of about two points in mean self-ratings on knowledge, skills, and confidence, with large effect sizes (d?>?0.8). The second half of the program was also quantitatively and qualitatively shown to be a vital learning experience, resulting in 18 % more students passing the oral assessments; a 19 % increase in mean scores for written assessments; and a newfound understanding of interpreter roles and ways to navigate them. CONCLUSIONS: This innovative program was successful in increasing volunteer interpreters' skills and knowledge of interpretation, as well as confidence in own abilities. Additionally, the program effectively taught how to navigate the roles of the interpreter to maintain clear communication.
Project description:OBJECTIVE:Assess effects of a bedside interpreter-phone intervention on hospital discharge preparedness among patients with limited English proficiency (LEP). METHODS:Mixed-methods study compared patient-reported discharge preparedness and knowledge of medications and follow-up appointments among 189 Chinese- and Spanish-speakers before (n=94) and after (n=95) bedside interpreter-phone implementation, and examined nurse and resident-physician interpreter-phone utilization through focus groups. RESULTS:Pre-post discharge preparedness (Care Transitions Measure mean 77.2 vs. 78.5; p=0.62) and patient-reported knowledge of follow-up appointments, discharge medication administration and side effects did not differ significantly. Pre-post knowledge of medication purpose increased in bivariate (88% vs. 97%, p=0.02) and propensity score adjusted analyses [aOR (adjusted odds ratio), 4.49; 95% CI, 1.09-18.4]. Nurses and physicians reported using interpreter-phones infrequently for discharge communication, preferring in-person interpreters for complex discharges and direct communication with family for routine discharges. Post-implementation patients reported continued use of ad-hoc family interpreters (43%) or no interpretation at all (22%). CONCLUSION:Implementation of a bedside interpreter-phone systems intervention did not consistently improve patient-reported measures of discharge preparedness, possibly due to limited uptake during discharges. PRACTICE IMPLICATIONS:Hospital systems must better understand clinician preferences for discharge communication to successfully increase professional interpretation and shift culture away from using family members as interpreters.