Identifying patterns of item missing survey data using latent groups: an observational study.
ABSTRACT: OBJECTIVES:To examine whether respondents to a survey of health and physical activity and potential determinants could be grouped according to the questions they missed, known as 'item missing'. DESIGN:Observational study of longitudinal data. SETTING:Residents of Brisbane, Australia. PARTICIPANTS:6901 people aged 40-65 years in 2007. MATERIALS AND METHODS:We used a latent class model with a mixture of multinomial distributions and chose the number of classes using the Bayesian information criterion. We used logistic regression to examine if participants' characteristics were associated with their modal latent class. We used logistic regression to examine whether the amount of item missing in a survey predicted wave missing in the following survey. RESULTS:Four per cent of participants missed almost one-fifth of the questions, and this group missed more questions in the middle of the survey. Eighty-three per cent of participants completed almost every question, but had a relatively high missing probability for a question on sleep time, a question which had an inconsistent presentation compared with the rest of the survey. Participants who completed almost every question were generally younger and more educated. Participants who completed more questions were less likely to miss the next longitudinal wave. CONCLUSIONS:Examining patterns in item missing data has improved our understanding of how missing data were generated and has informed future survey design to help reduce missing data.
Project description:OBJECTIVE:To test the feasibility of a framework for prioritizing new comparative effectiveness research (CER) questions related to management of primary open-angle glaucoma (POAG) using practice guidelines and a survey of clinicians. DESIGN:Cross-sectional survey. PARTICIPANTS:Members of the American Glaucoma Society (AGS). METHODS:We restated as an answerable clinical question each recommendation in the 2005 American Academy of Ophthalmology Preferred Practice Patterns (PPPs) regarding the management of POAG. We asked members of the AGS to rank the importance of each clinical question, on a scale of 0 (not important at all) to 10 (very important), using a 2-round Delphi survey conducted online between April and September 2008. Respondents had the option of selecting "no judgment" or "research has already answered this question" to each question in lieu of the 0 to 10 rating. We used the ratings assigned by the Delphi respondents to determine the importance of each clinical question. MAIN OUTCOME MEASURES:Ranking of importance of each clinical question. RESULTS:We derived 45 clinical questions from the POAG PPPs. Of the 620 AGS members invited to participate in the survey, 169 completed the Round 1 survey; 105 of 169 also completed Round 2. We observed 4 response patterns to the individual questions. Nine clinical questions were ranked as the most important: 4 questions on medical intervention, 4 questions on filtering surgery, and 1 question on adjustment of therapy. CONCLUSIONS:Our theoretical model for priority setting for CER questions is a feasible and pragmatic approach that merits testing in other medical settings.
Project description:OBJECTIVES:The primary aim was to investigate the impact of complaints on doctors' psychological welfare and health. The secondary aim was to assess whether doctors report exposure to a complaints process is associated with defensive medical practise. DESIGN:This was a cross-sectional anonymous survey study. Participants were stratified into recent/current, past, no complaints. Each group completed tailored versions of the survey. PARTICIPANTS:95?636 doctors were invited to participate. A total of 10?930(11.4%) responded, 7926 (8.3%) completed the full survey and were included in the complete analysis. MAIN OUTCOME MEASURES:Anxiety and depression were assessed using the standardised Generalised Anxiety Disorder scale and Physical Health Questionnaire. Defensive practise was evaluated using a new measure. Single-item questions measured stress-related illnesses, complaints-related experience, attitudes towards complaints and views on improving complaints processes. RESULTS:16.9% of doctors with current/recent complaints reported moderate/severe depression (relative risk (RR) 1.77 (95% CI 1.48 to 2.13) compared to doctors with no complaints (9.5%)). Fifteen per cent reported moderate/severe anxiety (RR=2.08 (95% CI 1.61 to 2.68) compared to doctors with no complaints (7.3%)). Distress increased with complaint severity, with highest levels after General Medical Council (GMC) referral (26.3% depression, 22.3% anxiety). Doctors with current/recent complaints were 2.08 (95% CI 1.61 to 2.68) times more likely to report thoughts of self-harm or suicidal ideation. Most doctors reported defensive practise: 82-89% hedging and 46-50% avoidance. Twenty per cent felt victimised after whistleblowing, 38% felt bullied, 27% spent over 1?month off work. Over 80% felt processes would improve with transparency, managerial competence, capacity to claim lost earnings and action against vexatious complainants. CONCLUSIONS:Doctors with recent/current complaints have significant risks of moderate/severe depression, anxiety and suicidal ideation. Morbidity was greatest in cases involving the GMC. Most doctors reported practising defensively, including avoidance of procedures and high-risk patients. Many felt victimised as whistleblowers or reported bullying. Suggestions to improve complaints processes included transparency and managerial competence.
Project description:Background:The perception of colostomy-related problems and their impact on health-related quality of life (QoL) may differ between patients and healthcare professionals. The aim of this study was to investigate this using the Colostomy Impact Score (CIS) tool. Methods:Healthcare professionals including consultant colorectal surgeons, stoma nurses, ward nurses, trainees and medical students were recruited. An online survey was designed. From the 17 items used to develop the CIS, participants chose the seven factors they thought to confer the strongest negative impact on the QoL of patients with a colostomy. They were then asked to rank the 12 responses made by patients to the final seven factors contained in the CIS. Results were compared with the original patient rankings at the time of development of the CIS. Results:A total of 156 healthcare professionals (50·4 per cent of the pooled professionals) from 17 countries completed the survey. Of the original seven items in the CIS, six were above the threshold for random selection. Ranking the responses, a poor match between participants and the original score was detected for 49·7 per cent of the professionals. The most under-rated item originally present in the CIS was stool consistency, reported by 47 of the 156 professionals (30·1 per cent), whereas frequency of changing the stoma bag was the item not included in the CIS that was chosen most often by professionals (124, 79·5 per cent). Significant differences were not observed between different groups of professionals. Conclusion:The perspective of colostomy-related problems differs between patients with a colostomy and healthcare professionals.
Project description:Suicide is a leading cause of death worldwide. Although research has made strides in better defining suicidal behaviors, there has been less focus on accurate measurement. Currently, the widespread use of self-report, single-item questions to assess suicide ideation, plans and attempts may contribute to measurement problems and misclassification. We examined the validity of single-item measurement and the potential for statistical errors. Over 1,500 participants completed an online survey containing single-item questions regarding a history of suicidal behaviors, followed by questions with more precise language, multiple response options and narrative responses to examine the validity of single-item questions. We also conducted simulations to test whether common statistical tests are robust against the degree of misclassification produced by the use of single-items. We found that 11.3% of participants that endorsed a single-item suicide attempt measure engaged in behavior that would not meet the standard definition of a suicide attempt. Similarly, 8.8% of those who endorsed a single-item measure of suicide ideation endorsed thoughts that would not meet standard definitions of suicide ideation. Statistical simulations revealed that this level of misclassification substantially decreases statistical power and increases the likelihood of false conclusions from statistical tests. Providing a wider range of response options for each item reduced the misclassification rate by approximately half. Overall, the use of single-item, self-report questions to assess the presence of suicidal behaviors leads to misclassification, increasing the likelihood of statistical decision errors. Improving the measurement of suicidal behaviors is critical to increase understanding and prevention of suicide.
Project description:OBJECTIVES:We investigate whether meta-data, specifically duration of responses to smartphone-delivered surveys, is correlated to elevated scores on the depression assessment PHQ-9 as well as the specific item around self-harm (item 9). METHODS:In this observational study, we recruited 92 smartphone-owning adults (??18) with schizophrenia (45) and healthy controls (43). We installed an open-source smartphone app called mindLAMP to collect survey results and latencies (response times) over a period of 3 months. Surveys were scheduled for twice a week, but participants were instructed to take the surveys naturally as much or as little as they wanted. A total of 1,218 PHQ-9 surveys were completed across all participants over 3 months. RESULTS:A total of 75 participants (39 with schizophrenia and 36 healthy controls) completed both the initial visit and follow-up, as well as provided at least one self-reported PHQ-9 survey through the app. We found that depression symptom severity and response latencies were correlated for both individuals with schizophrenia (Spearman's ? = .22, p = .037) and healthy controls (Spearman's ? = .58, p?<?.001). Participants with schizophrenia scored higher (more severe) and took longer for every item of the PHQ-9 when compared to controls (p?<?.05 for each item). Item 9 response value and latency was slightly correlated for participants with schizophrenia (Spearman's ? = .086, p = .035) but was not significant for controls (Spearman's ? = .036, p = .37). CONCLUSIONS:Meta-data revealed group differences between individuals with schizophrenia and healthy controls based on individual depression symptoms completed on a smartphone. Correlation between suicide specific question latency and severity for participants with schizophrenia but not for controls indicates the clinical potential and need for further research.
Project description:OBJECTIVE:To propose a permutation-based approach of anchor item detection and evaluate differential item functioning (DIF) related to language of administration (English vs. Spanish) for 9 questions assessing patients' perceptions of their providers from the Consumer Assessment of Healthcare Providers and Systems (CAHPS) Medicare 2.0 survey. METHOD AND STUDY DESIGN: CAHPS 2.0 health plan survey data collected from 703 Hispanics who completed the survey in Spanish were matched on personal characteristics to 703 Hispanics that completed the survey in English. Steps to be followed for the detection of anchor items using the permutation tests are proposed and these tests in conjunction with item response theory were used for the identification of anchor items and DIF detection. RESULTS:Of the questions studied, 4 were selected as anchor items and 3 of the remaining questions were found to have DIF (P < 0.05). The 3 questions with DIF asked about seeing the doctor within 15 minutes of the appointment time, respect for what patients had to say, and provider spending enough time with patients. CONCLUSIONS:Failure to account for language differences in CAHPS survey items may result in misleading conclusions about disparities in health care experiences between Spanish and English speakers. Statistical adjustments are needed when using the items with DIF.
Project description:BACKGROUND:The widespread availability and cost-effectiveness of new-wave software-based audience response systems (ARSs) have expanded the possibilities of collecting health data from hard-to-reach populations, including youth. However, with all survey methods, biases in the data may exist because of participant nonresponse. OBJECTIVE:The aims of this study were to (1) examine the extent to which an ARS could be used to gather health information from youths within a large-group school setting and (2) examine individual- and survey-level response biases stemming from this Web-based data collection method. METHODS:We used an ARS to deliver a mental health survey to 3418 youths in 4 high schools in the Midwestern United States. The survey contained demographic questions, depression, anxiety, and suicidality screeners, and questions about their use of offline resources (eg, parents, peers, and counselors) and Web-based resources (ie, telemental health technologies) when they faced stressful life situations. We then examined the response rates for each survey item, focusing on the individual- and survey-level characteristics that related to nonresponse. RESULTS:Overall, 25.39% (868/3418) of youths answered all 38 survey questions; however, missingness analyses showed that there were some survey structure factors that led to higher rates of nonresponse (eg, questions at the end of survey, sensitive questions, and questions for which precise answers were difficult to provide). There were also some personal characteristics that were associated with nonresponse (eg, not identifying as either male or female, nonwhite ethnicity, and higher levels of depression). Specifically, a multivariate model showed that male students and students who reported their gender as other had significantly higher numbers of missed items compared with female students (B=.30 and B=.47, respectively, P<.001). Similarly, nonwhite race (B=.39, P<.001) and higher depression scores (B=.39, P<.001) were positively related to the number of missing survey responses. CONCLUSIONS:Although our methodology-focused study showed that it is possible to gather sensitive mental health data from youths in large groups using ARSs, we also suggest that these nonresponse patterns need to be considered and controlled for when using ARSs for gathering population health data.
Project description:BACKGROUND: The objective of this study was to test the data quality, test-retest reliability and hospital-level reliability of the Patient-Reported Incident in Hospital Instrument (PRIH-I). METHODS: 13 incident questions were included in a national patient-experience survey in Norway during the spring of 2011. All questions and a composite incident index were assessed by calculating missing-item rates, test-retest reliability and hospital-level reliability. A multivariate linear regression on a global item regarding incorrect treatment was used to assess the main sources of variation in patient-perceived incorrect treatment at hospitals. RESULTS: Five of the 13 patient-incident questions had a missing-item rate of >20%. Only one item met the criterion of 0.7 for test-retest reliability (wrong or delayed diagnosis), seven items had a score of >0.5, while the remainder had a reliability score of <0.5. However, the reliability was >0.7 for six of 10 items tested at the hospital level, and >0.6 for the remaining four items. A patient-incident index based on 12 of the incident items had no missing data, the test-retest reliability was 0.6 and the hospital-level reliability was 0.85. CONCLUSIONS: The PRIH-I comprises 13 questions about patient-perceived incidents in hospitals, and can be easily and cost-effectively included in national patient-experience surveys with an acceptable increase in respondent burden. Although the missing-item rate and test-retest reliability were poor for several items, the hospital-level reliability was satisfactory for most of the items. The incident items contribute to a patient-reported incident index, with excellent data quality and hospital-level reliability.
Project description:BACKGROUND:Routine population-based screening for depression is an essential part of evolving health care models integrating care for mental health in primary care. Depression instruments often include questions about suicidal thoughts, but how patients experience these questions in primary care is not known and may have implications for accurate identification of patients at risk. OBJECTIVES:To explore the patient experience of routine population-based depression screening/assessment followed, for some, by suicide risk assessment and discussions with providers. DESIGN:Qualitative, interview-based study. PARTICIPANTS:Thirty-seven patients from Kaiser Permanente Washington who had recently screened positive for depression on the 2-item Patient Health Questionnaire [PHQ] and completed the full PHQ-9. APPROACH:Criterion sampling identified patients who had recently completed the PHQ-9 ninth question which asks about the frequency of thoughts about self-harm. Patients completed semi-structured interviews by phone, which were recorded and transcribed. Directive and conventional content analyses were used to apply knowledge from prior research and elucidate new information from interviews; thematic analysis was used to organize key content overall and across groups based on endorsement of suicide ideation. KEY RESULTS:Four main organizing themes emerged from analyses: (1) Participants believed being asked about suicidality was contextually appropriate and valuable, (2) some participants described a mismatch between their lived experience and the PHQ-9 ninth question, (3) suicidality disclosures involved weighing hope for help against fears of negative consequences, and (4) provider relationships and acts of listening and caring facilitated discussions about suicidality. CONCLUSIONS:All participants believed being asked questions about suicidal thoughts was appropriate, though some who disclosed suicidal thoughts described experiencing stigma and sometimes distanced themselves from suicidality. Direct communication with trusted providers, who listened and expressed empathy, bolstered comfort with disclosure. Future research should consider strategies for reducing stigma and encouraging fearless disclosure among primary care patients experiencing suicidality.
Project description:To compare text messaging and face-to-face interviews to conduct a survey on childhood diarrhoea and pneumonia.Caregivers of young children able to send text messages in Zhao County in rural China were included in this crossover study. Villages (clusters) were randomized into two groups using the ratio 1:1.6 to account for an expected higher drop-out in group 2. In group 1, participants first completed the face-to-face and then text messaging survey; this order was reversed in group 2. We determined data equivalence of 17 questions that were answered by participants who were the same person in both surveys. For the text messaging survey, we assessed the overall and item response rate.?=?0.004). Kappas were substantial for six (0.61-0.80), moderate for two (0.58 and 0.60), and fair for three questions (0.31, 0.35 and 0.37). The proportion of agreement was >90% for five questions; 80.0%-90.0% for five questions; 70.0%, 65.0% and 45.5%. The remaining questions had too small numbers to calculate these values.This study shows that text messaging data collection produces data similar to data from face-to-face interviews in a middle-income setting, but the response rate was insufficient for use in public health surveys. Improving the response rate is important, because text message surveys could be of greater value in rural remote areas due to the cost-saving potential.