The Influence of Respondent Characteristics on the Validity of Self-Reported Survey Responses.
Ontology highlight
ABSTRACT: Objective
To examine concordance between member self-reports and the organization's administrative claims data for two key health factors: number of chronic conditions, and number of prescription drugs.Data
Medicare Advantage plan claims data and member survey data from 2011 to 2012.Design
Mailed surveys to 15,000 members, enrolled minimum 6 months, drawn from a random sample of primary care physician practices with at least 200 members.Methods
Descriptive statistics were generated for extent of concordance. Multivariable logistic regressions were used to analyze the association of selected respondent characteristics with likelihood of concordance.Findings
Concordance for number of chronic conditions was 58.4 percent, with 27.3 percent under-reporting, 14.2 percent over-reporting. Concordance for number of prescription drugs was 56.6 percent with 38.9 percent under-reporting, 4.5 percent over-reporting. Number of prescriptions and assistance in survey completion were associated with higher likelihood of concordance for chronic conditions. Assistance in survey completion and number of chronic conditions were associated with higher concordance, and age and number of prescriptions were associated with lower concordance, for prescription drugs.Conclusions
Self-reported number of chronic conditions and prescription medications are not in high concordance with claims data. Health care researchers and policy makers using patient self-reported data should be aware of these potential biases.
SUBMITTER: Guerard B
PROVIDER: S-EPMC4874833 | biostudies-literature | 2016 Jun
REPOSITORIES: biostudies-literature
Publications
The Influence of Respondent Characteristics on the Validity of Self-Reported Survey Responses.
Guerard Barbara B Omachonu Vincent V Harvey Raymond A RA Hernandez S Robert SR Sen Bisakha B
Health services research 20150915 3
<h4>Objective</h4>To examine concordance between member self-reports and the organization's administrative claims data for two key health factors: number of chronic conditions, and number of prescription drugs.<h4>Data</h4>Medicare Advantage plan claims data and member survey data from 2011 to 2012.<h4>Design</h4>Mailed surveys to 15,000 members, enrolled minimum 6 months, drawn from a random sample of primary care physician practices with at least 200 members.<h4>Methods</h4>Descriptive statist ...[more]
Similar Datasets
Project description:BackgroundMost older adults prefer to continue living at home despite increasing care needs and demand for services. To aid in maintaining independence, integrated care models for community-dwelling older people are promoted as the most cost-effective approach. The implementation of such care models is challenging and often the end-users are not involved or their needs are not considered. We conducted a population survey in order to understand the needs and preferences of home-dwelling older adults living in Canton Basel-Landschaft, Switzerland. The aims of this paper are to chronicle the development of the INSPIRE Population Survey, outline its variables and measurements, describe the marketing strategy utilized for survey dissemination and report on the response rate and respondent characteristics.MethodsThe INSPIRE Population Survey, conducted between March and August 2019, is a cross-sectional survey of older adults aged 75 and older living at home in Canton Basel-Landschaft. The questionnaire was developed by expert input and stakeholder involvement. Its readability and acceptability were pilot-tested with older people. To ensure the likelihood of a high and representative response rate, a meticulous step-by-step marketing strategy was developed prior to the dissemination of the questionnaire.ResultsThe overall response rate was 30.7% (n = 8,846), with variations between 20.6 and 34.5% across the different care regions in the canton. A generally higher response rate was found in the care regions with a higher density and which bordered the urban city of Basel. We received support from local stakeholders, policy makers and media through using a broad combination of marketing channels and targeting our community partners who have a strong relationship with our target audience.ConclusionsAlthough recruiting older adults in research is challenging, our study shows that a high response rate can be achieved by developing the survey through expert input and by involving all important stakeholders, including older adults, throughout the entire process.
| S-EPMC8223353 | biostudies-literature
Project description:BackgroundAs technology continues to shape the landscape of health research, the utilization of web-based surveys for collecting sexual health information among adolescents and young adults has become increasingly prevalent. However, this shift toward digital platforms brings forth a new set of challenges, particularly the infiltration of automated bots that can compromise data integrity and the reliability of survey results.ObjectiveWe aimed to outline the data verification process used in our study design, which employed survey programming and data cleaning protocols.MethodsA 26-item survey was developed and programmed with several data integrity functions, including reCAPTCHA scores, RelevantID fraud and duplicate scores, verification of IP addresses, and honeypot questions. Participants aged 15-24 years were recruited via social media advertisements over 7 weeks and received a US $15 incentive after survey completion. Data verification occurred through a 2-part cleaning process, which removed responses that were incomplete, flagged as spam by Qualtrics, or from duplicate IP addresses, or those that did not meet the inclusion criteria. Final comparisons of reported age with date of birth and reported state with state inclusion criteria were performed. Participants who completed the study survey were linked to a second survey to receive their incentive. Responses without first and last names and full addresses were removed, as were those with duplicate IP addresses or the exact same longitude and latitude coordinates. Finally, IP addresses used to complete both surveys were compared, and consistent responses were eligible for an incentive.ResultsOver 7 weeks, online advertisements for a web-based survey reached 1.4 million social media users. Of the 20,585 survey responses received, 4589 (22.3%) were verified. Incentives were sent to 462 participants; of these, 14 responses were duplicates and 3 contained discrepancies, resulting in a final sample of 445 responses.ConclusionsConfidential web-based surveys are an appealing method for reaching populations-particularly adolescents and young adults, who may be reluctant to disclose sensitive information to family, friends, or clinical providers. Web-based surveys are a useful tool for researchers targeting hard-to-reach populations due to the difficulty in obtaining a representative sample. However, researchers face the ongoing threat of bots and fraudulent participants in a technology-driven world, necessitating the adoption of evolving bot detection software and tailored protocols for data collection in unique contexts.
| S-EPMC11648336 | biostudies-literature
Project description:ObjectiveThe aim of this research was to evaluate the validity of the questions developed by Thai periodontists on self-reported periodontal status and symptoms in identifying severe periodontitis amongst adults in Thailand.MethodsRegistered Electricity Generating Authority of Thailand (EGAT) employees completed medical examinations and full-mouth periodontal examinations. They also were interviewed using a self-reported questionnaire that was developed by Thai periodontists. The questions pertained to their periodontal status and symptoms comprising swollen gums, bad breath, loose teeth, bleeding on brushing, painful gums, and pus or abscesses. The participants were categorised as having nonsevere and severe periodontitis according to the Centers for Disease Control and Prevention in partnership with the American Academy of Periodontology (CDC/AAP) case definitions. Self-reported periodontal status and symptom results were compared with actual periodontal status. The area under the receiver operating characteristic curve (AUROCC), sensitivity, specificity, positive predictive value, negative predictive value, and accuracy were calculated.ResultsA total of 1393 EGAT employees participated in this study. The questions on self-reported periodontal status and all symptoms poorly identified patients with severe periodontitis, with an AUROCC of 0.52 to 0.60. The sensitivity was 5.0% to 40.2%; however, the specificity was good, at 73.2% to 99.1%. The validity of the self-reported questions was comparable amongst the sex, education, and income subgroups.ConclusionsThis study demonstrated that self-reported periodontal status and symptoms were inadequate in identifying patients with severe periodontal disease.
| S-EPMC9875259 | biostudies-literature
Project description:ObjectiveTo explore sampling bias as a result of survey format selection by examining associations between characteristics of people aging with long-term physical disability (PAwLTPD) and their preferences for phone or web-based survey format.DesignA cross-sectional study using a secondary data analysis approach.SettingData were from an ongoing longitudinal cohort study conducted in the community.ParticipantsConvenience sampling was used. PAwLTPD who participated in year 2 of the longitudinal cohort study were included. Inclusion criteria were age 45-65 years, English speaking, and self-reported onset of a physical disability at least 5 years prior to study recruitment. Two participants completed the survey using both phone and web formats and were thus excluded; 387 participants (N=387) were included in the analysis.InterventionsNot applicable.Main outcome measuresChoice of survey format and demographics (age, sex, race and ethnicity, marital status, living arrangement, socioeconomic status) were collected in addition to self-rated physical health.ResultsParticipants were on average 58.2±5.6 years old. A total of 33% were male, and 62% were White. Approximately 40% of participants completed phone surveys. The phone survey group was significantly older (t=-4.76, P<.001) and had lower education (U=11133, z=-6.65, P<.001) and lower self-rated physical health (U=15420, z=-2.38, P=.017) than the web survey group. Participants who were White (χ2=60.69; df=1; P<.001; odds ratio [OR], 0.18) or were in a long-term relationship were less likely to choose phone surveys (χ2=42.20; df=1; P<.001; OR, 0.21). Those who earned $10,008 or less annually (χ2=53.90; df=1; P<.001; OR, 5.22) or who lived alone (χ2=36.26; df=1; P<.001; OR, 3.64) were more likely to choose phone surveys. Participants with paid work (χ2=16.81, df=1, P<.001) tended to select web-based surveys, while those on disability leave (χ2=9.61, df=1, P<.01) were more likely to choose phone surveys.ConclusionsSociodemographics are associated with survey format choice in PAwLTPD. Findings largely support the existing understanding of digital literacy but also provide insight into the potential occurrence of sampling bias when multiple survey format options are not offered. These findings have implications for investigators who aim to reach a more representative sample of people with disabilities.
| S-EPMC8904875 | biostudies-literature
Project description:Self-reported measures of health, in the context of developed countries, are well-researched and commonly regarded as reliable predictors of the underlying health of the population. However, the validity of these measures is under-researched and questionable in the context of low- and middle-income countries. The authors used Longitudinal Ageing Study in India (LASI) survey data from India to compare self-reported hypertension with biometrically-measured hypertension. The results are reported in terms of sensitivity, specificity, and kappa as a measure of agreement. Logistic regression was undertaken to examine the characteristics of those who were unaware of their hypertensive status. Our analysis showed a low sensitivity of 56% and a high specificity of 90.5%. Agreement between self-reported data and biometric measurement of hypertension was observed to be moderate (κ = 0.48). Large variations were observed among states and sub-groups. The odds of false negative reporting of hypertension were lower in the individuals with higher age, high education, and greater wealth status. The authors conclude that self-reported hypertension has important limitations and may be a source of systematic bias. It is recommended that planning and policy-making in India be based more on an objective assessment of hypertension.
| S-EPMC9659862 | biostudies-literature
Project description:Although current guidelines emphasize the importance of cholesterol knowledge, little is known about accuracy of this knowledge, factors affecting accuracy, and the relationship of self-reported cholesterol with cardiovascular disease (CVD).The 39,876 female health professionals with no prior CVD in the Women's Health Study were asked to provide self-reported and measured levels of total and high-density lipoprotein (HDL) cholesterol. Demographic and cardiovascular risk factors were considered as determinants of awareness and accuracy. Accuracy was evaluated by the difference between reported and measured cholesterol. In addition, we examined the relationship of self-reported cholesterol with incident CVD over 10 years.Compared with women who were unaware of their cholesterol levels, aware women (84%) had higher levels of income, education, and exercise and were more likely to be married, normal in weight, treated for hypertension and hypercholesterolemia, nonsmokers, moderate drinkers, and users of hormone therapy. Women underestimated their total cholesterol by 9.7 mg/dL (95% CI: 9.2-10.2); covariates explained little of this difference (R (2) < .01). Higher levels of self-reported cholesterol were strongly associated with increased risk of CVD, which occurred in 741 women (hazard ratio 1.23/40 mg/dL cholesterol, 95% CI: 1.15-1.33). Women with elevated cholesterol who were unaware of their level had particularly increased risk (HR=1.88, P <. 001) relative to aware women with normal measured cholesterol.Women with obesity, smoking, untreated hypertension, or sedentary lifestyle have decreased awareness of their cholesterol levels. Self-reported cholesterol underestimates measured values, but is strongly related to CVD. Lack of awareness of elevated cholesterol is associated with increased risk of CVD.
| S-EPMC1852917 | biostudies-other
Project description:BackgroundLack of physicians' knowledge regarding mental health, including Autism Spectrum Disorder (ASD) could have adverse effects on affected individuals' health and quality of life. The purpose of this study was to provide construct validity for a modified questionnaire in order to determine the self-reported competency for underlying sub-constructs in ASD, make inferences on perceived competence in ASD based on a sample of Romanian physicians, and identify physicians' characteristics associated with these sub-domains of competency.MethodsFor this survey, we modified a questionnaire that was used in Pakistan and Turkey, and administered it to a sample of 383 practicing physicians in Romania to assess their perceived competency regarding ASD. Exploratory factor analysis on 12 knowledge questions revealed five sub-domains: stigma, potential causes, children's behavior, misconceptions, and educational needs associated with ASD knowledge. Using General Linear Models, we determined physicians' characteristics that predict the total competency score and various competency sub-scores.ResultsSeventy-five percent of the responding physicians were female and 30% had over 30 years practicing medicine. The majority (73-94%) of physicians have correctly responded to some basic questions regarding knowledge about ASD. We also found that younger physicians were more knowledgeable about potential causes of ASD than older physicians (Adjusted Mean Score (AMS): 2.90 vs. 2.18, P < 0.01), while older physicians knew more about the behavior of children with ASD (AMS: 0.64 vs. 0.37, P = 0.02). We found a significant interaction (P < 0.01) between television as source of ASD knowledge and city where the clinic is located in relation to knowledge of the physicians regarding stigma related to ASD. However, the total score was not associated with the variables associated with sub-domains.ConclusionUsing factor analysis, we demonstrated construct validity of five sub-domains related to Romanian physicians' knowledge about ASD that include stigma, potential causes, behavior in ASD children, special education needs, and misconceptions related to ASD. The lack of significant association of the knowledge of physicians on ASD neither with the Psychiatry nor the Pediatric ward rotations at medical school may support the need for improving the curriculum on ASD in Romanian medical schools.
| S-EPMC8605522 | biostudies-literature
Project description:Chronic rhinosinusitis (CRS) is a frequently occurring chronic respiratory disease. There is evidence that effective treatment of CRS can improve patients' quality of life, but the data regarding the extent to which CRS impairs patients' quality of life (QoL) is sparse. This study aimed to evaluate the effect of self-reported CRS on health-related QoL and to determine whether the influence was associated with gender, age and socio-economic status. A four-stage random sampling method was used to select the participants from the general population in Guangzhou, China. All participants were interviewed face-to-face at their homes using a standardized questionnaire. The health-related QoL of each participant was assessed using the SF-36 Health Survey. The scores of the SF-36 after adjusting for gender, age, socioeconomic conditions, smoking and some important comorbid conditions were compared between the CRS group and the non-CRS group using analysis of covariance. A multiple linear regression model with interaction terms was established to determine whether CRS affected QoL to the same degree across the different subpopulations. Among a total of 1,411 participants aged at least 15 years, 118 persons (8.4%) had self-reported CRS. Subjects with CRS had an increased prevalence of allergic rhinitis, chronic obstructive pulmonary disease and gout than subjects without CRS. The CRS group had lower scores in all eight domains and the physical and mental component summary than those without CRS (P<0.05), and the greatest differences were in role emotional function (RE), general health (GH) and role physical function (RP). The impairments of the CRS participants in RE and RP were greater among the females than the males. Moreover, physical domains were affected to greater degrees among the elderly and those with high-level education. In conclusion, CRS is a common chronic disorder. Persons with self-reported CRS perceived themselves as having impaired QoL in both the physical and mental domains. These findings shed new light on the health burden of CRS and should be taken into account by clinicians involved in the care of CRS patients.
| S-EPMC4433264 | biostudies-literature
Project description:IntroductionThis study assesses validity of self-report for the use of major classes of illicit drugs and opioid-maintenance therapy among pregnant women at a substance abuse treatment program.MethodsAnalyses used data collected from 83 pregnant women in a prospective cohort study at the University of New Mexico. Study participants with a history of substance abuse were screened and, if eligible, enrolled during an early prenatal care visit. A follow-up interview was conducted shortly after delivery. Self-reported information about drug use later in pregnancy was compared with urine drug screen (UDS) results collected during the third trimester. Simple kappa (k) and prevalence-and-bias-adjusted kappa (PABAK) coefficients were calculated as the measures of agreement. Sensitivity and specificity of self-report for each drug class were estimated using UDS as the 'gold standard'.ResultsThe sample included a large proportion of ethnic minority (80% Hispanic/Latina and 7% American Indian) and socially disadvantaged (50% less than high school education and 94% Medicaid-insured) pregnant women. On average, patients had 4.8 ± 3.0 urine drug screens during the third trimester. Sensitivity of self-report was low (<60%) for all classes of illicit drugs; however, marijuana and opioids demonstrated slightly higher sensitivity (57.9 and 58.3%, respectively) than other classes (<47%).ConclusionsThis study found substantial underreporting for all classes of illicit drugs among pregnant women in a substance abuse treatment program. Rates of underreporting are expected to be higher among the general population of pregnant women.
| S-EPMC4713294 | biostudies-literature
Project description:PurposeThe objective of this study is to determine whether summary estimates of a self-report physical activity questionnaire that does not specifically assess frequency or duration (the Coronary Artery Risk Development in Young Adults (CARDIA) physical activity history (PAH)) differs from the summary estimates of one that does (CARDIA Supplemental Questionnaire).MethodsAfter the year 25 examination (2010-2011), 203 CARDIA black and white men and women (age 50.3 ± 3.6 yr) at the Oakland, CA, site participated in this comparison study. The between-questionnaire association and agreement were determined for continuous and categorical estimates on the basis of 1) quartiles and 2) meeting 2008 physical activity guidelines. Differences in participant characteristics by concordance/discordance status were also examined. Finally, receiver operating characteristic curves were computed to determine the accuracy of the PAH compared with the supplemental questionnaire.ResultsReported physical activity levels were high and varied significantly by race and sex (all P < 0.01). Between-questionnaire estimates were significantly correlated (rho = 0.75 to 0.90, all P < 0.001) and had high agreement (κ = 0.51 to 0.80) across all race/sex groups. A higher proportion of women than men were classified as concordant by quartile of vigorous intensity (P = 0.001), but no other participant characteristics were associated with concordant/discordant quartile ranking. Participants classified as concordant on the basis of physical activity guidelines had lower body mass index than those classified as discordant (both P < 0.05). The area under the curve was 0.95, suggesting that the PAH has high accuracy for classifying individuals as meeting physical activity guidelines.ConclusionsAlthough it is inconvenient that the PAH is not expressed in more standard units, these findings support the practice of not directly assessing frequency and duration, which are frequent sources of reporting error.
| S-EPMC5842916 | biostudies-literature