Reactions to uncertainty and the accuracy of diagnostic mammography.
ABSTRACT: Reactions to uncertainty in clinical medicine can affect decision making.To assess the extent to which radiologists' reactions to uncertainty influence diagnostic mammography interpretation.Cross-sectional responses to a mailed survey assessed reactions to uncertainty using a well-validated instrument. Responses were linked to radiologists' diagnostic mammography interpretive performance obtained from three regional mammography registries.One hundred thirty-two radiologists from New Hampshire, Colorado, and Washington.Mean scores and either standard errors or confidence intervals were used to assess physicians' reactions to uncertainty. Multivariable logistic regression models were fit via generalized estimating equations to assess the impact of uncertainty on diagnostic mammography interpretive performance while adjusting for potential confounders.When examining radiologists' interpretation of additional diagnostic mammograms (those after screening mammograms that detected abnormalities), a 5-point increase in the reactions to uncertainty score was associated with a 17% higher odds of having a positive mammogram given cancer was diagnosed during follow-up (sensitivity), a 6% lower odds of a negative mammogram given no cancer (specificity), a 4% lower odds (not significant) of a cancer diagnosis given a positive mammogram (positive predictive value [PPV]), and a 5% higher odds of having a positive mammogram (abnormal interpretation).Mammograms interpreted by radiologists who have more discomfort with uncertainty have higher likelihood of being recalled.
Project description:Earlier studies of diagnostic mammography found wide unexplained variability in accuracy among radiologists. We assessed patient and radiologist characteristics associated with the interpretive performance of two types of diagnostic mammography.Radiologists interpreting mammograms in seven regions of the United States were invited to participate in a survey that collected information on their demographics, practice setting, breast imaging experience, and self-reported interpretive volume. Survey data from 244 radiologists were linked to data on 274,401 diagnostic mammograms performed for additional evaluation of a recent abnormal screening mammogram or to evaluate a breast problem, between 1998 and 2008. These data were also linked to patients' risk factors and follow-up data on breast cancer. We measured interpretive performance by false-positive rate, sensitivity, and AUC. Using logistic regression, we evaluated patient and radiologist characteristics associated with false-positive rate and sensitivity for each diagnostic mammogram type.Mammograms performed for additional evaluation of a recent mammogram had an overall false-positive rate of 11.9%, sensitivity of 90.2%, and AUC of 0.894; examinations done to evaluate a breast problem had an overall false-positive rate of 7.6%, sensitivity of 83.9%, and AUC of 0.871. Multiple patient characteristics were associated with measures of interpretive performance, and radiologist academic affiliation was associated with higher sensitivity for both indications for diagnostic mammograms.These results indicate the potential for improved radiologist training, using evaluation of their own performance relative to best practices, and for improved clinical outcomes with health care system changes to maximize access to diagnostic mammography interpretation in academic settings.
Project description:<h4>Purpose</h4>To investigate the association between radiologist interpretive volume and diagnostic mammography performance in community-based settings.<h4>Materials and methods</h4>This study received institutional review board approval and was HIPAA compliant. A total of 117,136 diagnostic mammograms that were interpreted by 107 radiologists between 2002 and 2006 in the Breast Cancer Surveillance Consortium were included. Logistic regression analysis was used to estimate the adjusted effect on sensitivity and the rates of false-positive findings and cancer detection of four volume measures: annual diagnostic volume, screening volume, total volume, and diagnostic focus (percentage of total volume that is diagnostic). Analyses were stratified by the indication for imaging: additional imaging after screening mammography or evaluation of a breast concern or problem.<h4>Results</h4>Diagnostic volume was associated with sensitivity; the odds of a true-positive finding rose until a diagnostic volume of 1000 mammograms was reached; thereafter, they either leveled off (P < .001 for additional imaging) or decreased (P = .049 for breast concerns or problems) with further volume increases. Diagnostic focus was associated with false-positive rate; the odds of a false-positive finding increased until a diagnostic focus of 20% was reached and decreased thereafter (P < .024 for additional imaging and P < .001 for breast concerns or problems with no self-reported lump). Neither total volume nor screening volume was consistently associated with diagnostic performance.<h4>Conclusion</h4>Interpretive volume and diagnostic performance have complex multifaceted relationships. Our results suggest that diagnostic interpretive volume is a key determinant in the development of thresholds for considering a diagnostic mammogram to be abnormal. Current volume regulations do not distinguish between screening and diagnostic mammography, and doing so would likely be challenging.
Project description:When making decisions under uncertainty, people in all walks of life, including highly trained medical professionals, tend to resort to using 'mental shortcuts', or heuristics. Anchoring-and-adjustment (AAA) is a well-known heuristic in which subjects reach a judgment by starting from an initial internal judgment ('anchored position') based on available external information ('anchoring information') and adjusting it until they are satisfied. We studied the effects of the AAA heuristic during diagnostic decision-making in mammography. We provided practicing radiologists (<i>N</i> = 27 across two studies) a random number that we told them was the estimate of a previous radiologist of the probability that a mammogram they were about to see was positive for breast cancer. We then showed them the actual mammogram. We found that the radiologists' own estimates of cancer in the mammogram reflected the random information they were provided and ignored the actual evidence in the mammogram. However, when the heuristic information was not provided, the same radiologists detected breast cancer in the same set of mammograms highly accurately, indicating that the effect was solely attributable to the availability of heuristic information. Thus, the effects of the AAA heuristic can sometimes be so strong as to override the actual clinical evidence in diagnostic tasks.
Project description:To examine whether U.S. radiologists' interpretive volume affects their screening mammography performance.Annual interpretive volume measures (total, screening, diagnostic, and screening focus [ratio of screening to diagnostic mammograms]) were collected for 120 radiologists in the Breast Cancer Surveillance Consortium (BCSC) who interpreted 783?965 screening mammograms from 2002 to 2006. Volume measures in 1 year were examined by using multivariate logistic regression relative to screening sensitivity, false-positive rates, and cancer detection rate the next year. BCSC registries and the Statistical Coordinating Center received institutional review board approval for active or passive consenting processes and a Federal Certificate of Confidentiality and other protections for participating women, physicians, and facilities. All procedures were compliant with the terms of the Health Insurance Portability and Accountability Act.Mean sensitivity was 85.2% (95% confidence interval [CI]: 83.7%, 86.6%) and was significantly lower for radiologists with a greater screening focus (P = .023) but did not significantly differ by total (P = .47), screening (P = .33), or diagnostic (P = .23) volume. The mean false-positive rate was 9.1% (95% CI: 8.1%, 10.1%), with rates significantly higher for radiologists who had the lowest total (P = .008) and screening (P = .015) volumes. Radiologists with low diagnostic volume (P = .004 and P = .008) and a greater screening focus (P = .003 and P = .002) had significantly lower false-positive and cancer detection rates, respectively. Median invasive tumor size and proportion of cancers detected at early stages did not vary by volume.Increasing minimum interpretive volume requirements in the United States while adding a minimal requirement for diagnostic interpretation could reduce the number of false-positive work-ups without hindering cancer detection. These results provide detailed associations between mammography volumes and performance for policymakers to consider along with workforce, practice organization, and access issues and radiologist experience when reevaluating requirements.
Project description:<h4>Purpose</h4>To examine radiologists' screening performance in relation to the number of diagnostic work-ups performed after abnormal findings are discovered at screening mammography by the same radiologist or by different radiologists.<h4>Materials and methods</h4>In an institutional review board-approved HIPAA-compliant study, the authors linked 651 671 screening mammograms interpreted from 2002 to 2006 by 96 radiologists in the Breast Cancer Surveillance Consortium to cancer registries (standard of reference) to evaluate the performance of screening mammography (sensitivity, false-positive rate [ FPR false-positive rate ], and cancer detection rate [ CDR cancer detection rate ]). Logistic regression was used to assess the association between the volume of recalled screening mammograms ("own" mammograms, where the radiologist who interpreted the diagnostic image was the same radiologist who had interpreted the screening image, and "any" mammograms, where the radiologist who interpreted the diagnostic image may or may not have been the radiologist who interpreted the screening image) and screening performance and whether the association between total annual volume and performance differed according to the volume of diagnostic work-up.<h4>Results</h4>Annually, 38% of radiologists performed the diagnostic work-up for 25 or fewer of their own recalled screening mammograms, 24% performed the work-up for 0-50, and 39% performed the work-up for more than 50. For the work-up of recalled screening mammograms from any radiologist, 24% of radiologists performed the work-up for 0-50 mammograms, 32% performed the work-up for 51-125, and 44% performed the work-up for more than 125. With increasing numbers of radiologist work-ups for their own recalled mammograms, the sensitivity (P = .039), FPR false-positive rate (P = .004), and CDR cancer detection rate (P < .001) of screening mammography increased, yielding a stepped increase in women recalled per cancer detected from 17.4 for 25 or fewer mammograms to 24.6 for more than 50 mammograms. Increases in work-ups for any radiologist yielded significant increases in FPR false-positive rate (P = .011) and CDR cancer detection rate (P = .001) and a nonsignificant increase in sensitivity (P = .15). Radiologists with a lower annual volume of any work-ups had consistently lower FPR false-positive rate , sensitivity, and CDR cancer detection rate at all annual interpretive volumes.<h4>Conclusion</h4>These findings support the hypothesis that radiologists may improve their screening performance by performing the diagnostic work-up for their own recalled screening mammograms and directly receiving feedback afforded by means of the outcomes associated with their initial decision to recall. Arranging for radiologists to work up a minimum number of their own recalled cases could improve screening performance but would need systems to facilitate this workflow.
Project description:Digital mammography is the dominant modality for breast cancer screening in the United States. No previous studies have investigated as to how introducing digital mammography affects downstream breast-related care.Compare breast-related health care use after a screening mammogram before and after introduction of digital mammography.Longitudinal study of screening mammograms from 14 radiology facilities contributing data to the Breast Cancer Surveillance Consortium performed 1 year before and 4 years after each facility introduced digital mammography, along with linked Medicare claims. We included 30,211 mammograms for women aged 66 years and older without breast cancer.Rates of false-positive recall and short-interval follow-up were based on radiologists' assessments and recommendations; rates of follow-up mammography, ultrasound, and breast biopsy use were based on Medicare claims.False-positive recall rates increased after the introduction of digital mammography. Follow-up mammography use was significantly higher across all 4 years after a facility began using digital mammography compared with the year before [year 1 odds ratio (OR) = 1.7, 95% confidence interval (CI), 1.4-2.1]. Among women with false-positive mammography results, use of ultrasound decreased significantly in the second through fourth years after digital mammography began (year 2 OR = 0.4, 95% CI, 0.3-0.6).Introduction of a new technology led to changes in health care use that persisted for at least 4 years. Comparative effectiveness research on new technologies should consider not only diagnostic performance but also downstream utilization attributable to this apparent learning curve.
Project description:<h4>Background</h4>U.S. professional organizations have provided conflicting recommendations on annual vs. biennial mammography screening. Potential harms of more frequent screening include increased anxiety and costs of false positive results, including unnecessary breast biopsies and overdiagnosis.<h4>Objective</h4>To characterize current practices and beliefs surrounding mammography screening frequency and perspectives on using risk-based screening to inform screening intervals.<h4>Design</h4>Semi-structured interviews informed by the Consolidated Framework for Implementation Research (CFIR).<h4>Participants</h4>Patients, primary care providers (PCPs), third-party stakeholders (breast radiologists, radiology administrators, patient advocates).<h4>Main measures</h4>Qualitative data, with a codebook developed based upon prespecified implementation science constructs.<h4>Key results</h4>We interviewed 25 patients, 11 PCPs, and eight key stakeholders, including three radiologists, two radiology administrators, and three patient advocates. Most patients reported having annual mammograms, however, half believed having mammograms every two years was acceptable. Some women were worried early breast cancer would be missed if undergoing biennial screening. PCPs were equally split between recommending annual and biennial mammograms. Although PCPs were interested in using breast cancer risk models to inform screening decisions, concerns raised include time burden and lack of familiarity with breast cancer risk assessment tools. All breast radiologists believed patients should receive annual mammograms, while patient advocates and radiology administrators were split between annual vs. biennial. Radiologists were worried about missing breast cancer diagnoses when mammograms are not performed yearly. Patient advocates and radiology administrators were more open to biennial mammograms and utilizing risk-based screening.<h4>Conclusions</h4>Uncertainty remains across stakeholder groups regarding appropriate mammogram screening intervals. Radiologists recommend annual mammography, whereas patients and PCPs were evenly split between annual vs. biennial screening, although both favored annual screening among higher-risk women. Breast cancer risk assessment tools may help facilitate decisions about screening intervals, but face barriers to widespread implementation in the primary care setting. These results will inform future implementation strategies to adopt risk-stratified breast cancer screening.
Project description:After the US Food and Drug Administration (FDA) approved computer-aided detection (CAD) for mammography in 1998, and the Centers for Medicare and Medicaid Services (CMS) provided increased payment in 2002, CAD technology disseminated rapidly. Despite sparse evidence that CAD improves accuracy of mammographic interpretations and costs over $400 million a year, CAD is currently used for most screening mammograms in the United States.To measure performance of digital screening mammography with and without CAD in US community practice.We compared the accuracy of digital screening mammography interpreted with (n = 495 818) vs without (n = 129 807) CAD from 2003 through 2009 in 323 973 women. Mammograms were interpreted by 271 radiologists from 66 facilities in the Breast Cancer Surveillance Consortium. Linkage with tumor registries identified 3159 breast cancers in 323 973 women within 1 year of the screening.Mammography performance (sensitivity, specificity, and screen-detected and interval cancers per 1000 women) was modeled using logistic regression with radiologist-specific random effects to account for correlation among examinations interpreted by the same radiologist, adjusting for patient age, race/ethnicity, time since prior mammogram, examination year, and registry. Conditional logistic regression was used to compare performance among 107 radiologists who interpreted mammograms both with and without CAD.Screening performance was not improved with CAD on any metric assessed. Mammography sensitivity was 85.3% (95% CI, 83.6%-86.9%) with and 87.3% (95% CI, 84.5%-89.7%) without CAD. Specificity was 91.6% (95% CI, 91.0%-92.2%) with and 91.4% (95% CI, 90.6%-92.0%) without CAD. There was no difference in cancer detection rate (4.1 in 1000 women screened with and without CAD). Computer-aided detection did not improve intraradiologist performance. Sensitivity was significantly decreased for mammograms interpreted with vs without CAD in the subset of radiologists who interpreted both with and without CAD (odds ratio, 0.53; 95% CI, 0.29-0.97).Computer-aided detection does not improve diagnostic accuracy of mammography. These results suggest that insurers pay more for CAD with no established benefit to women.
Project description:To examine time trends in radiologists' interpretive performance at screening mammography between 1996 and 2004.All study procedures were institutional review board approved and HIPAA compliant. Data were collected on subsequent screening mammograms obtained from 1996 to 2004 in women aged 40-79 years who were followed up for 1 year for breast cancer. Recall rate, sensitivity, and specificity were examined annually. Generalized estimating equation (GEE) and random-effects models were used to test for linear trend. The area under the receiver operating characteristic curve (AUC), tumor histologic findings, and size of the largest dimension or diameter of the tumor were also examined.Data on 2,542,049 subsequent screening mammograms and 12,498 cancers diagnosed in the follow-up period were included in this study. Recall rate increased from 6.7% to 8.6%, sensitivity increased from 71.4% to 83.8%, and specificity decreased from 93.6% to 91.7%. In GEE models, adjusted odds ratios per calendar year were 1.04 (95% confidence interval [CI]: 1.02, 1.05) for recall rate, 1.09 (95% CI: 1.07. 1.12) for sensitivity, and 0.96 (95% CI: 0.95, 0.98) for specificity (P < .001 for all). Random-effects model results were similar. The AUC increased over time: 0.869 (95% CI: 0.861, 0.877) for 1996-1998, 0.884 (95% CI: 0.879, 0.890) for 1999-2001, and 0.891 (95% CI: 0.885, 0.896) for 2002-2004 (P < .001). Tumor histologic findings and size remained constant.Recall rate and sensitivity for screening mammograms increased, whereas specificity decreased from 1996 to 2004 among women with a prior mammogram. This trend remained after accounting for risk factors. The net effect was an improvement in overall discrimination, a measure of the probability that a mammogram with cancer in the follow-up period has a higher Breast Imaging Reporting and Data System assessment category than does a mammogram without cancer in the follow-up period.
Project description:<h4>Objective</h4>To investigate mammography facilities' follow-up times, population vulnerability, system-based processes, and association with cancer stage at diagnosis.<h4>Data sources</h4>Prospectively collected from San Francisco Mammography Registry (SFMR) 2005-2011, California Cancer Registry 2005-2012, SFMR facility survey 2012.<h4>Study design</h4>We examined time to biopsy for 17 750 abnormal mammogram results (BI-RADS 4/5), categorizing eight facilities as short or long follow-up based on proportion of mammograms with biopsy at 30 days. We examined facility population vulnerability (race/ethnicity, language, education), and system processes. Among women with a cancer diagnosis, we modeled odds of advanced-stage (?IIb) cancer diagnosis by facility follow-up group.<h4>Data extraction methods</h4>Merged SFMR, Cancer Registry and facility survey data.<h4>Principal findings</h4>Facilities (N = 4) with short follow-up completed biopsies by 30 days for 82% of mammograms compared with 62% for facilities with long follow-up (N = 4) (P < 0.0001). All facilities serving high proportions of vulnerable women were long follow-up facilities. The long follow-up facilities had fewer radiologists, longer biopsy appointment wait times, and less communication directly with women. Having the index abnormal mammogram at a long follow-up facility was associated with higher adjusted odds of advanced-stage cancer (OR 1.45; 95% CI 1.10-1.91).<h4>Conclusions</h4>Providing mammography facilities serving vulnerable women with appropriate resources may decrease disparities in abnormal mammogram follow-up and cancer diagnosis stage.